SCCM 2403 KB29166583 Hotfix

[edd080]

Hello to all after installing the KB29166583 hotfix I am having issues imaging any device through OSD; the device is able to pxe boot and I also arrive up to the point where i can select an image to install however when choosing any image I get the below error:

And the following is the smstslog :

The adk is also updated to version 10.1.26100.1 which is the latest version available.

Is anyone having these problems? Anything I can try to maybe be able to get the deployments up and running again?

I am at a loss.

Thank you for any help.

[#2]

Update:

It seems that the update has been pulled now from SCCM update (https://x.com/msconfigmgrteam/status/1831771857898369471?s=46&t=qYn7xwz1DMLHlQ-bpyx6bw)

Thank you for the lovely half baked update!

on another note if anyone is running into my same issues I followed this guide here and I am now able to image a test device, so far so good.

I have also learnt my lesson with MS updates from now on.

[#3]

The fix does not seems to work, did not for us, and not for the person in the link above:

Edit5: Microsoft confirmed the workaround is not working. Reinstalling the MP role does not resolve the issue either. Let´s see for further steps during the weekend. Restoring the server from backup from before the upgrade was mentioned, but this is our last option to consider. We delay this until after the weekend.

The only thing so far is to restart the sms agent host service on all MPs at an interval before to many connections build up and the system halt.

[#4]

@Shogo 

Yep just seeing the updated thread, thank you; and yes the console does not even load for me; seeing the multiple (thousands) connections to the sql server by simply doing a netstat -an command.

The fix was good while it lasted.

Have to wait for MS to get this fixed anyway I suppose.

[#5]

Its a further bad sign for the former best and innovative MS Onpremise Product. First no more innovations and then updates with no quality check at all, which are destroying Prod Environments.
Sorry MS (and the product team) im always fair but you have to correct your course or send a clear message to your customers that you have given up the product already.

[#6]

It seems that an update has been posted in the Reddit thread:

"Edit6: The temporary fix is to revert the LocationMgr.dll file in the management point installation folder(s). Either from an backup or receiving the file from Microsoft. They are working on an re-release of the patch. The registry keys are still in place at the moment but I think they are not required. With the next update they will anyway be removed if the MP role re installs."

I backed up the original dll file and replaced with a backed up one a few days before this update was released, I am seeing far less connections to the SQL server, seems to have stabilized, but as always this is not the official fix so there is always some risk, however so far so good. Console is responsive and imaging is back up and running.

[#7]

@edd080 

I opened a ticket with MS and this fixed it for us:

add the below reg keys on all the Management Points and restart (SMS Agent Host on the MPs ) after adding them.

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\MP

disableExtendedValidations = 1 (REG_DWORD)

disableRequestValidations =1 (REG_DWORD)

[#8]

In my initial posts i posted a link to Reddit showing this exact workaround, it worked, then stopped working after some hours. The sql connections started to pile up between the management point and sql server up to a point that not even the sccm console was responsive .
I applied the dll fix, and it is now working once again, the sql connections decreased considerably.
The same Reddit thread i linked also confirms that the registry workaround does not work anymore. So you might want to monitor the server from time to time.

[#9]

For now we have activated a script that is scheduled to run every 10th minute, restarts the sms agent host service on each MP if connections is over 10000 to avoid system halt, i have to evaluate if this limit is enough, we have two MPs so then it can be a total of max 20000 connections.

‎EDIT: Sep 06 2024
After some days with the above workaround, the services on MPs did restart between every 30-120 minutes, causing some disturbance.
We have now changed to the "replace with old dll"-workaround, and it seems fine, at least after one day, still no official fix from Microsoft.

[#10]

Thank you 👍🏻 will certainly keep your script handy in case this dll workaround does not work.

[#11]

@edd080 

Yep, spoke too soon. Sporadic imaging issues. Asking Microsoft about the DLL recommendation.

Thanks 

[#12]

In my opinion, It clearly shows that this update was not even tested before being released publicly, otherwise it would not have been released.
Up till now the dll fix is the only way the server is stable, at least in our situation.

[#13]

Super nice of them to silently release something so sensitive. I luckily installed it the day it was released after finally updating to 2403 because I was expecting a different hotfix in the console, now I can't even get a hold of support to get the dll and my backup is already overwritten with the broken dll. Wonder how many weeks my organization going to be stuck sitting on their hands?

[#14]

CPU usage has increased 25% over the past 4 days after applying this hotfix. After reboot, CPU back to pre-update levels. Single VM serves a low number of clients (<100) probably saved me.

[#15]

did you apply any workaround? registry fix or the dll replacement? our server is still fine up till now.

[#16]

Has anyone got any feedback about this?

Nearly a week has passed and it seems that there is no official fix from MS for this, yet.

[#17]

@edd080 so far only got the regkey suggestion. Still waiting on support to get back to me with the locationmgr.dll to try. 7 days with an unusable sccm environment. My team is not happy.

[#18]

What a total mess. Thanks for taking time to reply. 👍🏻

[#19]

Same issue here, after installing the update, sccm is bombing our SQL Server.

[#20]

The only way this solved it for us was using the dll workaround.