SOLVED

CMG broken after Upgrade to MECM 2002 (RESOLVED)

%3CLINGO-SUB%20id%3D%22lingo-sub-1476364%22%20slang%3D%22en-US%22%3ERe%3A%20CMG%20broken%20after%20Upgrade%20to%20MECM%202002%20(Resolved)%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1476364%22%20slang%3D%22en-US%22%3E%3CP%3EFor%20anyone%20with%20this%20same%20issue%2C%20the%20fix%20for%20me%20was%20to%20remove%20the%20%22Cloud%20Management%20Gateway%20connection%20point%20role%22%20from%20our%20onprem%20site%20server%20and%20then%20add%20it%20back.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3CP%3Eadministration%20-%20Site%20configuration%20-%20Servers%20and%20Site%20System%20Roles%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1472925%22%20slang%3D%22en-US%22%3ECMG%20broken%20after%20Upgrade%20to%20MECM%202002%20(RESOLVED)%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1472925%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%3C%2FP%3E%3CP%3Eeverything%20else%20with%20the%202002%20upgrade%20appears%20to%20be%20fine%20however%20our%20CMG%20is%20now%20broken.%20There%20were%20no%20issues%20with%20it%20prior%20to%20the%20upgrade.%3C%2FP%3E%3CP%3EAll%20CMG%20clients%20show%20as%20%22Device%20online%20from%20internet%20-%20True%22%2C%20%22Device%20Online%20Status%20-%20Offline%22%3C%2FP%3E%3CP%3Ewe%20have%20around%20200%20clients%20regularly%20connected%20via%20CMG.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECMG%20is%20listed%20with%20a%20status%20of%20%22Ready%22%20and%20status%20description%20of%20%22Configuration%20update%20completed%22%2C%20however%20if%20I%20run%20the%20connection%20analyser%20it%20fails%20on%20the%20last%20step%20%22Testing%20the%20CMG%20channel%20for%20Management%20point%3AOURInternalMECMServer.fqdn%22%3C%2FP%3E%3CP%3EThe%20description%20is%3C%2FP%3E%3CP%3E%22Failed%20to%20get%20ConfigMgr%20token%20with%20Azure%20AD%20token.%20Status%20code%20is%20'500'%20and%20status%20description%20is%20'CMGService_No_Connector'.%3CBR%20%2F%3EA%20possible%20reason%20for%20this%20failure%20is%20the%20CMG%20service%20failed%20to%20forward%20the%20message%20to%20the%20CMG%20connection%20point.%20There%20is%20no%20CMG%20connection%20point%20that%20is%20connecting%20to%20the%20CMG%20service.%20For%20more%20information%2C%20see%20the%20SMS_CLOUD_PROXYCONNECTOR.log%20on%20the%20CMG%20connection%20point.%22%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20have%20tried%20stopping%20and%20restarting%20the%20service.%20no%20change.%3C%2FP%3E%3CP%3ECloud%20engineers%20informed%20me%20that%20when%20I%20upgraded%20the%20azure%20resource%20was%20locked%20but%20only%20with%20do%20not%20delete%20lock%20so%20don't%20believe%20that%20should%20have%20been%20an%20issue%2C%20but%20had%20them%20remove%20the%20lock%20while%20I%20restarted%20the%20service%20to%20see%20if%20anything%20kicked%20in%2C%20nothing%20changed.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESMS_Cloud_Proxyconnector.log%20has%20clusters%20of%20these%20warnings%20starting%20when%20the%20upgrade%20happened.%3C%2FP%3E%3CP%3E%22ERROR%3A%20Failed%20to%20handle%20response%20from%20server.%20Cound%20be%20intermittent%20network%20issue.%20Exception%3A%20System.Net.WebException%3A%20The%20request%20was%20aborted%3A%20The%20request%20was%20canceled.~~%20at%20System.Net.HttpWebRequest.EndGetResponse(IAsyncResult%20asyncResult)~~%20at%20Microsoft.ConfigurationManager.CloudConnection.ProxyConnector.HttpConnection.HandleResponseCallback(IAsyncResult%20asynchronousResult%2C%20Int32%26amp%3B%20statusCode%2C%20Byte%5B%5D%26amp%3B%20payload)~~%20at%20Microsoft.ConfigurationManager.CloudConnection.ProxyConnector.ConnectionBase.ResponseCallBack(IAsyncResult%20asynchronousResult)%22%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECloudmgr.log%20is%20error%20free%2C%20everything%20in%20there%20looks%20functional.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECMG-ourcmgname-ProxyService_In_0-CMGService.log%20is%20full%20of%3C%2FP%3E%3CP%3E%22WARNING%3A%20There%20is%20no%20proxy%20connector%20connected%20to%20forward%20the%20message%22%3C%2FP%3E%3CP%3Eand%20the%20occasional%20message%20like%3C%2FP%3E%3CP%3E%22ERROR%3A%20The%20HTTP%20client%20203.24.71.17%3A41062%20sends%20invalid%20request.%20Exception%3A%20System.IO.InvalidDataException%3A%20Disconnected%20client%20sending%20message~~%20at%20Microsoft.ConfigurationManager.BgbServerChannel.BgbHttpListener.ReceiveMessage(String%20connectionId%2C%20String%20clientCert%2C%20BgbMessageType%20messageType%2C%20UInt16%20messageMajorVersion%2C%20UInt16%20messageMinorVersion%2C%20String%20header%2C%20String%20body%2C%20Byte%5B%5D%20bodyBuffer%2C%20HttpListenerContext%20context)~~%20at%20Microsoft.ConfigurationManager.BgbServerChannel.BgbHttpListener.listenerCallback(IAsyncResult%20result)%22%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EAre%20there%20changes%20required%20for%20the%20CMG%20after%20you%20upgrade%20to%202002%20if%20it%20was%20working%20prior%20to%20the%20upgrade%3F%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%3Eour%20machines%20are%20azure%20AD%20joined.%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%3Einternal%20machines%20are%20receiving%20he%20new%20client%2C%20however%20none%20on%20CMG%20have%20updated.%26nbsp%3B%20the%20client%20package%20shows%20under%20content%20for%20the%20CMG%20DP.%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%3EOur%20Cloud%20Engineers%20restarted%20CMG%2C%20no%20change.%26nbsp%3B%20%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EI'm%20running%20out%20of%20ideas%2C%20anybody%20have%20any%20thoughts%20on%20what%20I%20can%20do%20next%3F%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1472925%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3E(RES%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Contributor

Hi,

everything else with the 2002 upgrade appears to be fine however our CMG is now broken. There were no issues with it prior to the upgrade.

All CMG clients show as "Device online from internet - True", "Device Online Status - Offline"

we have around 200 clients regularly connected via CMG.  

 

CMG is listed with a status of "Ready" and status description of "Configuration update completed", however if I run the connection analyser it fails on the last step "Testing the CMG channel for Management point:OURInternalMECMServer.fqdn"

The description is

"Failed to get ConfigMgr token with Azure AD token. Status code is '500' and status description is 'CMGService_No_Connector'.
A possible reason for this failure is the CMG service failed to forward the message to the CMG connection point. There is no CMG connection point that is connecting to the CMG service. For more information, see the SMS_CLOUD_PROXYCONNECTOR.log on the CMG connection point."

 

I have tried stopping and restarting the service. no change.

Cloud engineers informed me that when I upgraded the azure resource was locked but only with do not delete lock so don't believe that should have been an issue, but had them remove the lock while I restarted the service to see if anything kicked in, nothing changed.

 

SMS_Cloud_Proxyconnector.log has clusters of these warnings starting when the upgrade happened.

"ERROR: Failed to handle response from server. Cound be intermittent network issue. Exception: System.Net.WebException: The request was aborted: The request was canceled.~~ at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)~~ at Microsoft.ConfigurationManager.CloudConnection.ProxyConnector.HttpConnection.HandleResponseCallback(IAsyncResult asynchronousResult, Int32& statusCode, Byte[]& payload)~~ at Microsoft.ConfigurationManager.CloudConnection.ProxyConnector.ConnectionBase.ResponseCallBack(IAsyncResult asynchronousResult)"

 

Cloudmgr.log is error free, everything in there looks functional.

 

CMG-ourcmgname-ProxyService_In_0-CMGService.log is full of

"WARNING: There is no proxy connector connected to forward the message"

and the occasional message like

"ERROR: The HTTP client 203.24.71.17:41062 sends invalid request. Exception: System.IO.InvalidDataException: Disconnected client sending message~~ at Microsoft.ConfigurationManager.BgbServerChannel.BgbHttpListener.ReceiveMessage(String connectionId, String clientCert, BgbMessageType messageType, UInt16 messageMajorVersion, UInt16 messageMinorVersion, String header, String body, Byte[] bodyBuffer, HttpListenerContext context)~~ at Microsoft.ConfigurationManager.BgbServerChannel.BgbHttpListener.listenerCallback(IAsyncResult result)"

 

 

Are there changes required for the CMG after you upgrade to 2002 if it was working prior to the upgrade?

our machines are azure AD joined. 

internal machines are receiving he new client, however none on CMG have updated.  the client package shows under content for the CMG DP. 

Our Cloud Engineers restarted CMG, no change. 

 

I'm running out of ideas, anybody have any thoughts on what I can do next?

1 Reply
Best Response confirmed by PaulKlerkx (Contributor)
Solution

For anyone with this same issue, the fix for me was to remove the "Cloud Management Gateway connection point role" from our onprem site server and then add it back.  

administration - Site configuration - Servers and Site System Roles