AATP Service not starting after an upgrade

[kengaul]

Got updated to sensor version 2.114.8037.53586 today and all sensors services are failing to start.

(old version was 2.114.8034.28632 so not a huge leap)

Event log says "Service cannot be started. The service process could not connect to the service controller"

Logs have a NullReferenceException on DirectoryServicesResolver+<CreateSecurityPrincipalAsync>d__108

Not sure if the service controller is external or internal?

Original install had the proxy config on the command line, could it be that has been lost during the upgrade - clearly there have been many upgrades since I originally installed it that have gone without issue.

Any bright ideas?

[#2]

@kengaul , known issue, we are working on a fix right now.

[#3]

Hi yes we had the same issue last night when all the sensors were upgraded to 2.114.8037.53586 at 6:00pm GMT they were failing to start. It looks like 5 hours later there was a second update to 2.114.8044.7220 at 11:00pm GMT that fixed the issue and we saw the sensors coming back online.

Additionally on the 18th again we lost sensors for all our servers roughly 6:00pm as well. The odd thing was even the ATP portal was offline I was getting a error 500 the page wouldn't load.

I understand that there are works and maintenance that happen, however I cannot find anything in the Health Centre, web documentation or Twitter notifications that state there was any issues with ATP either on the 18th or 19th. Is there anything we can reference?

[#4]

@TlTUS, you can subscribe to get push emails on live site issues with AATP here:
https://health.atp.azure.com/

the issues you mention were posted there as well:
https://health.atp.azure.com/incidents/ztbt75s0kbtp
https://health.atp.azure.com/incidents/jps3jcf9nnsd

As for the portal Error 500, this is not something we are aware off, and we didn't get any other complaints on such a problem.
Can you tell the exact UTC times when it happened and when it resolved so I can try to track back what happened?
Also, you can share privately your workspace ID and I will try to check if there was a specific issue with it...

I do recommend that if you encounter such issue and it's not documented on the health page I shared above to immediately open a support case.
We do have the portal auto monitored, but I didn't see any alert on our systems on portal availability  on that day...