Final Update: Monday, 10 May 2021 18:37 UTC
We've confirmed that all systems are back to normal with no customer impact as of 05/10, 18:15 UTC. Our logs show the incident started on 05/10, 10:30 UTC and that during the 7 hours 45 mins that it took to resolve the issue customers experienced errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions.
-Anupama
We've confirmed that all systems are back to normal with no customer impact as of 05/10, 18:15 UTC. Our logs show the incident started on 05/10, 10:30 UTC and that during the 7 hours 45 mins that it took to resolve the issue customers experienced errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions.
- Root Cause: The failure was due to configuration change in backend services which caused some permissions to drop from the service which is responsible for service to service communication.
- Incident Timeline: 7 Hours & 45 minutes - 05/10, 10:30 UTC through 05/10, 18:15 UTC
-Anupama
Update: Monday, 10 May 2021 17:32 UTC
Root cause has been isolated to a configuration change in backend services caused some permissions to drop from the service which is responsible for service to service communication. To address this issue we are restoring the required permissions and working on dependent services to refresh the token. Some customers may experience errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions and we estimate 2 hours before all the errors are addressed.
Root cause has been isolated to a configuration change in backend services caused some permissions to drop from the service which is responsible for service to service communication. To address this issue we are restoring the required permissions and working on dependent services to refresh the token. Some customers may experience errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions and we estimate 2 hours before all the errors are addressed.
- Work Around: None
- Next Update: Before 05/10 20:00 UTC
Update: Monday, 10 May 2021 12:41 UTC
We continue to investigate issues within Azure Monitor. Some customers continue to experience errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions.
We continue to investigate issues within Azure Monitor. Some customers continue to experience errors while using various features of Azure Monitor solutions including Azure Log Analytics, Service Map, VM Insights, Log Search Alerts, Scheduled Search Alerts, Application Insights, and Azure Sentinel in multiple regions.
initial findings indicate that the problem began at 05/10 ~10:30 UTC. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 05/10 17:00 UTC
Initial Update: Monday, 10 May 2021 11:28 UTC
We are aware of issues within Application Insights and are actively investigating. Some customers may experience data access issues and delayed or missed Log Search Alerts.
-Anmol
We are aware of issues within Application Insights and are actively investigating. Some customers may experience data access issues and delayed or missed Log Search Alerts.
- Work Around: None
- Next Update: Before 05/10 13:30 UTC
-Anmol
Updated May 10, 2021
Version 5.0
