Experiencing Data Access Issue in Azure portal for Log Analytics - 06/01 - Resolved

[Azure-Monitor-Team]

Final Update: Monday, 01 June 2020 22:43 UTC

We've confirmed that all systems are back to normal with no customer impact as of 6/1, 21:50 UTC. Our logs show the incident started on 6/1, 12:05 UTC and that during the 9 hours and 45 minutes that it took to resolve the issue 33% of customers experienced possible query execution errors, as well as the misbehavior or alerts configured in Log Search alerts, and Smart Detection alerts.

• Root Cause: The failure was due to a backend service getting into a bad state.
• Incident Timeline: 9 Hours & 45 minutes - 6/1, 12:05 UTC through 6/1, 21:50 UTC

We understand that customers rely on Azure Log Analytics as a critical service and apologize for any impact this incident caused.

-Jeff

---

Update: Monday, 01 June 2020 22:13 UTC

Root cause has been isolated to a backend service that is in a bad state which was impacting query execution, Log Search alerts, and Smart Detection alerts. To address this issue we updated backend service code and it is now working as expected. 

• Work Around: None
• Next Update: Before 06/01 23:30 UTC

-Jeff

---

Update: Monday, 01 June 2020 19:49 UTC

We continue to investigate issues within Log Analytics. Root cause has been narrowed to a backend service that is in a bad state. Some customers continue to experience timeouts and failures when running queries and issues with missed or delayed Log Search and Smart Detection alerts hosted in multiple region. Initial findings indicate that the problem began at 06/01 12:22 UTC. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 06/02 00:00 UTC

-Jeff

---

Update: Monday, 01 June 2020 16:37 UTC

We continue to investigate issues within Log Analytics. Root cause is not fully understood at this time. Customer using Azure Monitor who may have continue to experience timeouts and failures when running queries and issues with missed or delayed Log Search and Smart Detection alerts hosted in multiple region. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 06/01 19:00 UTC

-Anmol

---

Initial Update: Monday, 01 June 2020 14:29 UTC

We are aware of issues within Log Analytics and are actively investigating. Some customers may experience timeouts and failures when running queries and issues with missed or delayed Log Search and Smart Detection alerts hosted in East US region.

• Work Around: None
• Next Update: Before 06/01 16:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Anmol

---