Experiencing Data Access Issue in Azure portal for Log Analytics - 07/16 - Resolved

[Azure-Monitor-Team]

Final Update: Tuesday, 16 July 2019 17:55 UTC

We've confirmed that all systems are back to normal with no customer impact as of 7/16, 15:45 UTC. Our logs show the incident started on 7/15, 14:40 UTC and that during the of the incident retrying might have worked.

• Root Cause: Yet to be determined.
• Incident Timeline: 23 Hours & 5 minutes - 7/15, 14:40 UTC through 7/16, 15:45 UTC

We understand that customers rely on Azure Log Analytics as a critical service and apologize for any impact this incident caused.

-Dharmendra

---

Update: Tuesday, 16 July 2019 14:38 UTC

Root cause has been isolated to the same as described below. To address this issue we have carried out series of mitigation steps which has given us partial results.We are still working through mitigation plan to get the impacted cluster in South East Asia region back to normal operating levels. Some customers from Southeast asia region may experience query failures which accessing Data.

• Work Around: None
• Next Update: Before 07/16 19:00 UTC

-Durga

---

Update: Tuesday, 16 July 2019 10:09 UTC

Root cause has been isolated to configuration change which inadvertently caused performance degradation to 2 of the impacted backend clusters  . To address this issue we have rolled back the change and also performed additional mitigation. Southeast Australia region is now working as expected. Some customers may still experience query and data access issues with Southeast Asia region as we are working on mitigation in this region to stabilize impacted backend cluster.

• Work Around: None
• Next Update: Before 07/16 14:30 UTC

-Durga

---

Update: Tuesday, 16 July 2019 07:08 UTC

We continue to investigate issues within Log Analytics. Root cause is not fully understood at this time. Very small subset of customers continue to experience data access issue using Log Analytics in Southeast Asia and Southeast Australia region. We are working to establish the start time for the issue, initial findings indicate that the problem began at 07/15 ~21:51 UTC. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 07/16 10:30 UTC

-Rama

---

Update: Tuesday, 16 July 2019 02:52 UTC

We continue to investigate issues within Log Analytics. Root cause is not fully understood at this time. Some customers may continue to experience issue using Log Analytics in Southeast Asia and Australia Southeast. We are working to establish the start time for the issue. Initial findings indicate that the problem began at 07/15 21:51 UTC. We currently have no estimate for resolution.

• Work Around: None

• Next Update: Before 07/16 05:00 UTC

-Dharmendra

---

Update: Tuesday, 16 July 2019 01:18 UTC

We continue to investigate issues within Log Analytics. Root cause is not fully understood at this time. Some customers may continue to experience issue using Log Analytics in Southeast Asia and Australia Southeast. We are working to establish the start time for the issue. Initial findings indicate that the problem began at 07/15 21:51 UTC. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 07/16 02:30 UTC

-Dharmendra

---

Initial Update: Tuesday, 16 July 2019 00:01 UTC

We are aware of issues within Log Analytics and are actively investigating. Some customers may experience issue using Log Analytics in Southeast Asia and Australia Southeast .

• Work Around: None at the time of incident  
• Next Update: Before 07/16 01:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Dharmendra

---