Final Update: Wednesday, 18 August 2021 18:03 UTC
We've confirmed that all systems are back to normal as of 08/18, 17:30 UTC. Our logs show the incident started on 08/17, 09:20 UTC. During the timeline of the incident customers experienced latency for platform logs data .
- Root Cause: The failure was due to underlying cache infrastructure that became unhealthy.
- Incident Timeline: 08/17, 09:20 through 08/18, 17:30 UTC.
We understand that customers rely on Azure Monitor as a critical service and apologize for any impact this incident caused.
-Chandar
Update: Wednesday, 18 August 2021 16:59 UTC
We continue to clear the backlog of log requests that have been pending. Based on progress observed so far and the trend, we are expecting the backlog to clear within the next 3 hours
- Work Around:
- Next Update: Before 08/18 20:00 UTC
-Chandar
Update: Wednesday, 18 August 2021 04:12 UTC
We have identified a potential root cause of an unhealthy, dependent Redis Cache and have applied mitigation to return that cache to a healthy state. We continue to clear the backlog of log requests that have been pending. Executing the safest and most effective strategy to clear the backlog could potentially take up to 10 hours.
- Work Around: None
- Next Update: Before 08/18 16:30 UTC
-Mohini
Update: Tuesday, 17 August 2021 22:36 UTC
We
have applied mitigation steps and are working clear the backlog of log requests
that have been pending. We are continuing to apply further mitigation steps in
order to expedite the processing of the backlog and have engaged additional
teams to assist in leveraging their resources to help quickly process pending
requests. The next update will be provided in 2 hours, or as events warrant.- Work Around: None
- Next Update: Before 08/18 01:00 UTC
-Chandar
Update: Tuesday, 17 August 2021 18:29 UTC
We continue to investigate issues within Azure Monitor. Root cause is not fully understood at this time. Some customers in impacted regions continue to experience high latency for platform logs configured via Diagnostics settings.. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 08/17 20:30 UTC
-Chandar
Initial Update: Tuesday, 17 August 2021 17:09 UTC
Starting at 09:20 UTC on 17 Aug 2021 you have been identified as a customer using Azure Monitor who may be experiencing high latency for platform logs configured via Diagnostics settings.
- Work Around: None
- Next Update: Before 08/17 18:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Chandar
