Final Update: Saturday, 25 May 2019 02:49 UTC
We've confirmed that all systems are back to normal with no customer impact as of 05/25, 2:35 UTC. Our logs show the incident started on 05/24, 20:50 UTC and that during the 5 hours and 45 min that it took to resolve the issue approximately 31% of customers in FairFax(USGov) experienced delay in Data Ingestion and also some customers experienced issues with alert notifications configured based on webhooks in FairFax(USGov)
-Venkat
We've confirmed that all systems are back to normal with no customer impact as of 05/25, 2:35 UTC. Our logs show the incident started on 05/24, 20:50 UTC and that during the 5 hours and 45 min that it took to resolve the issue approximately 31% of customers in FairFax(USGov) experienced delay in Data Ingestion and also some customers experienced issues with alert notifications configured based on webhooks in FairFax(USGov)
- Root Cause: The failure was due to issue with storage service.
- Incident Timeline: 5 Hours & 45 minutes - 05/24, 20:50 UTC through 05/25, 2:35 UTC
-Venkat
Update: Saturday, 25 May 2019 01:41 UTC
We continue to investigate issues within Log Analytics and Alert Notification.Root cause has been isolated to a storage issue. Some Log Analytics customers continue to experience delay in Data Ingestion and also issues with alert notifications configured based on webhooks in FairFax(USGov). We are working to establish the start time for the issue, initial findings indicate that the problem began at 05/24 ~20:50 UTC. We currently have no estimate for resolution.
We continue to investigate issues within Log Analytics and Alert Notification.Root cause has been isolated to a storage issue. Some Log Analytics customers continue to experience delay in Data Ingestion and also issues with alert notifications configured based on webhooks in FairFax(USGov). We are working to establish the start time for the issue, initial findings indicate that the problem began at 05/24 ~20:50 UTC. We currently have no estimate for resolution.
- Work Around: none
- Next Update: Before 05/25 06:00 UTC
Initial Update: Friday, 24 May 2019 22:08 UTC
We are aware of issues within Log Analytics and are actively investigating. Some customers may experience Data Ingestion delays in FairFax(USGov) region.
We are aware of issues within Log Analytics and are actively investigating. Some customers may experience Data Ingestion delays in FairFax(USGov) region.
- Work Around: none
- Next Update: Before 05/25 02:30 UTC
-Venkat
Updated May 25, 2019
Version 4.0
Azure-Monitor-Team
Microsoft
Microsoft