Experiencing latency for notifications for Azure Monitor Alerts - 01/25 - Resolved
Published Jan 24 2020 11:08 PM 1,922 Views
Final Update: Saturday, 25 January 2020 12:53 UTC

We've confirmed that all systems are back to normal with no customer impact as of 01/26, 04:50 UTC. Our logs show the incident started on 01/25, 05:00 UTC and that during the 23 hours and 50 minutes that it took to resolve the issue most of the customers may have experienced issues with missed or delayed Alerts notification for Azure Monitor service.
  • Root Cause: The failure was due to one of our internal services impacted due to the SQL DB outage happened in US Govt regions.
  • Incident Timeline: 23 Hours & 50 minutes - 01/25, 05:00 UTC through 01/26, 04:50 UTC
We understand that customers rely on Azure Monitor as a critical service and apologize for any impact this incident caused.

-Anmol

Update: Saturday, 25 January 2020 10:01 UTC

This issue started on 01/25 05:00 UTC and is caused by our dependency database system. Our team has been investigating this with the partner Azure database team but we do not have any root cause identified yet. Customers using Azure Monitor Services in the US Gov region continue to experience missed or delayed Alerts notifications for all type of notifications. We currently have no estimate for resolution.
  • Work Around: Triggered alerts can viewed in Azure Portal alerts page (link).  Please use the alerts page to actively monitor the health of resources till the issue is resolved.
  • Next Update: Before 01/25 16:30 UTC
-Anmol

Initial Update: Saturday, 25 January 2020 07:08 UTC

We are aware of issues within Log Search Alerts and are actively investigating. Some customers may experience latency for Webhook and Email notifications.
  • Work Around: None
  • Next Update: Before 01/25 09:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Anmol

Version history
Last update:
‎Jan 25 2020 10:09 PM
Updated by: