Experiencing Alerting failure issue in Azure Portal for Many Data Types - 12/12 - Resolved

[Azure-Monitor-Team]

First published on MSDN on Dec 12, 2018

Final Update: Wednesday, 12 December 2018 15:28 UTC

We've confirmed that all systems are back to normal with no customer impact as of 12/12, 15:00 UTC. Our logs show the incident started on 12/12, 08:30 UTC and that during the 6 hours and 30 minutes that it took to resolve the issue  a very small subset of customers who have configured their web tests in West US region may have experienced alerting failures.

• Root Cause: The failure was due to performance degradation in one of our backend services.
• Incident Timeline: 6 Hours & 30 minutes - 12/12, 08:30 UTC through 12/12, 15:00 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Varun

Update: Wednesday, 12 December 2018 14:45 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers who have configured their web tests in West US region may continue to experience Alerting failure. We are working to establish the start time for the issue, initial findings indicate that the problem began at 12/12 ~08:30 UTC>. We currently have no estimate for resolution.

• Work Around: Web tests running in regions other than West US are not impacted. Customers should ensure their availability tests are running from at least three locations to ensure redundant coverage.
• Next Update: Before 12/12 19:00 UTC

-Varun

Initial Update: Wednesday, 12 December 2018 10:45 UTC

We are aware of issues within Application Insights and are actively investigating. A very small subset of customers who have configured their web tests in West US region may experience Alerting failure.

• Work Around: Customers should ensure their availability tests are running from at least three locations to ensure redundant coverage.
• Next Update: Before 12/12 15:00 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Varun