Final Update: Wednesday, 05 June 2019 23:58 UTC
We've confirmed that all systems are back to normal with no customer impact as of June 5th 2019, 21:36 UTC. Our logs show the incident started on June 5th 2019, 15:00 UTC and that during the time that it took to resolve the issue subset of customers would have missed receiving Alerting notifications. Customers might be receiving these missed notifications in a delayed timeline as the system processes the backlog.
- Root Cause: The failure was due to a performance issues of a downstream system
- Lessons Learned: Chance of recurrence of this issue is low, and we will continue to monitor the system.
- Incident Timeline: June 5th 2019, 21:36 UTC through June 5th 2019, 15:00 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.
-chandar.
Update: Wednesday, 05 June 2019 18:44 UTC
Root cause has been isolated to downstream service which was impacting
alerting notification for subset of customers. To address this issue we are in the process of rolling out a fix. Some customers may experience notification failures and we estimate 1 hour before all notification impacts are addressed.
- Work Around: none
- Next Update: Before 06/05 21:00 UTC
-Chandar.