Final Update: Saturday, 28 September 2019 11:03 UTC
We've confirmed that all systems are back to normal with no customer impact as of 09/28, 10:48 UTC. Our logs show the incident started on 09/28, 07:15 UTC and that during the 3 hours & 35 minutes that it took to resolve the issue of some customers experienced Alerting failure due to Ingestion Latency in South East Australia.
Root Cause: The failure was due to issue in back-end service.
Incident Timeline: 3 Hours & 35 minutes - 09/28, 07:15 UTC through 09/28, 10:48 UTC
We understand that customers rely on Metric Alerts as a critical service and apologize for any impact this incident caused.
Initial Update: Saturday, 28 September 2019 08:41 UTC
We are aware of issues within Metric Alerts and are actively investigating. Some customers may experience ingestion latency causing alerting failure in South East Australia.
Work Around: None
Next Update: Before 09/28 12:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience. -Naresh