Final Update: Sunday, 21 February 2021 21:04 UTC
We've confirmed that all systems are back to normal with no customer impact as of 02/21, 20:40 UTC. Our logs show the incident started on 02/21, 14:50 UTC and that during the 5 hours and 50 minutes that it took to resolve the issue 95% of customers in Southeast Australia region experienced latent or missed log alerts.
-
Root Cause: The failure was due to a backend component becoming overloaded with an unexpectedly large number of alerts which was impacting the ability to process new alerts.
-
Incident Timeline: 5 Hours & 50 minutes - 02/21, 14:50 UTC through 02/21, 20:40 UTC
We understand that customers rely on Log Search Alerts as a critical service and apologize for any impact this incident caused.
-Jeff