Experiencing Alerting failure for Log Search Alerts - 04/11 - Resolved
Published Apr 10 2020 05:44 PM 996 Views
Final Update: Saturday, 11 April 2020 02:22 UTC

We've confirmed that all systems are back to normal with no customer impact as of 4/11, 01:43 UTC. Our logs show the incident started on 4/10 22:15 UTC and that during the 3 hours 28 minutes that it took to resolve the issue, customers may experience issues with missed or delayed Log Search Alerts in the South Africa North region.
  • Root Cause: The failure was due to an unhealthy service in our back end. 
  • Incident Timeline: 3Hours & 28 minutes - 4/10 22:15 UTC through 4/11, 01:43 UTC
We understand that customers rely on Log Search Alerts as a critical service and apologize for any impact this incident caused.

-Ian

Update: Saturday, 11 April 2020 02:16 UTC

Root cause has been isolated to an unhealthy service in our back end.  This was Log Search Alerts in the South Africa North region. Customers may experience issues with missed or delayed Log Search Alerts. We believe the issue should be mitigated shortly as the backend service has appears to be healthy.
  • Work Around: none
  • Next Update: Before 04/11 03:30 UTC 

We apologize for the inconvenience.

-Ian

Initial Update: Saturday, 11 April 2020 00:41 UTC

We are aware of issues within Log Search Alerts and are actively investigating. Some customers may experience issues with missed or delayed Log Search Alerts in the South Africa North region. The impact began at 4/10 22:15 UTC
  • Work Around: None
  • Next Update: Before 04/11 02:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Ian

Version history
Last update:
‎Apr 10 2020 07:27 PM
Updated by: