Final Update: Thursday, 27 August 2020 22:36 UTC
We've confirmed that all systems are back to normal with no customer impact as of 08/27, 21:15 UTC. Our logs show the incident started on 08/27, 10:30 UTC and that during the 10 hours and 45 minutes that it took to resolve the issue some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
-Eric Singleton
We've confirmed that all systems are back to normal with no customer impact as of 08/27, 21:15 UTC. Our logs show the incident started on 08/27, 10:30 UTC and that during the 10 hours and 45 minutes that it took to resolve the issue some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
- Root Cause: The failure was due to unexpected increase in load to the backend service, affecting cluster health.
- Incident Timeline: 10 Hours & 45 minutes - 08/27, 10:30 UTC through 08/27, 21:15 UTC
-Eric Singleton
Update: Thursday, 27 August 2020 20:46 UTC
Engineers are continuing to work on removing the load and monitoring cluster health. Some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
Engineers are continuing to work on removing the load and monitoring cluster health. Some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
- Work Around: None
- Next Update: Before 08/27 23:00 UTC
Update: Thursday, 27 August 2020 18:39 UTC
Root cause has been isolated to an unexpected increase in load to the backend service affecting cluster health. To address this, engineers are currently working on removing the load.
Root cause has been isolated to an unexpected increase in load to the backend service affecting cluster health. To address this, engineers are currently working on removing the load.
- Work Around: None
- Next Update: Before 08/27 21:00 UTC
Update: Thursday, 27 August 2020 15:21 UTC
We continue to investigate issues within Log Search Alerts in East Japan. Some customers continue to experience issues with missed or delayed Log Search Alerts . Initial findings indicate that the problem began on 08/27 10:30 UTC. We currently have no estimate for resolution.
We continue to investigate issues within Log Search Alerts in East Japan. Some customers continue to experience issues with missed or delayed Log Search Alerts . Initial findings indicate that the problem began on 08/27 10:30 UTC. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 08/27 18:30 UTC
Initial Update: Thursday, 27 August 2020 11:54 UTC
We are aware of issues within Log Search Alerts and are actively investigating. Some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
-Subhash
We are aware of issues within Log Search Alerts and are actively investigating. Some customers may experience issues with missed or delayed Log Search Alerts in East Japan.
- Work Around: None
- Next Update: Before 08/27 16:00 UTC
-Subhash
Updated Aug 27, 2020
Version 6.0