%3CLINGO-SUB%20id%3D%22lingo-sub-1273476%22%20slang%3D%22en-US%22%3EExperiencing%20email%20notification%20failures%20in%20Azure%20Government%20cloud%20-%2004%2F02%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1273476%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Thursday%2C%2002%20April%202020%2005%3A05%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3E%3CDIV%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%2004%2F02%2C%2004%3A00%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2004%2F01%2C%2023%3A30%20UTC%20and%20that%20during%20the%2004%20hours%2030%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20in%20Azure%20US%20Government%20cloud%20might%20have%20experienced%20issues%20receiving%20emails%20from%20their%20notifications%20for%20all%20types%20of%20Azure%20Alerts%3C%2FDIV%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20one%20of%20our%20back%20end%20service.%3C%2FLI%3E%0A%20%20%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%204%20Hours%20%26amp%3B%2030%20minutes%20-%2004%2F01%2C%2023%3A30%20UTC%20through%2004%2F02%2C%2004%3A00%20UTC%3CBR%20%2F%3E%3C%2FLI%3E%0A%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Azure%20Alerts%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E-Syed%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Thursday%2C%2002%20April%202020%2002%3A55%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Azure%20Monitoring%20and%20are%20actively%20investigating.%20Some%20customers%20in%20Azure%20US%20Government%20cloud%20may%20experience%20issues%20receiving%20%3CB%3Eemails%3C%2FB%3E%20from%20their%20notifications%20for%20all%20types%20of%20Azure%20Alerts.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20none%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2004%2F02%2007%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Jayadev%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1273476%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EActivity%20Log%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESmart%20Diagnostics%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Thursday, 02 April 2020 05:05 UTC

We've confirmed that all systems are back to normal with no customer impact as of 04/02, 04:00 UTC. Our logs show the incident started on 04/01, 23:30 UTC and that during the 04 hours 30 minutes that it took to resolve the issue some customers in Azure US Government cloud might have experienced issues receiving emails from their notifications for all types of Azure Alerts
  • Root Cause: The failure was due to one of our back end service.
  • Incident Timeline: 4 Hours & 30 minutes - 04/01, 23:30 UTC through 04/02, 04:00 UTC
We understand that customers rely on Azure Alerts as a critical service and apologize for any impact this incident caused.
-Syed

Initial Update: Thursday, 02 April 2020 02:55 UTC

We are aware of issues within Azure Monitoring and are actively investigating. Some customers in Azure US Government cloud may experience issues receiving emails from their notifications for all types of Azure Alerts.
  • Work Around: none
  • Next Update: Before 04/02 07:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Jayadev