%3CLINGO-SUB%20id%3D%22lingo-sub-1072251%22%20slang%3D%22en-US%22%3EExperiencing%20failure%20in%20alert%20creation%20or%20updation%20for%20Classic%20alerts%20-%2012%2F18%20-%20Investigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1072251%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Wednesday%2C%2018%20December%202019%2016%3A05%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Classic%20Alerts%20and%20are%20actively%20investigating.%20Some%20customers%20in%20South%20Central%20US%20may%20experience%20failure%20in%20updation%20or%20creation%20of%20new%20alerts.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%26nbsp%3B%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2012%2F18%2018%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Madhuri%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1072251%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Wednesday, 18 December 2019 17:26 UTC

We've confirmed that all systems are back to normal with no customer impact as of 12/18, 16:40 UTC. Our logs show the incident started on 12/17, 17:00 UTC and that during the 23 hours 40 mins that it took to resolve the issue 90% of customers in South Central US may have received failure notifications when performing service management operations - such as create, update, delete and read for classic metric alerts hosted in this region.
  • Root Cause: Engineers determined that a recent configuration change caused a backend service in charge of processing service requests to become unhealthy, preventing requests from completing.. 
  • Mitigation: Engineers performed a change to the service configuration to mitigate the issue.
  • Incident Timeline: 23 Hours & 40 minutes - 12/17, 17:00 UTC through 12/18, 16:40 UTC
We understand that customers rely on Azure Monitor as a critical service and apologize for any impact this incident caused.

-Leela

Initial Update: Wednesday, 18 December 2019 16:05 UTC

We are aware of issues within Classic Alerts and are actively investigating. Some customers in South Central US may experience failure in updation or creation of new alerts.
  • Work Around: None 
  • Next Update: Before 12/18 18:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Madhuri