%3CLINGO-SUB%20id%3D%22lingo-sub-1117918%22%20slang%3D%22en-US%22%3EExperiencing%20failure%20in%20alert%20creation%20or%20updation%20for%20Metric%20Alerts%20-%2001%2F20%20-%20Mitigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1117918%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Tuesday%2C%2021%20January%202020%2011%3A37%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EHotfix%20deployment%20is%20still%20in%20process.%20Some%20customers%20may%20still%20experience%20issues%20while%20creating%20Azure%20monitor%20metric%20alert%26nbsp%3Bwith%20aggregation%20max%20or%20min.%20Initial%20findings%20indicate%20that%20the%20problem%20began%20from%2001%2F08%2023%3A00%20UTC.%26nbsp%3B%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2001%2F22%2000%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Anmol%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Monday%2C%2020%20January%202020%2023%3A21%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3ERoot%20cause%20has%20been%20isolated%20to%20previous%20deployment%20which%20was%20impacting%20creating%20alerts%20with%20aggregation%20max%20or%20min.%20To%20address%20this%20issue%20engineers%20are%20working%20on%20deploying%20a%20fix.%20Some%20customers%20may%20experience%20issues%20while%20creating%20alerts%20with%20aggregation%20max%20or%20min.%20We%20estimate%20~12%20hours%20before%20this%20issue%20is%20addressed.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20none%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2001%2F21%2011%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Anupama%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Monday%2C%2020%20January%202020%2011%3A17%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Metric%20Alerts%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20failures%20in%20creating%20Azure%20Monitor%20metric%20alert%20with%20aggregations%20Max%20or%20Min.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2001%2F20%2023%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Anmol%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1117918%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Tuesday, 21 January 2020 17:34 UTC

We've confirmed that all systems are back to normal with no customer impact at present. Our logs show the incident started on 01/08, 23:00 UTC and that during the 13 days that it took to resolve the issue customers experienced issues while creating Azure monitor metric alerts with aggregation max and min.
  • Root Cause: Engineers determined that a deployment task introduced a configuration change which failed to include certain scenarios needed for these specific Azure Metrics Alerts. 
  • Incident Timeline:  13 days - 01/08, 23:00 UTC through 1/21, 16:40 UTC
We understand that customers rely on Metric Alerts as a critical service and apologize for any impact this incident caused.

-Anupama

Update: Tuesday, 21 January 2020 11:37 UTC

Hotfix deployment is still in process. Some customers may still experience issues while creating Azure monitor metric alert with aggregation max or min. Initial findings indicate that the problem began from 01/08 23:00 UTC. 
  • Work Around: None
  • Next Update: Before 01/22 00:00 UTC
-Anmol

Update: Monday, 20 January 2020 23:21 UTC

Root cause has been isolated to previous deployment which was impacting creating alerts with aggregation max or min. To address this issue engineers are working on deploying a fix. Some customers may experience issues while creating alerts with aggregation max or min. We estimate ~12 hours before this issue is addressed.
  • Work Around: none
  • Next Update: Before 01/21 11:30 UTC
-Anupama

Initial Update: Monday, 20 January 2020 11:17 UTC

We are aware of issues within Metric Alerts and are actively investigating. Some customers may experience failures in creating Azure Monitor metric alert with aggregations Max or Min.
  • Work Around: None
  • Next Update: Before 01/20 23:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Anmol