%3CLINGO-SUB%20id%3D%22lingo-sub-1530709%22%20slang%3D%22en-US%22%3EExperiencing%20Latency%2C%20Data%20Gap%20and%20Alerting%20failure%20for%20Azure%20Monitoring%20-%2007%2F18%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1530709%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Saturday%2C%2018%20July%202020%2015%3A37%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%2007%2F18%2C%2011%3A40%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2007%2F18%2C%2007%3A50%20UTC%20and%20that%20during%20the%203%20hours%2050%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20may%20have%20experienced%20Data%20access%2C%20Data%20latency%2C%20Data%20Loss%2C%20incorrect%20Alert%20activation%2C%20missed%20or%20delayed%20Alerts%20and%20Azure%20Alerts%20created%20during%20the%20impact%20duration%20may%20have%20been%20available%20to%20be%20viewed%20with%20some%20delay%20in%20the%20Azure%20portal%20in%20multiple%20regions.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20an%20issue%20in%20one%20of%20our%20dependent%20services.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%203%20Hours%20%26amp%3B%2050%20minutes%20-%2007%2F18%2C%207%3A50%20UTC%20through%2007%2F18%2C%2011%3A40%20UTC%3CBR%20%2F%3E%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Anmol%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22%22%3E%3CU%20style%3D%22font-size%3A%2014px%3B%22%3EUpdate%3C%2FU%3E%3A%20Saturday%2C%2018%20July%202020%2011%3A17%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Azure%20Monitoring%20services.%20Some%20customers%20continue%20to%20experience%20Data%20access%2C%20Data%20latency%20and%20Data%20Loss%2C%20incorrect%20Alert%20activation%2C%20missed%20or%20delayed%20Alerts%20and%20Azure%20Alerts%20created%20during%20the%20impact%20duration%20may%20not%20be%20available%20to%20be%20viewed%20in%20the%20Azure%20portal%20in%20multiple%20regions.%20We%20are%20working%20to%20establish%20the%20start%20time%20for%20the%20issue%2C%20initial%20findings%20indicate%20that%20the%20problem%20began%20at%2007%2F18%20~07%3A58%20UTC.%20We%20currently%20have%20no%20estimate%20for%20resolution.%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2007%2F18%2014%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Anmol%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Saturday%2C%2018%20July%202020%2008%3A58%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Application%20Insights%20and%20Log%20Analytics%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20Data%20access%20issues%20in%20the%20Azure%20portal%2C%20Incorrect%20Alert%20Activation%2C%20Latency%20and%20Data%20Loss%20in%20multiple%20regions.%26nbsp%3B%3CBR%20%2F%3E%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None.%26nbsp%3B%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2007%2F18%2011%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Madhuri%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1530709%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EActivity%20Log%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Search%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESmart%20Diagnostics%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Saturday, 18 July 2020 15:37 UTC

We've confirmed that all systems are back to normal with no customer impact as of 07/18, 11:40 UTC. Our logs show the incident started on 07/18, 07:50 UTC and that during the 3 hours 50 minutes that it took to resolve the issue some customers may have experienced Data access, Data latency, Data Loss, incorrect Alert activation, missed or delayed Alerts and Azure Alerts created during the impact duration may have been available to be viewed with some delay in the Azure portal in multiple regions.
  • Root Cause: The failure was due to an issue in one of our dependent services.
  • Incident Timeline: 3 Hours & 50 minutes - 07/18, 7:50 UTC through 07/18, 11:40 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Anmol

Update: Saturday, 18 July 2020 11:17 UTC

We continue to investigate issues within Azure Monitoring services. Some customers continue to experience Data access, Data latency and Data Loss, incorrect Alert activation, missed or delayed Alerts and Azure Alerts created during the impact duration may not be available to be viewed in the Azure portal in multiple regions. We are working to establish the start time for the issue, initial findings indicate that the problem began at 07/18 ~07:58 UTC. We currently have no estimate for resolution.
  • Work Around: None
  • Next Update: Before 07/18 14:30 UTC
-Anmol

Initial Update: Saturday, 18 July 2020 08:58 UTC

We are aware of issues within Application Insights and Log Analytics and are actively investigating. Some customers may experience Data access issues in the Azure portal, Incorrect Alert Activation, Latency and Data Loss in multiple regions. 
  • Work Around: None. 
  • Next Update: Before 07/18 11:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Madhuri