Experiencing data latency and data gap for Azure Monitor - 10/07 - Resolved

Published Oct 06 2021 11:40 PM 916 Views
Final Update: Thursday, 07 October 2021 09:03 UTC

We've confirmed that all systems are back to normal with no customer impact as of 10/07, 08:40 UTC. Our logs show the incident started on 10/07, 03:02 UTC and that during the 5 hours and 38 minutes that it took to resolve the issue some customers using Log Analytics in the Australia East region may have experienced data latency, data gaps and missed or delayed alerts for storage telemetry, event hub telemetry and log analytics workspace data.

  • Root Cause: We determined that the impact was caused by an incorrect service configuration, and was triggered by an automatic upgrade of the underlying infrastructure.
  • Incident Timeline: 5 Hours & 38 minutes - 10/07, 03:02 UTC through 10/07, 08:40 UTC
We understand that customers rely on Azure Log Analytics as a critical service and apologize for any impact this incident caused.

-Anmol

Initial Update: Thursday, 07 October 2021 06:18 UTC

We are aware of issues within Azure Monitor and are actively investigating. Some customers using Log Analytics in Australia East region may experience data latency, data gaps, and missed or delayed alerts for storage telemetry, event hub telemetry and log analytics workspace data.
  • Work Around: None
  • Next Update: Before 10/07 09:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Anmol

%3CLINGO-SUB%20id%3D%22lingo-sub-2820868%22%20slang%3D%22en-US%22%3EExperiencing%20data%20latency%20and%20data%20gap%20for%20Azure%20Monitor%20-%2010%2F07%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2820868%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Thursday%2C%2007%20October%202021%2009%3A03%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3E%3CP%20style%3D%22color%3A%20rgb(14%2C%2016%2C%2026)%3B%20background%3A%20transparent%3B%20margin-top%3A0pt%3B%20margin-bottom%3A0pt%3B%22%3E%3CSPAN%20style%3D%22background%3A%20transparent%3B%20margin-top%3A%200pt%3B%20margin-bottom%3A%200pt%3B%22%20data-preserver-spaces%3D%22true%22%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%2010%2F07%2C%2008%3A40%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2010%2F07%2C%2003%3A02%20UTC%20and%20that%20during%20the%205%20hours%20and%2038%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20using%20Log%20Analytics%20in%20the%20Australia%20East%20region%20may%20have%20experienced%20data%20latency%2C%20data%20gaps%20and%20missed%20or%20delayed%20alerts%20for%20storage%20telemetry%2C%20event%20hub%20telemetry%20and%20log%20analytics%20workspace%20data.%3C%2FSPAN%3E%3C%2FP%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20We%20determined%20that%20the%20impact%20was%20caused%20by%20an%20incorrect%20service%20configuration%2C%20and%20was%20triggered%20by%20an%20automatic%20upgrade%20of%20the%20underlying%20infrastructure.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%205%20Hours%20%26amp%3B%2038%20minutes%20-%2010%2F07%2C%2003%3A02%20UTC%20through%2010%2F07%2C%2008%3A40%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Azure%20Log%20Analytics%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Anmol%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Thursday%2C%2007%20October%202021%2006%3A18%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3E%3CDIV%3EWe%20are%20aware%20of%20issues%20within%20Azure%20Monitor%20and%20are%20actively%20investigating.%20Some%20customers%20using%20Log%20Analytics%20in%20Australia%20East%20region%20may%20experience%20data%20latency%2C%20data%20gaps%2C%20and%20missed%20or%20delayed%20alerts%20for%20storage%20telemetry%2C%20event%20hub%20telemetry%20and%20log%20analytics%20workspace%20data.%3C%2FDIV%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2010%2F07%2009%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Anmol%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Version history
Last update:
‎Oct 07 2021 02:05 AM
Updated by: