Latency and Data Loss for Application Insights and Log Analytics - 05/18

Published May 17 2022 06:28 PM 527 Views
Final Update: Wednesday, 18 May 2022 02:03 UTC

We've confirmed that all systems are back to normal with no customer impact as of 5/18, 01:30 UTC. Our logs show the incident started on 5/17, 19:16 UTC and that during the 6 hours and 14 minutes that it took to resolve the issue customers experienced data latency and data gaps for Application Insights and Azure Log Analytics. This could have resulted in delayed or misfired alerts. 
  • Root Cause: A power disruption impacted a portion of our storage services.
  • Incident Timeline: 6 Hours & 14 minutes - 5/17, 19:16 UTC through 5/18, 01:30 UTC
We understand that customers rely on Application Insights and Log Analytics as critical services and apologize for any impact this incident caused.

-Ian

Update: Wednesday, 18 May 2022 01:21 UTC

We continue to investigate issues within Application Insights and Log Analytics. A power disruption event compromised portions of our storage in the South Africa North region. Customers may experience data gaps, data latency and failed or misfired alerts

  • Work Around: none

  • Next Update: Before 05/18 03:30 UTC

-Ian


%3CLINGO-SUB%20id%3D%22lingo-sub-3390623%22%20slang%3D%22en-US%22%3ELatency%20and%20Data%20Loss%20for%20Application%20Insights%20and%20Log%20Analytics%20-%2005%2F18%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3390623%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Wednesday%2C%2018%20May%202022%2002%3A03%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%205%2F18%2C%2001%3A30%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%205%2F17%2C%2019%3A16%20UTC%20and%20that%20during%20the%206%20hours%20and%2014%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20customers%20experienced%20data%20latency%20and%20data%20gaps%20for%20Application%20Insights%20and%20Azure%20Log%20Analytics.%20This%20could%20have%20resulted%20in%20delayed%20or%20misfired%20alerts.%26nbsp%3B%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20A%20power%20disruption%20impacted%20a%20portion%20of%20our%20storage%20services.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%206%20Hours%20%26amp%3B%2014%20minutes%20-%205%2F17%2C%2019%3A16%20UTC%20through%205%2F18%2C%2001%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20and%20Log%20Analytics%20as%20critical%20services%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Ian%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CP%20class%3D%22MsoNormal%22%20style%3D%22margin-bottom%3A0in%3Bline-height%3Anormal%22%3E%3CU%3E%3CSPAN%20style%3D%22font-size%3A%2010.5pt%3B%20font-family%3A%20Roboto%3B%20background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3EUpdate%3C%2FSPAN%3E%3C%2FU%3E%3CSPAN%20style%3D%22font-size%3A%2010.5pt%3B%20font-family%3A%20Roboto%3B%20background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3E%3A%20Wednesday%2C%2018%20May%202022%2001%3A21%20UTC%3C%2FSPAN%3E%3CSPAN%20style%3D%22font-size%3A%2010.5pt%3B%20font-family%3A%20Roboto%3B%22%3E%3CBR%20%2F%3E%20%3CBR%20%2F%3E%20%3CSPAN%20style%3D%22background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3EWe%20continue%20to%20investigate%20issues%20within%20Application%20Insights%20and%20Log%20Analytics.%20A%20power%20disruption%20event%20compromised%20portions%20of%20our%20storage%20in%20the%20South%20Africa%20North%20region.%20Customers%20may%20experience%20data%20gaps%2C%20data%20latency%20and%20failed%20or%20misfired%20alerts%3C%2FSPAN%3E%3CBR%20%2F%3E%20%3C!--%5Bif%20!supportLineBreakNewLine%5D--%3E%3CBR%20%2F%3E%20%3C!--%5Bendif%5D--%3E%3C%2FSPAN%3E%3CSPAN%20style%3D%22font-size%3A12.0pt%3Bfont-family%3A%22%20times%3D%22%22%20new%3D%22%22%20roman%3D%22%22%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3C%2FP%3E%3CP%3E%3C%2FP%3E%0A%0A%3CUL%20type%3D%22disc%22%3E%0A%20%3CLI%20class%3D%22MsoNormal%22%20style%3D%22line-height%3A%20normal%3B%20background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3E%3CU%3E%3CSPAN%20style%3D%22font-size%3A10.5pt%3Bfont-family%3ARoboto%3B%0A%20%20%20%20%20mso-fareast-font-family%3A%22%20times%3D%22%22%20new%3D%22%22%20roman%3D%22%22%3EWork%20Around%3C%2FSPAN%3E%3C%2FU%3E%3CSPAN%20style%3D%22font-size%3A10.5pt%3Bfont-family%3ARoboto%3Bmso-fareast-font-family%3A%22%20times%3D%22%22%20new%3D%22%22%20roman%3D%22%22%3E%3A%20none%3CP%3E%3C%2FP%3E%3C%2FSPAN%3E%3C%2FLI%3E%0A%20%3CLI%20class%3D%22MsoNormal%22%20style%3D%22line-height%3A%20normal%3B%20background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3E%3CU%3E%3CSPAN%20style%3D%22font-size%3A10.5pt%3Bfont-family%3ARoboto%3B%0A%20%20%20%20%20mso-fareast-font-family%3A%22%20times%3D%22%22%20new%3D%22%22%20roman%3D%22%22%3ENext%20Update%3C%2FSPAN%3E%3C%2FU%3E%3CSPAN%20style%3D%22font-size%3A10.5pt%3Bfont-family%3ARoboto%3Bmso-fareast-font-family%3A%22%20times%3D%22%22%20new%3D%22%22%20roman%3D%22%22%3E%3A%20Before%2005%2F18%2003%3A30%20UTC%3CP%3E%3C%2FP%3E%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%0A%3CP%20class%3D%22MsoNormal%22%3E%3CSPAN%20style%3D%22font-size%3A%2010.5pt%3B%20line-height%3A%20107%25%3B%20font-family%3A%20Roboto%3B%20background-image%3A%20initial%3B%20background-position%3A%20initial%3B%20background-size%3A%20initial%3B%20background-repeat%3A%20initial%3B%20background-attachment%3A%20initial%3B%20background-origin%3A%20initial%3B%20background-clip%3A%20initial%3B%22%3E-Ian%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3C%2FP%3E%3CP%3E%3C%2FP%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Version history
Last update:
‎May 17 2022 07:09 PM
Updated by: