Experiencing Data Access Issue in Azure portal for Log Analytics - 05/06 - Resolved

Published 05-06-2021 01:18 AM 723 Views
Final Update: Thursday, 06 May 2021 13:52 UTC

We've confirmed that all systems are back to normal with no customer impact as of 05/06, 09:45 UTC. Our logs show the incident started on 05/06, 07:40 UTC and that during the 2 hours & 5 minutes that it took to resolve the issue some customers may have experienced data access issues and delayed or missed Log Search Alerts in West Europe region.
  • Root Cause: The failure was due to issues with our backend service.
  • Incident Timeline: 2 Hours & 5 minutes - 05/06, 07:40 UTC through 05/06, 09:45 UTC
We understand that customers rely on Azure Log Analytics as a critical service and apologize for any impact this incident caused.

-Deepika

Update: Thursday, 06 May 2021 11:27 UTC

Root cause has been isolated to issue with few of our backend service machines CPU reaching 100%, causing them to go offline. To address this, our engineers are currently scaling out the service to clear the backlog and rebooting the offline machines.
  • Work Around: None
  • Next Update: Before 05/06 15:30 UTC
-Deepika

Initial Update: Thursday, 06 May 2021 08:16 UTC

We are aware of issues within Log Analytics and are actively investigating. Some customers may experience data access issues and delayed or missed Log Search Alerts in West Europe region.
  • Work Around: None
  • Next Update: Before 05/06 11:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Mohsin Inamdar

%3CLINGO-SUB%20id%3D%22lingo-sub-2330240%22%20slang%3D%22en-US%22%3EExperiencing%20Data%20Access%20Issue%20in%20Azure%20portal%20for%20Log%20Analytics%20-%2005%2F06%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2330240%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Thursday%2C%2006%20May%202021%2013%3A52%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%2005%2F06%2C%2009%3A45%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2005%2F06%2C%2007%3A40%20UTC%20and%20that%20during%20the%202%20hours%20%26amp%3B%205%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20may%20have%20experienced%26nbsp%3Bdata%20access%20issues%20and%20delayed%20or%20missed%20Log%20Search%20Alerts%20in%20West%20Europe%20region.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20issues%20with%20our%20backend%20service.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%202%20Hours%20%26amp%3B%205%20minutes%20-%2005%2F06%2C%2007%3A40%20UTC%20through%2005%2F06%2C%2009%3A45%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Azure%20Log%20Analytics%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Deepika%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22%22%3E%3CU%20style%3D%22font-size%3A%2014px%3B%22%3EUpdate%3C%2FU%3E%3A%20Thursday%2C%2006%20May%202021%2011%3A27%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3ERoot%20cause%20has%20been%20isolated%20to%20issue%20with%20few%20of%20our%20backend%20service%20machines%20CPU%20reaching%20100%25%2C%20causing%20them%20to%20go%20offline.%20To%20address%20this%2C%20our%20engineers%20are%20currently%20scaling%20out%20the%20service%20to%20clear%20the%20backlog%20and%20rebooting%20the%20offline%20machines.%3CBR%20%2F%3E%3CUL%20style%3D%22font-size%3A%2014px%3B%22%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F06%2015%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Deepika%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Thursday%2C%2006%20May%202021%2008%3A16%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Log%20Analytics%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20data%20access%20issues%20and%20delayed%20or%20missed%20Log%20Search%20Alerts%20in%20West%20Europe%20region.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F06%2011%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Mohsin%20Inamdar%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Version history
Last update:
‎May 06 2021 07:17 AM
Updated by: