Experiencing Data Access and Latency issues in Azure Portal - 03/15 - Resolved

Published 04-07-2021 11:07 AM 499 Views
Final Update: Tuesday, 16 March 2021 12:46 UTC

We've confirmed that all systems are back to normal with no customer impact as of 03/16, 12:00 UTC. Our logs show the incident started on 03/15, 11:10 UTC and that during the 1 day and 50 minutes that it took to resolve the issue some customers using Azure Monitor Service may have experienced intermittent data latency, data access, and incorrect alert activation in East US and West Europe region.
  • Root Cause: The root cause has been isolated to failures in Azure Active Directory which was impacting all of Azure Monitor Service.
  • Incident Timeline: 1 Day & 50 minutes - 03/15, 11:10 UTC through 03/16, 12:00 UTC
We understand that customers rely on Azure Monitor as a critical service and apologize for any impact this incident caused.

-Anmol

Update: Tuesday, 16 March 2021 11:36 UTC

Root cause has been isolated to failures in Azure Active Directory which was impacting all of Azure Monitor Service. Azure Active Directory is now recovering, though each service may recover completely at a different time, due to differences in the timing of refreshing Azure Active Directory tokens. Some customer using Azure Monitor Service may continue to experience intermittent data latency, data access and incorrect alert activation in East US and West Europe region.

  • Work Around: None
  • Next Update: Before 03/16 18:00 UTC
-Madhav

Update: Tuesday, 16 March 2021 03:23 UTC

There is currently just one service within Azure Monitor that is still experiencing problems:
  • Data ingestion: Some data gaps as well as continued delays in East US
  • Next Update: Before 03/16 05:30 UTC
-Jack

Update: Monday, 15 March 2021 21:51 UTC

Root cause has been isolated to failures in Azure Active Directory which was impacting all of Azure Monitor. Azure Active Directory is now recovering, though each service may recover completely at a different time, due to differences in the timing of refreshing Azure Active Directory tokens. Some customers may continue to experience challenges connecting to the data access API or with data latency. We estimate 1 - 2 hours before all problems are completely addressed.
  • Next Update: Before 03/15 23:00 UTC
-Jack

%3CLINGO-SUB%20id%3D%22lingo-sub-2212073%22%20slang%3D%22en-US%22%3EExperiencing%20Data%20Access%20and%20Latency%20issues%20in%20Azure%20Portal%20-%2003%2F15%20-%20Mitigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2212073%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Monday%2C%2015%20March%202021%2023%3A30%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EThere%20are%20currently%20several%20services%20within%20Azure%20Monitor%20that%20are%20still%20experiencing%20problems%3A%3C%2FDIV%3E%3CUL%3E%3CLI%20style%3D%22font-size%3A%2014px%3B%22%3EAzure%20Alerts%3A%20approximately%2050%25%20of%20alerts%20are%20not%20firing%20properly%3C%2FLI%3E%3CLI%20style%3D%22font-size%3A%2014px%3B%22%3EEnd-to-End%20Transaction%20Details%3A%20missing%20data%3C%2FLI%3E%3CLI%20style%3D%22font-size%3A%2014px%3B%22%3EData%20Access%20API%3A%20continuing%20failures%20to%20connect%20or%20inability%20to%20access%20data%20specifically%20in%20West%20Europe%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2003%2F16%2000%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E-Jack%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Monday%2C%2015%20March%202021%2021%3A51%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3ERoot%20cause%20has%20been%20isolated%20to%20failures%20in%20Azure%20Active%20Directory%20which%20was%20impacting%20all%20of%20Azure%20Monitor.%20Azure%20Active%20Directory%20is%20now%20recovering%2C%20though%20each%20service%20may%20recover%20completely%20at%20a%20different%20time%2C%20due%20to%20differences%20in%20the%20timing%20of%20refreshing%20Azure%20Active%20Directory%20tokens.%20Some%20customers%20may%20continue%20to%20experience%20challenges%20connecting%20to%20the%20data%20access%20API%20or%20with%20data%20latency.%20We%20estimate%201%20-%202%20hours%20before%20all%20problems%20are%20completely%20addressed.%3CUL%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2003%2F15%2023%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Jack%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2212073%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Search%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎Mar 16 2021 05:48 AM
Updated by: