Experiencing Data Access issue in Azure Portal for Some Metrics - 08/31 - Resolved

Published Aug 30 2021 07:33 PM 1,237 Views
Final Update: Wednesday, 01 September 2021 16:40 UTC

We've confirmed that all systems are back to normal with no customer impact as of 2021-09-01 18:58 UTC. Our logs show the incident started on 2021-08-29 13:22 UTC and that during the two days, 5.5 hours that it took to resolve the issue 100% of customers who queried metrics from the Microsoft.Network/privateEndpoints and Microsoft.Network/privateLinkService resource providers will have returned 400 error codes and not been able to query their metrics.
  • Root Cause: The failure was due to new feature rollout that contained an error.
  • Lessons Learned: We will be applying greater monitoring in lower level environments to catch errors like these before they reach production environments.
  • Incident Timeline: 2 Days, 5 Hours & 36 minutes - 2021-08-29 13:22 UTC through 2021-09-01 18:58 UTC
We understand that customers rely on Azure Monitor and Azure Metrics as critical services and apologize for any impact this incident caused.

-Jack Cantwell

Update: Tuesday, 31 August 2021 02:26 UTC

Root cause has been isolated to a feature update which inadvertently caused metrics for the resource providers Microsoft.Network/privateEndpoints and Microsoft.Network/privateLinkService to return 400 errors when queried. To address this issue we are rolling out a hotfix. The hotfix rollout process across all regions will take approximately 18 hours. Some customers will continue to experience these issues until all regions have have gotten the hotfix applied.
  • Next Update: Before 08/31 20:30 UTC
-Jack Cantwell

Version history
Last update:
‎Sep 01 2021 09:46 AM
Updated by: