Final Update: Monday, 09 September 2019 08:05 UTC
We've confirmed that all systems are back to normal with no customer impact as of 09/08, 13:00 UTC. Our logs show the incident started on 08/22, 13:00 UTC and that during the 17 days that it took to resolve the issue customers who were trying to create metric alerts would have experienced failure for a subset of metrics (Current Capacity Units, Current Compute Units, Backend Response Status, Client TLS Protocol, Bytes Sent, Bytes Received, Client RTT, Application Gateway Total Time, Backend Connect Time, Backend First Byte Response Time, Backend Last Byte Response Time) for Application Gateways resource type. These metrics alerts were not live for customer but due to bad deployment these metrics were shown up in portal. We have now disabled the subset of metrics from portal.
-
Root Cause: The subset of metrics for Application Gateways resource type would not have shown up in portal as they were still in testing phase.
-
Incident Timeline: 17 days - 08/22, 13:00 UTC through 09/08, 13:00 UTC
We understand that customers rely on Log Search Alerts as a critical service and apologize for any impact this incident caused.
-Rama
Update: Friday, 06 September 2019 10:38 UTC
We continue to investigate issues with Metric alert creation on a subset of metrics for Application Gateways resource type. Root cause is not fully understood at this time. Customers continue to experience metric alerts creation failure for a subset of metrics (Current Capacity Units, Current Compute Units, Backend Response Status, Client TLS Protocol, Bytes Sent, Bytes Received, Client RTT, Application Gateway Total Time, Backend Connect Time, Backend First Byte Response Time, Backend Last Byte Response Time) for Application Gateways resource type. We are working to establish the start time for the issue, initial findings indicate that the problem began on 08/22. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 09/07 11:00 UTC
-Rama
Update: Wednesday, 04 September 2019 13:11 UTC
We continue to investigate issues with Metric alert creation on a subset of metrics for Application Gateways resource type. Root cause is not fully understood at this time. Customers continue to experience metric alerts creation failure for a subset of metrics (Current Capacity Units, Current Compute Units, Backend Response Status, Client TLS Protocol, Bytes Sent, Bytes Received, Client RTT, Application Gateway Total Time, Backend Connect Time, Backend First Byte Response Time, Backend Last Byte Response Time) for Application Gateways resource type. We are working to establish the start time for the issue, initial findings indicate that the problem began on 08/22. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 09/05 13:30 UTC
-Rama
Update: Tuesday, 03 September 2019 13:48 UTC
We continue to investigate issues with Metric alert creation on a subset of metrics for Application Gateways resource type. Root cause is not fully understood at this time. Customers continue to experience metric alerts creation failure for a subset of metrics (Current Capacity Units, Current Compute Units, Backend Response Status, Client TLS Protocol, Bytes Sent, Bytes Received, Client RTT, Application Gateway Total Time, Backend Connect Time, Backend First Byte Response Time, Backend Last Byte Response Time) for Application Gateways resource type. We are working to establish the start time for the issue, initial findings indicate that the problem began on 08/22. We currently have no estimate for resolution.
- Work Around: None
- Next Update: Before 09/04 14:00 UTC
-Rama
Initial Update: Tuesday, 03 September 2019 07:52 UTC
We are aware of issues in creating metrics alerts on a subset of metrics for Application Gateway resources and are actively investigating. Some customers may experience failures in creating metric alerts on a subset of metrics for Application Gateway resources.
- Work Around: None
- Next Update: Before 09/03 10:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Rama
