Experiencing Continous Export failures - 01/18 - Mitigated

[Azure-Monitor-Team]

First published on MSDN on Jan 19, 2019

Final Update: Saturday, 19 January 2019 00:44 UTC

We've confirmed that Continuous Exports works as expected after applying the work around. Our logs show the incident started on 1/16 12:25 PM UTC and that during the 2 days that it took to resolve the issue <1% of customers experienced intermittent failures in Continuous Export service.

• Root Cause: The failure was due to Storage account issues that led to export failures.
• Lessons Learned: If the issue re-occurs the issue can we resolved by applying workaround steps.
• Incident Timeline:  During the period of 1/16 to 1/18, around 1% of customers could have experienced intermittent export failures due to storage access issues.

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-v-anurud

Update: Friday, 18 January 2019 21:30 UTC

We continue to investigate issues within Application Insights. Impacted customers must apply workaround to resolve failures in export like creating new container, reconfiguring and after some buffer time go back to old container. We will keep investigating for any failures even after applying this workaround.

• Work Around: Creating new container, reconfigure Export with new container, wait for status to turn green and go back to old container.
• Next Update: Before 01/19 03:30 UTC

-v-anurud

Update: Friday, 18 January 2019 03:56 UTC

We continue to investigate issues within Application Insights. Impacted customers must apply workaround to resolve failures in export like creating new container, reconfiguring and after some buffer time go back to old container. We will keep investigating for any failures even after applying this workaround.

• Work Around: Creating new container, reconfigure Export with new container, wait for status to turn green and go back to old container.
• Next Update: Before 01/18 22:00 UTC

-v-anurud

Update: Thursday, 17 January 2019 22:32 UTC

We continue to investigate issues within Application Insights. Root cause could be due to Storage account access issue. Some customers continue to experience Continuous Export failures. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 01/18 03:00 UTC

-v-anurud

Update: Thursday, 17 January 2019 02:41 UTC

We continue to investigate issues within Application Insights. Some customers continue to experience failures in Continuous export. Initial findings indicate that the problem could be due to access issue. We currently have no estimate for resolution.

• Work Around: None
• Next Update: Before 01/17 21:00 UTC

-v-anurud

Update: Thursday, 17 January 2019 00:18 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers continue to experience failures in Continuous Export. We are working to establish the start time for the issue, initial findings indicate that the problem began at 1/16 12:25 PM UTC. We currently have no estimate for resolution.

• Work Around: none
• Next Update: Before 01/17 03:30 UTC

-v-anurud

Initial Update: Wednesday, 16 January 2019 22:19 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data access issue in Continuous Export.

• Work Around: none
• Next Update: Before 01/17 00:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-v-anurud