%3CLINGO-SUB%20id%3D%22lingo-sub-963126%22%20slang%3D%22en-US%22%3ERe%3A%20Experiencing%20Issues%20in%20Application%20Insights%20services%20-%2010%2F29%20-%20Investigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-963126%22%20slang%3D%22en-US%22%3E%3CP%3EPerhaps%20%3CA%20href%3D%22https%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%3C%2FA%3E%20should%20be%20updated%20then%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-963000%22%20slang%3D%22en-US%22%3EExperiencing%20Issues%20in%20Application%20Insights%20services%20-%2010%2F29%20-%20Mitigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-963000%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Thursday%2C%2031%20October%202019%2010%3A24%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20determined%20that%20this%20issue%20was%20resolved%20temporarily%20on%2010%2F29%20when%20resource%20provisioning%20service%20was%20brought%20to%20healthy%20state.%20We%20continue%20to%20have%20occasional%20recurrences%20of%20this%20issue%20which%20would%20surface%20to%20customers%20as%20failures%20while%20creating%20or%20managing%20components%2C%20web%20tests%2C%20or%20classic%20alert%20rules.%20Engineers%20have%20identified%20a%20fix%2C%20but%20it%20might%20take%20~24%20hours%20to%20deploy%20in%20all%20the%20locations.%20We%20provide%20an%20update%20as%20we%20make%20progress%20on%20deployments.%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CBR%20%2F%3E-Naresh%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Wednesday%2C%2030%20October%202019%2017%3A40%20UTC%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CBR%20%2F%3E%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3EWe%20determined%20that%20this%20issue%20was%20resolved%20temporarily%20on%2010%2F29%20when%20resource%20provisioning%20service%20was%20brought%20to%20healthy%20state.%20We%20continue%20to%20have%20occasional%20recurrences%20of%20this%20issue%20which%20would%20surface%20to%20customers%20as%20failures%20while%20creating%20or%20managing%20components%2C%20web%20tests%2C%20or%20classic%20alert%20rules.%20Engineers%20have%20identified%20a%20fix%2C%20but%20it%20might%20take%20~24%20hours%20to%20deploy%20in%20all%20the%20locations.%20We%20provide%20an%20update%20as%20we%20make%20progress%20on%20deployments.%3CBR%20%2F%3E%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CBR%20%2F%3E%3C%2FDIV%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E-Arvind%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Tuesday%2C%2029%20October%202019%2016%3A50%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20as%20of%2010%2F29%2C%2016%3A30%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2010%2F29%2C%2015%3A50%20UTC%20and%20that%20during%20the%2040%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20customers%20would%20have%20experienced%20failures%20while%20creating%20new%20resources%20or%20accessing%20existing%20resources.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20unhealthy%20component%20which%20is%20responsible%20for%20resource%20provisioning.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%26nbsp%3B%2040%20minutes%20-%2010%2F29%2C%2015%3A50%20UTC%20through%2010%2F29%2C%2016%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Arvind%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Tuesday%2C%2029%20October%202019%2016%3A32%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Application%20Insights%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20failures%20while%20creating%20new%20resources%20as%20well%20as%20while%20accessing%20their%20existing%20resources.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%3CNONE%20or%3D%22%22%20details%3D%22%22%3E%3C%2FNONE%3E%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2010%2F29%2019%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Arvind%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-963000%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Thursday, 31 October 2019 15:02 UTC

There were no reoccurrences from this intermittent issue after 10/30 at 21:00 UTC and all services are functioning normally at this time.

-Matt

Update: Thursday, 31 October 2019 10:24 UTC

We determined that this issue was resolved temporarily on 10/29 when resource provisioning service was brought to healthy state. We continue to have occasional recurrences of this issue which would surface to customers as failures while creating or managing components, web tests, or classic alert rules. Engineers have identified a fix, but it might take ~24 hours to deploy in all the locations. We provide an update as we make progress on deployments.

-Naresh

Update: Wednesday, 30 October 2019 17:40 UTC

We determined that this issue was resolved temporarily on 10/29 when resource provisioning service was brought to healthy state. We continue to have occasional recurrences of this issue which would surface to customers as failures while creating or managing components, web tests, or classic alert rules. Engineers have identified a fix, but it might take ~24 hours to deploy in all the locations. We provide an update as we make progress on deployments.

-Arvind

Final Update: Tuesday, 29 October 2019 16:50 UTC

We've confirmed that all systems are back to normal as of 10/29, 16:30 UTC. Our logs show the incident started on 10/29, 15:50 UTC and that during the 40 minutes that it took to resolve the issue customers would have experienced failures while creating new resources or accessing existing resources.
  • Root Cause: The failure was due to unhealthy component which is responsible for resource provisioning.
  • Incident Timeline:  40 minutes - 10/29, 15:50 UTC through 10/29, 16:30 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Arvind

Initial Update: Tuesday, 29 October 2019 16:32 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience failures while creating new resources as well as while accessing their existing resources.
  • Work Around:
  • Next Update: Before 10/29 19:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Arvind

1 Comment
Occasional Visitor

Perhaps https://status.azure.com/en-us/status should be updated then?