Home
%3CLINGO-SUB%20id%3D%22lingo-sub-633840%22%20slang%3D%22en-US%22%3EExperiencing%20Data%20Latency%20issue%20in%20Azure%20Portal%20for%20Many%20Data%20Types%20-%2005%2F22%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-633840%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Thursday%2C%2023%20May%202019%2018%3A12%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%205%2F23%2C%2018%3A10%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2005%2F21%2C%2016%3A00%20UTC%20and%20that%20during%20the%202%20days%20%2C%202%20hours%20and%2010%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20in%20West%20US%202%20Region%20experienced%20Data%20Latency%20for%20Continuous%20Export.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20issue%20with%20network%20device.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%202%20days%2C%202%20hours%20and%2010%20minutes%20-%2005%2F21%2C%2016%3A00%20UTC%20through%2005%2F23%2C%2018%3A10%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Jayadev%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Thursday%2C%2023%20May%202019%2001%3A50%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3ERoot%20cause%20has%20been%20isolated%20to%20be%20an%20issue%20with%20network%20device%20which%20was%20impacting%20Data%20Latency%20for%20Continuous%20Export%20in%20West%20US%202%20Region.%20To%20address%20this%20issue%20we%20had%20to%20isolate%20the%20network%20device.%20Continuous%20Export%20is%20still%20having%20Latency%20in%20West%20US%202%20Region%20with%20delay%20being%204%20hours%20and%20we%20are%20seeing%20improvements%20in%20our%20telemetry.%20Some%20customers%20may%20still%20experience%20Data%20Latency%20and%20we%20estimate%2012%20hours%20before%20all%20Latency%20is%20addressed.%3CUL%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F23%2020%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Jayadev%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Wednesday%2C%2022%20May%202019%2023%3A00%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Application%20Insights.%20Root%20cause%20is%20not%20fully%20understood%20at%20this%20time.%20Some%20customers%20continue%20to%20experience%20%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20transparent%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%26quot%3BHelvetica%20Neue%26quot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3EData%20Latency%20for%20Continuous%20Export%20in%20West%20US%202%20region%20with%20delay%20being%203%20hours%20and%2030%20minutes%3C%2FSPAN%3E.%20Initial%20findings%20indicate%20the%20problem%20began%20at%2005%2F21%2016%3A00%20UTC.%20We%20currently%20have%20no%20estimate%20for%20resolution.%3CUL%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F23%2005%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Jayadev%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Wednesday%2C%2022%20May%202019%2019%3A14%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Application%20Insights.%20Root%20cause%20is%20not%20fully%20understood%20at%20this%20time.%20Some%20customers%20continue%20to%20experience%20Data%20Latency%20for%20Continuous%20Export%20in%20West%20US%202%20region%20with%20delay%20being%203%20hours%20and%2030%20minutes%20.%20We%20are%20working%20to%20establish%20the%20start%20time%20for%20the%20issue%2C%20initial%20findings%20indicate%20that%20the%20problem%20began%20at%2005%2F21%2016%3A00%20UTC.%20We%20currently%20have%20no%20estimate%20for%20resolution.%3CUL%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F22%2023%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Jayadev%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Wednesday%2C%2022%20May%202019%2016%3A15%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Application%20Insights%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20Data%20Latency%20for%20Continuous%20Export%20in%20West%20US%202%20region%20with%20a%20delay%20of%203%20hours%20and%2030%20minutes.%3CUL%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2005%2F22%2019%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Jayadev%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-633840%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Thursday, 23 May 2019 18:12 UTC

We've confirmed that all systems are back to normal with no customer impact as of 5/23, 18:10 UTC. Our logs show the incident started on 05/21, 16:00 UTC and that during the 2 days , 2 hours and 10 minutes that it took to resolve the issue some customers in West US 2 Region experienced Data Latency for Continuous Export.
  • Root Cause: The failure was due to issue with network device.
  • Incident Timeline: 2 days, 2 hours and 10 minutes - 05/21, 16:00 UTC through 05/23, 18:10 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Jayadev

Update: Thursday, 23 May 2019 01:50 UTC

Root cause has been isolated to be an issue with network device which was impacting Data Latency for Continuous Export in West US 2 Region. To address this issue we had to isolate the network device. Continuous Export is still having Latency in West US 2 Region with delay being 4 hours and we are seeing improvements in our telemetry. Some customers may still experience Data Latency and we estimate 12 hours before all Latency is addressed.
  • Next Update: Before 05/23 20:00 UTC
-Jayadev

Update: Wednesday, 22 May 2019 23:00 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers continue to experience Data Latency for Continuous Export in West US 2 region with delay being 3 hours and 30 minutes. Initial findings indicate the problem began at 05/21 16:00 UTC. We currently have no estimate for resolution.
  • Next Update: Before 05/23 05:00 UTC
-Jayadev

Update: Wednesday, 22 May 2019 19:14 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers continue to experience Data Latency for Continuous Export in West US 2 region with delay being 3 hours and 30 minutes . We are working to establish the start time for the issue, initial findings indicate that the problem began at 05/21 16:00 UTC. We currently have no estimate for resolution.
  • Next Update: Before 05/22 23:30 UTC
-Jayadev

Initial Update: Wednesday, 22 May 2019 16:15 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Latency for Continuous Export in West US 2 region with a delay of 3 hours and 30 minutes.
  • Next Update: Before 05/22 19:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Jayadev