Home
%3CLINGO-SUB%20id%3D%22lingo-sub-798705%22%20slang%3D%22en-US%22%3EExperiencing%20Data%20Latency%20issue%20in%20Azure%20Portal%20for%20Many%20Data%20Types%20-%2008%2F11%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-798705%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Sunday%2C%2011%20August%202019%2002%3A56%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%208%2F11%2C%20%3CSPAN%20style%3D%22text-align%3A%20left%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20text-transform%3A%20none%3B%20text-indent%3A%200px%3B%20letter-spacing%3A%20normal%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20text-decoration%3A%20none%3B%20word-spacing%3A%200px%3B%20display%3A%20inline%20!important%3B%20white-space%3A%20normal%3B%20orphans%3A%202%3B%20float%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%22%3E02%3A56%3C%2FSPAN%3EUTC.%20Our%20logs%20show%20the%20incident%20started%20on%20%3CSPAN%20style%3D%22text-align%3A%20left%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20text-transform%3A%20none%3B%20text-indent%3A%200px%3B%20letter-spacing%3A%20normal%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20text-decoration%3A%20none%3B%20word-spacing%3A%200px%3B%20display%3A%20inline%20!important%3B%20white-space%3A%20normal%3B%20orphans%3A%202%3B%20float%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%22%3E22%3A00%3C%2FSPAN%3E%3CFONT%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%26amp%3Bquot%3BSegoe%20UI%26amp%3Bquot%3B%2C%26amp%3Bquot%3BHelvetica%20Neue%26amp%3Bquot%3B%2C%26amp%3Bquot%3BApple%20Color%20Emoji%26amp%3Bquot%3B%2C%26amp%3Bquot%3BSegoe%20UI%20Emoji%26amp%3Bquot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3EUTC%3C%2FFONT%3Eand%20that%20during%20the%205%20hours%20that%20it%20took%20to%20resolve%20the%20issue%20could%20have%20impacted%20up%20to%2035%25%20of%20customers%20in%20the%20East%20US%20region%20who%20would%20have%20experienced%20data%20latency.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20Engineers%20determined%20that%20instances%20of%20a%20backend%20service%20became%20unhealthy%2C%20causing%20data%20latency%20within%20the%20Azure%20portal.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%204%20Hours%20%26amp%3B%2056%20minutes%20-%208%2F11%2C%202%3CSPAN%20style%3D%22background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20display%3A%20inline%3B%20float%3A%20none%3B%20font-family%3A%20%26amp%3Bquot%3BHelvetica%20Neue%26amp%3Bquot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3E2%3A00%3C%2FSPAN%3E%3CFONT%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%26amp%3Bquot%3BHelvetica%20Neue%26amp%3Bquot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3EUTC%3C%2FFONT%3Ethrough%20%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%26quot%3BHelvetica%20Neue%26quot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3E8%2F11%2C%20%3C%2FSPAN%3E%3CSPAN%20style%3D%22background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20display%3A%20inline%3B%20float%3A%20none%3B%20font-family%3A%20%26amp%3Bquot%3BHelvetica%20Neue%26amp%3Bquot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3E02%3A56%3C%2FSPAN%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%26quot%3BHelvetica%20Neue%26quot%3B%2CHelvetica%2CArial%2Csans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20400%3B%20letter-spacing%3A%20normal%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3EUTC%3C%2FSPAN%3E%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Ian%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Sunday%2C%2011%20August%202019%2001%3A48%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Application%20Insights.%20Root%20cause%20is%20not%20fully%20understood%20at%20this%20time.%20Some%20customers%20continue%20to%20experience%20data%20latency.%20We%20are%20working%20to%20establish%20the%20start%20time%20for%20the%20issue%2C%20initial%20findings%20indicate%20that%20the%20problem%20began%20at%2022%3A00%3CFONT%20face%3D%22%26quot%3BSegoe%20UI%26quot%3B%2C%26quot%3BHelvetica%20Neue%26quot%3B%2C%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2CHelvetica%2CArial%2Csans-serif%22%3E%3C%2FFONT%3Eon%2010%20Aug%202019.%20We%20currently%20have%20no%20estimate%20for%20resolution.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20none%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2008%2F11%2004%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Ian%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Sunday%2C%2011%20August%202019%2000%3A18%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Application%20Insights%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20Data%20Latency.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20none%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2008%2F11%2002%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Ian%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-798705%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-799215%22%20slang%3D%22en-US%22%3ERe%3A%20Experiencing%20Data%20Latency%20issue%20in%20Azure%20Portal%20for%20Many%20Data%20Types%20-%2008%2F11%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-799215%22%20slang%3D%22en-US%22%3E%3CP%3Ethanks%20.....%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D9%87%D8%A7%DB%8C%DA%A9%20%D9%88%DB%8C%DA%98%D9%86%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D9%86%D9%85%D8%A7%DB%8C%D9%86%D8%AF%DA%AF%DB%8C%20%D9%87%D8%A7%DB%8C%DA%A9%20%D9%88%DB%8C%DA%98%D9%86%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D9%87%D8%A7%DB%8C%D9%84%D9%88%DA%A9%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D9%86%D9%85%D8%A7%DB%8C%D9%86%D8%AF%DA%AF%DB%8C%20%D9%87%D8%A7%DB%8C%D9%84%D9%88%DA%A9%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D8%AF%D9%88%D8%B1%D8%A8%DB%8C%D9%86%20%D9%85%D8%AF%D8%A7%D8%B1%D8%A8%D8%B3%D8%AA%D9%87%20%D9%87%D8%A7%DB%8C%DA%A9%20%D9%88%DB%8C%DA%98%D9%86%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehikvision%3C%2FA%3E%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fcctvhikvision.center%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%D9%81%D8%B1%D9%88%D8%B4%DA%AF%D8%A7%D9%87%20%D9%87%D8%A7%DB%8C%DA%A9%20%D9%88%DB%8C%DA%98%D9%86%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Final Update: Sunday, 11 August 2019 02:56 UTC

We've confirmed that all systems are back to normal with no customer impact as of 8/11, 02:56 UTC. Our logs show the incident started on 22:00 UTC and that during the 5 hours that it took to resolve the issue could have impacted up to 35% of customers in the East US region who would have experienced data latency.
  • Root Cause: Engineers determined that instances of a backend service became unhealthy, causing data latency within the Azure portal.
  • Incident Timeline: 4 Hours & 56 minutes - 8/11, 22:00 UTC through 8/11, 02:56 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Ian

Update: Sunday, 11 August 2019 01:48 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers continue to experience data latency. We are working to establish the start time for the issue, initial findings indicate that the problem began at 22:00 on 10 Aug 2019. We currently have no estimate for resolution.
  • Work Around: none
  • Next Update: Before 08/11 04:00 UTC
-Ian

Initial Update: Sunday, 11 August 2019 00:18 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Latency.
  • Work Around: none
  • Next Update: Before 08/11 02:30 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Ian