Experiencing Latency and Data Loss for Application Insights (South central US) - 03/07 - Resolved

Published Mar 07 2022 11:37 AM 521 Views

Final Update: Tuesday, 07 March 2022  19:04 UTC

We've confirmed that all systems are back to normal in South Central US region. Our logs show the incident started on 3/7,  18:13 UTC and that during the 51 minutes that it took to resolve the issue, customers in South Central US region using Application Insights may have experienced data latency and data loss which could lead to delayed, missed or misfired Alerts.

  • Root Cause: We determined that one of our backend service became unhealthy. This resulted in the dependent service to be unavailable.
  • Incident Timeline: 51 minutes - 3/7, 18:13 UTC through 3/7, 19:04 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.


-Shwetha

%3CLINGO-SUB%20id%3D%22lingo-sub-3249536%22%20slang%3D%22en-US%22%3EExperiencing%20Latency%20and%20Data%20Loss%20for%20Application%20Insights%20(South%20central%20US)%20-%2003%2F07%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-3249536%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CP%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20margin-right%3A%200px%3B%20margin-bottom%3A%200px%3B%20margin-left%3A%200px%3B%20color%3A%20rgb(36%2C%2036%2C%2036)%3B%20font-family%3A%20%26quot%3BSegoe%20UI%26quot%3B%2C%20system-ui%2C%20%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%20%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2C%20sans-serif%3B%20font-size%3A%2014px%3B%22%3E%3CU%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3EFinal%20Update%3C%2FU%3E%3CSPAN%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3E%3A%20Tuesday%2C%2007%20March%202022%26nbsp%3B%2019%3A04%20UTC%3C%2FSPAN%3E%3CBR%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%20%2F%3E%3CBR%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%20%2F%3E%3CSPAN%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20in%20South%20Central%20US%20region.%20Our%20logs%20show%20the%20incident%20started%20on%203%2F7%2C%26nbsp%3B%2018%3A13%20UTC%20and%20that%20during%20the%2051%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%2C%20customers%20in%20South%20Central%20US%20region%20using%20Application%20Insights%20may%20have%20experienced%20data%20latency%20and%20data%20loss%20which%20could%20lead%20to%20delayed%2C%20missed%20or%20misfired%20Alerts.%3C%2FSPAN%3E%3CBR%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%20%2F%3E%3C%2FP%3E%3CUL%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20clear%3A%20left%3B%20font-family%3A%20%26quot%3BSegoe%20UI%26quot%3B%2C%20system-ui%2C%20%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%20%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2C%20sans-serif%3B%20font-size%3A%2014px%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3E%3CLI%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%22%3E%3CU%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%22%3ERoot%20Cause%3C%2FU%3E%3A%20We%20determined%20that%20one%20of%20our%20backend%20service%20became%20unhealthy.%20This%20resulted%20in%20the%20dependent%20service%20to%20be%20unavailable.%3C%2FLI%3E%3CLI%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%22%3E%3CU%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%22%3EIncident%20Timeline%3C%2FU%3E%3A%2051%20minutes%20-%203%2F7%2C%2018%3A13%20UTC%20through%203%2F7%2C%2019%3A04%20UTC%3C%2FLI%3E%3C%2FUL%3E%3CP%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20margin-right%3A%200px%3B%20margin-bottom%3A%200px%3B%20margin-left%3A%200px%3B%20color%3A%20rgb(36%2C%2036%2C%2036)%3B%20font-family%3A%20%26quot%3BSegoe%20UI%26quot%3B%2C%20system-ui%2C%20%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%20%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2C%20sans-serif%3B%20font-size%3A%2014px%3B%22%3E%3CSPAN%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3C%2FSPAN%3E%3C%2FP%3E%3CP%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20margin-right%3A%200px%3B%20margin-bottom%3A%200px%3B%20margin-left%3A%200px%3B%20color%3A%20rgb(36%2C%2036%2C%2036)%3B%20font-family%3A%20%26quot%3BSegoe%20UI%26quot%3B%2C%20system-ui%2C%20%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%20%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2C%20sans-serif%3B%20font-size%3A%2014px%3B%22%3E%3CSPAN%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20color%3A%20rgb(97%2C%2097%2C%2097)%3B%20font-style%3A%20inherit%3B%20font-weight%3A%20inherit%3B%22%3E%3CBR%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%20style%3D%22-webkit-font-smoothing%3A%20antialiased%3B%20margin-right%3A%200px%3B%20margin-bottom%3A%200px%3B%20margin-left%3A%200px%3B%20font-family%3A%20%26quot%3BSegoe%20UI%26quot%3B%2C%20system-ui%2C%20%26quot%3BApple%20Color%20Emoji%26quot%3B%2C%20%26quot%3BSegoe%20UI%20Emoji%26quot%3B%2C%20sans-serif%3B%20font-size%3A%2014px%3B%22%3E-Shwetha%3C%2FP%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Version history
Last update:
‎Mar 07 2022 11:37 AM
Updated by: