%3CLINGO-SUB%20id%3D%22lingo-sub-1317369%22%20slang%3D%22en-US%22%3EExperiencing%20Data%20Gaps%20issue%20in%20Azure%20Monitor%20for%20Many%20Data%20Types%20-%2004%2F18%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1317369%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Saturday%2C%2018%20April%202020%2003%3A34%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%204%2F18%2C%203%3A30%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%204%2F17%2C%2022%3A50%20UTC%20and%20that%20during%20the%204.5%20hours%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20experienced%20data%20access%20issues%2C%20data%20latency%2C%20data%20loss%2C%20misfired%20alerts%20or%20alerts%20not%20firing%20for%20their%20resources.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20bug%20in%20back%20end%20authentication%20service.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%204%20Hours%20%26amp%3B%2040%20minutes%20-%204%2F17%2C%2022%3A50%20UTC%20through%204%2F18%2C%203%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Application%20Insights%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Anupama%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Saturday%2C%2018%20April%202020%2002%3A15%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3ERoot%20cause%20has%20been%20isolated%20to%20an%20intermediate%20service%20having%20authentication%20failures%20which%20was%20impacting%20metric%20alerts%2C%20custom%20metrics%2C%20auto-scale.%20To%20address%20this%20issue%20engineers%20have%20provided%20a%20substitute%20authentication.%26nbsp%3B%20Some%20customers%20may%20experience%20data%20gaps%2C%20data%20latency%20while%20accessing%20or%20querying%20their%20metrics%20data.%20We%20estimate%20~1%20hour%20before%20all%20issues%20are%20addressed.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2004%2F18%2004%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Anupama%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EInitial%20Update%3C%2FU%3E%3A%20Saturday%2C%2018%20April%202020%2000%3A41%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20are%20aware%20of%20issues%20within%20Application%20Insights%2C%20Smart%20alerts%2C%20Metric%20alerts%2C%20Auto-scale%20and%20are%20actively%20investigating.%20Some%20customers%20accessing%20or%20querying%20their%20metrics%20data%20may%20experience%20data%20gaps.%20%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2004%2F18%2004%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3CBR%20%2F%3E-Anupama%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1317369%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESmart%20Diagnostics%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Final Update: Saturday, 18 April 2020 03:34 UTC

We've confirmed that all systems are back to normal with no customer impact as of 4/18, 3:30 UTC. Our logs show the incident started on 4/17, 22:50 UTC and that during the 4.5 hours that it took to resolve the issue some customers experienced data access issues, data latency, data loss, misfired alerts or alerts not firing for their resources.
  • Root Cause: The failure was due to bug in back end authentication service.
  • Incident Timeline: 4 Hours & 40 minutes - 4/17, 22:50 UTC through 4/18, 3:30 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Anupama

Update: Saturday, 18 April 2020 02:15 UTC

Root cause has been isolated to an intermediate service having authentication failures which was impacting metric alerts, custom metrics, auto-scale. To address this issue engineers have provided a substitute authentication.  Some customers may experience data gaps, data latency while accessing or querying their metrics data. We estimate ~1 hour before all issues are addressed.
  • Work Around: None
  • Next Update: Before 04/18 04:30 UTC
-Anupama

Initial Update: Saturday, 18 April 2020 00:41 UTC

We are aware of issues within Application Insights, Smart alerts, Metric alerts, Auto-scale and are actively investigating. Some customers accessing or querying their metrics data may experience data gaps.
  • Work Around: None
  • Next Update: Before 04/18 04:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Anupama