Home
%3CLINGO-SUB%20id%3D%22lingo-sub-991350%22%20slang%3D%22en-US%22%3ERe%3A%20Experiencing%20Issues%20for%20Azure%20Monitor%20services%20in%20West%20Europe%20-%2011%2F07%20-%20Investigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-991350%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F283621%22%20target%3D%22_blank%22%3E%40Azure-Monitor-Team%3C%2FA%3E%26nbsp%3B%20Any%20update%20on%20this%20%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-991718%22%20slang%3D%22en-US%22%3ERe%3A%20Experiencing%20Issues%20for%20Azure%20Monitor%20services%20in%20West%20Europe%20-%2011%2F07%20-%20Investigating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-991718%22%20slang%3D%22en-US%22%3E%3CP%3EPlease%20update%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%3C%2FA%3E%26nbsp%3Bif%20your%20service%20is%20not%20behaving%20as%20expected.%20Thanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-990570%22%20slang%3D%22en-US%22%3EExperiencing%20Issues%20for%20Azure%20Monitor%20services%20in%20West%20Europe%20-%2011%2F07%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-990570%22%20slang%3D%22en-US%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EFinal%20Update%3C%2FU%3E%3A%20Thursday%2C%2007%20November%202019%2012%3A29%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe've%20confirmed%20that%20all%20systems%20are%20back%20to%20normal%20with%20no%20customer%20impact%20as%20of%2011%2F07%2C%2012%3A50%20UTC.%20Our%20logs%20show%20the%20incident%20started%20on%2011%2F06%2C%2021%3A10%20UTC%20and%20that%20during%20the%2015%20hours%20and%2040%20minutes%20that%20it%20took%20to%20resolve%20the%20issue%20some%20customers%20using%20Azure%20Monitor%20Services%20in%20West%20Europe%20may%20have%20experienced%20error%20while%20querying%20or%2Fand%20ingesting%20data%20along%20with%20alerts%20failures%20or%20latent%20alerts.%20Customers%20who%20are%20using%20Service%20Map%20in%20West%20Europe%20may%20have%20also%20seen%20Ingestion%20delays%20and%20latency..%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3ERoot%20Cause%3C%2FU%3E%3A%20The%20failure%20was%20due%20to%20azure%20storage%20outage%20in%20West%20Europe%20region.%3C%2FLI%3E%3CLI%3E%3CU%3EIncident%20Timeline%3C%2FU%3E%3A%2015%20Hours%20%26amp%3B%2040%20minutes%20-%2011%2F06%2C%2021%3A10%20UTC%20through%2011%2F07%2C%2012%3A50%20UTC%3CBR%20%2F%3E%3C%2FLI%3E%3C%2FUL%3EWe%20understand%20that%20customers%20rely%20on%20Azure%20Log%20Analytics%20as%20a%20critical%20service%20and%20apologize%20for%20any%20impact%20this%20incident%20caused.%3CBR%20%2F%3E%3CBR%20%2F%3E-Mohini%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Thursday%2C%2007%20November%202019%2008%3A20%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Azure%20monitor%20services.%20This%20issue%20started%20since%2011%2F06%2F2019%2021.10%20UTC%20and%20is%20caused%20by%20our%20dependency%20storage%20system.%20Our%20team%20has%20been%20investigating%20this%20with%20partner%20Azure%20storage%20team%20but%20we%20do%20not%20have%20any%20root%20cause%20identified%20yet.%20Customers%20using%20Azure%20Monitor%20Services%20in%20West%20Europe%20may%20experience%20error%20while%20querying%20or%2Fand%20ingesting%20data%20along%20with%20alerts%20failures%20or%20latent%20alerts.%20Customers%20who%20are%20using%20Service%20Map%20in%20West%20Europe%20may%20also%20experience%20Ingestion%20delays%20and%20latency.%20We%20provide%20an%20update%20as%20we%20learn.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2011%2F07%2012%3A30%20UTC%3C%2FLI%3E%3C%2FUL%3E-Mohini%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%3EUpdate%3C%2FU%3E%3A%20Thursday%2C%2007%20November%202019%2004%3A55%20UTC%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20continue%20to%20investigate%20issues%20within%20Azure%20monitor%20services.%20This%20issue%20started%20since%2011%2F06%2F2019%2021.10%20UTC%20and%20is%20caused%20by%20our%20dependency%20storage%20system.%20Our%20team%20has%20been%20investigating%20this%20with%20partner%20Azure%20storage%20team%20but%20we%20do%20not%20have%20any%20root%20cause%20identified%20yet.%20Customers%20using%20Azure%20Monitor%20Services%20in%20West%20Europe%20may%20experience%20error%20while%20querying%20or%2Fand%20ingesting%20data%20along%20with%20alerts%20failures%20or%20latent%20alerts.%20Customers%20who%20are%20using%20Service%20Map%20in%20West%20Europe%20may%20also%20experience%20Ingestion%20delays%20and%20latency.%20We%20provide%20an%20update%20as%20we%20learn.%3CBR%20%2F%3E%3CUL%3E%3CLI%3E%3CU%3EWork%20Around%3C%2FU%3E%3A%20None%3C%2FLI%3E%3CLI%3E%3CU%3ENext%20Update%3C%2FU%3E%3A%20Before%2011%2F07%2008%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E-Mohini%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CDIV%20style%3D%22font-size%3A14px%3B%22%3E%3CU%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3EInitial%20Update%3C%2FU%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3E%3A%20Thursday%2C%2007%20November%202019%2000%3A49%20UTC%3C%2FSPAN%3E%3CBR%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%20%2F%3E%3CBR%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%20%2F%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3EWe%20are%20aware%20of%20issues%20within%20Log%20Analytics%20and%20are%20actively%20investigating.%20Some%20customers%20may%20experience%20data%20access%20issues%20for%20Log%20Analytics%20and%20also%20issues%20with%20Log%20alerts%20not%20being%20triggered%20as%20expected%20in%20West%20Europe%20region.%26nbsp%3B%3C%2FSPAN%3E%3CBR%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%20%2F%3E%3CUL%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3E%3CLI%20style%3D%22box-sizing%3A%20border-box%3B%22%3E%3CU%20style%3D%22box-sizing%3A%20border-box%3B%22%3EWork%20Around%3C%2FU%3E%3A%20None%26nbsp%3B%3C%2FLI%3E%3CLI%20style%3D%22box-sizing%3A%20border-box%3B%22%3E%3CU%20style%3D%22box-sizing%3A%20border-box%3B%22%3ENext%20Update%3C%2FU%3E%3A%20Before%2011%2F07%2003%3A00%20UTC%3C%2FLI%3E%3C%2FUL%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3EWe%20are%20working%20hard%20to%20resolve%20this%20issue%20and%20apologize%20for%20any%20inconvenience.%3C%2FSPAN%3E%3CBR%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%20%2F%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20rgb(255%2C%20255%2C%20255)%3B%20color%3A%20rgb(0%2C%200%2C%200)%3B%20font-family%3A%20%22%20helvetica%3D%22%22%20neue%3D%22%22%3E-Sindhu%3C%2FSPAN%3E%3CB%3E%3C%2FB%3E%3CI%3E%3C%2FI%3E%3CU%3E%3C%2FU%3E%3CSUB%3E%3C%2FSUB%3E%3CSUP%3E%3C%2FSUP%3E%3CSTRIKE%3E%3C%2FSTRIKE%3E%3CBR%20%2F%3E%3C%2FDIV%3E%3CHR%20style%3D%22border-top-color%3Alightgray%22%20%2F%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-990570%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EActivity%20Log%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ELog%20Search%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMetric%20Alerts%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EService%20Map%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESmart%20Diagnostics%20Alerts%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-994418%22%20slang%3D%22en-US%22%3ERe%3A%20Experiencing%20Issues%20for%20Azure%20Monitor%20services%20in%20West%20Europe%20-%2011%2F07%20-%20Resolved%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-994418%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20are%20still%20experiencing%20issues%20with%20this.%20We%20have%20no%20data%20after%2018%3A00%20UTC%207th%20November.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Final Update: Thursday, 07 November 2019 12:29 UTC

We've confirmed that all systems are back to normal with no customer impact as of 11/07, 12:50 UTC. Our logs show the incident started on 11/06, 21:10 UTC and that during the 15 hours and 40 minutes that it took to resolve the issue some customers using Azure Monitor Services in West Europe may have experienced error while querying or/and ingesting data along with alerts failures or latent alerts. Customers who are using Service Map in West Europe may have also seen Ingestion delays and latency..
  • Root Cause: The failure was due to azure storage outage in West Europe region.
  • Incident Timeline: 15 Hours & 40 minutes - 11/06, 21:10 UTC through 11/07, 12:50 UTC
We understand that customers rely on Azure Log Analytics as a critical service and apologize for any impact this incident caused.

-Mohini

Update: Thursday, 07 November 2019 08:20 UTC

We continue to investigate issues within Azure monitor services. This issue started since 11/06/2019 21.10 UTC and is caused by our dependency storage system. Our team has been investigating this with partner Azure storage team but we do not have any root cause identified yet. Customers using Azure Monitor Services in West Europe may experience error while querying or/and ingesting data along with alerts failures or latent alerts. Customers who are using Service Map in West Europe may also experience Ingestion delays and latency. We provide an update as we learn.
  • Work Around: None
  • Next Update: Before 11/07 12:30 UTC
-Mohini

Update: Thursday, 07 November 2019 04:55 UTC

We continue to investigate issues within Azure monitor services. This issue started since 11/06/2019 21.10 UTC and is caused by our dependency storage system. Our team has been investigating this with partner Azure storage team but we do not have any root cause identified yet. Customers using Azure Monitor Services in West Europe may experience error while querying or/and ingesting data along with alerts failures or latent alerts. Customers who are using Service Map in West Europe may also experience Ingestion delays and latency. We provide an update as we learn.
  • Work Around: None
  • Next Update: Before 11/07 08:00 UTC
-Mohini

Initial Update: Thursday, 07 November 2019 00:49 UTC

We are aware of issues within Log Analytics and are actively investigating. Some customers may experience data access issues for Log Analytics and also issues with Log alerts not being triggered as expected in West Europe region. 
  • Work Around: None 
  • Next Update: Before 11/07 03:00 UTC
We are working hard to resolve this issue and apologize for any inconvenience.
-Sindhu

3 Comments
Frequent Visitor

@Azure-Monitor-Team  Any update on this ?

Occasional Visitor

Please update https://status.azure.com/en-us/status if your service is not behaving as expected. Thanks!

Occasional Visitor

We are still experiencing issues with this. We have no data after 18:00 UTC 7th November.