%3CLINGO-SUB%20id%3D%22lingo-sub-1722475%22%20slang%3D%22en-US%22%3EAzure%20Lab%20Services%20Outage%3A%20Certain%20lab%20operations%20may%20not%20work%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1722475%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20would%20like%20to%20inform%20you%20that%20we%20are%20investigating%20an%20issue%20where%20certain%20lab%20operations%20may%20fail.%20Based%20on%20our%20current%20investigation%2C%20this%20is%20happening%20due%20to%20an%20outage%20within%20the%20Azure%20Active%20Directory%20service%20and%20is%20non-regional.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EMore%20details%20on%20the%20outage%20here%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAs%20of%2022%3A40%20UTC%2C%20following%20lab%20operations%20may%20not%20work%20as%20expected%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3ELabs%20Account%20owners%20may%20not%20be%20able%20to%20sign%20in%20to%20the%20Azure%20portal%3C%2FLI%3E%0A%3CLI%3ELabs%20owners%2Fusers%20may%20not%20be%20able%20to%20sign%20in%20to%20Azure%20Lab%20Services(labs.azure.com)%20portal%20and%2For%20their%20virtual%20machines%26nbsp%3B%26nbsp%3B%3C%2FLI%3E%0A%3CLI%3ELab%20schedules%20may%20fail%20to%20run%20as%20expected%3C%2FLI%3E%0A%3CLI%3EIf%20the%20VM's%20are%20currently%20running%2C%20enabled%20auto-shutdown%20features(Idle%2C%20etc.)%20may%20not%20be%20triggered%20as%20expected%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EWe%20will%20continue%20to%20monitor%20for%20a%20fix%20and%20post%20updates%20here.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWe%20apologize%20for%20the%20inconvenience%20and%20thank%20you%20for%20your%20patience.%20Please%20add%20a%20comment%20for%20any%20questions.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E-%20Azure%20Lab%20Services%20Team%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1722475%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20would%20like%20to%20inform%20you%20that%20we%20are%20investigating%20an%20issue%20where%20certain%20lab%20operations%20may%20fail.%20Based%20on%20our%20current%20investigation%2C%20this%20is%20happening%20due%20to%20an%20outage%20within%20the%20Azure%20Active%20Directory%20service%20and%20is%20non-regional.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EMore%20details%20on%20the%20outage%20here%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1722475%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ELab%20Services%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1722634%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Lab%20Services%20Outage%3A%20Certain%20lab%20operations%20may%20not%20work%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1722634%22%20slang%3D%22en-US%22%3E%3CP%3EQuick%20Update%20-%20Authentication%20failures%20are%20showing%20signs%20of%20recovery%20and%20we%20see%20customers%20successfully%20logging%20into%20both%20Azure%20Portal%20and%20Azure%20Lab%20Services.%20Here%20is%20the%20latest%20status%20-%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstatus.azure.com%2Fen-us%2Fstatus%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E
Microsoft

Certain lab operations might have failed for a subset of customers approximately between 21:25 UTC on Sep 28 2020 and 00:23 UTC on Sep 29 2020. This has happened due to an outage within the Azure Active Directory service and was non-regional. The outage is now mitigated and customers shouldn't be facing any issues!

 

More details on the outage here: https://status.azure.com/en-us/status/history/

 

Following lab operations might not have worked as expected between 21:25 UTC on Sep 28 2020 and 00:23 UTC on Sep 29 2020 -

  • Labs Account owners might not have been able to sign in to the Azure portal
  • Labs owners/users might not have been able to sign in to Azure Lab Services(labs.azure.com) portal and/or their virtual machines  
  • Lab schedules might have failed to run as expected
  • If the VM's were running, enabled auto-shutdown features(Idle, etc.) might not have been triggered as expected

In general, please verify and shut-down any VM's that might have not been shutdown because of the schedule or auto-shutdown trigger failures to stop accruing additional costs.

 

We apologize for the inconvenience and thank you for your patience. Please add a comment for any questions. 

 

- Azure Lab Services Team 

2 Comments
Microsoft

Quick Update - Authentication failures are showing signs of recovery and we see customers successfully logging into both Azure Portal and Azure Lab Services. Here is the latest status - https://status.azure.com/en-us/status

Microsoft

Authentication failures are now mitigated and the blog post has been updated accordingly! Please feel free to comment if you have any questions. Sincere apologies for any inconvenience caused!