SOLVED
Home

Azure Activity Log missing legacy auth failed attempts or account lockouts for AAD Powershell

%3CLINGO-SUB%20id%3D%22lingo-sub-133844%22%20slang%3D%22en-US%22%3EAzure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-133844%22%20slang%3D%22en-US%22%3E%3CP%3EIn%20my%20testing%2C%20I%20am%20not%20seeing%20any%20logging%20of%20failed%20attempts%20or%20account%20lockouts%20in%20the%20Azure%20Active%20Directory%20Activity%20Sign-In%20Logs%20when%20the%20legacy%20module%20of%20Azure%20Active%20Directory%20is%20used.%3C%2FP%3E%0A%3CP%3EModern%20Authentication%20clients%20are%20logged%20for%20failed%20sign-ins%20and%20account%20lockouts%2C%20but%20not%20when%20legacy%20authentication%20is%20used%20in%20the%20Azure%20Active%20Directory%20powershell%20module.%3C%2FP%3E%0A%3CP%3EThe%20risk%20%2F%20concern%20here%20is%20that%20attackers%20can%20go%20undetected%20in%20their%20brute%20force%20attempts.%3C%2FP%3E%0A%3CP%3EI%20realize%20the%20risk%20is%20small%2C%20since%20accounts%20are%20locked%20out%20after%2010%20invalid%20attempts%2C%20however%2C%20it%20would%20still%20be%20nice%20to%20have%20visibility.%3C%2FP%3E%0A%3CP%3EAccording%20to%20the%20documentation%2C%20it%20can%20take%20up%20to%208%20hours%20for%20legacy%20apps%20to%20show%20up%20in%20the%20logs%2C%20however%2C%20I%20have%20waited%2012%20hours%20and%20I%20still%20see%20no%20sign%20of%20my%20simulated%20brute%20force%20activity.%3C%2FP%3E%0A%3CP%3E%22For%20some%20sign-ins%20activity%20data%20coming%20from%20legacy%20office%20applications%2C%20it%20can%20take%20to%208%20hours%20for%20the%20reporting%20data%20to%20show%20up%22%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Factive-directory%2Factive-directory-reporting-latencies-azure-portal%23%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Factive-directory%2Factive-directory-reporting-latencies-azure-portal%23%3C%2FA%3E%3C%2FP%3E%0A%3CP%3EThe%20note%20above%20says%20%22legacy%20office%20applications%22%20so%20I%20assume%20that%20Azure%20AD%20Powershell%20would%20fit%20into%20that%20category%20rather%20than%20not%20logging%20any%20activity%20at%20all%20from%20it.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-133844%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3Eauditing%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3Elogging%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EPowerShell%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-136094%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-136094%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20Joe.%20I%20guess%20the%20events%20I'm%20seeing%20are%20ExO%20PowerShell%20then%2C%20or%20something%20else.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAnd%20Microsoft%2C%20really%3F!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-136064%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-136064%22%20slang%3D%22en-US%22%3EUpdate%2012%2F11%2F2017%20-%20Microsoft%20Premier%20Support%20said%20this%20is%20working%20%22by%20design%22%20and%20will%20either%20provide%20a%20public%20facing%20article%20that%20states%20this%20and%2For%20will%20open%20a%20%22Design%20Change%20Request%22%20to%20log%20these%20legacy%20authentication%20failure%20events.%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-134370%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-134370%22%20slang%3D%22en-US%22%3E%3CP%3ELooking%20at%20the%20tests%20I%20did%2C%20it%20definitely%20does%20not%20log%20every%20single%20failed%20attempt.%20I%20did%20get%20one%20additional%20entry%20from%20today%2C%20which%20seems%20to%20correspond%20to%20the%20dozen%20or%20so%20attempts%20I%20made.%20And%20I'm%20starting%20to%20think%20that%20you%20are%20correct%20here%20-%20I%20was%20expecting%20at%20least%20two%20different%20types%20of%20%22applications%22%20reported%20as%20I%20tried%20both%20the%20MSOnline%20module%20and%20(legacy%20auth)%20ExO%20PowerShell%2C%20but%20only%20one%20of%20them%20is%20visible%20-%20which%20I%20guess%20is%20the%20ExO%20failure%2C%20not%20the%20MSOnline%20one.%20Let%20us%20know%20if%20you%20get%20an%20official%20answer%20please.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-134228%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-134228%22%20slang%3D%22en-US%22%3E%3CP%3EThe%20downloaded%20report%20is%20also%20missing%20legacy%20authentication%20from%20Azure%20AD%20Powershell.%3C%2FP%3E%0A%3CP%3EAfter%20simulating%20brute%20force%20against%20three%20separate%20AAD%20tenants%20using%20legacy%20powershell%20and%20waiting%20more%20than%2012%20hours%26nbsp%3Bafter%20each%20attempt%2C%20I%20have%20yet%20to%20find%20a%20case%20where%20it%20is%20a%20logged%20event.%20I'll%20open%20a%20ticket%20with%20MSFT%20Support%20to%20see%20if%20they%20are%20aware%20of%20this.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-134108%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-134108%22%20slang%3D%22en-US%22%3E%3CP%3EThe%20UI%20is%20pretty%20crappy%20if%20you%20ask%20me%2C%20simply%20download%20the%20report%20and%20work%20with%20it.%20Then%20again%2C%20I%20might%20simply%20be%20wrong%20and%20the%20%22Office%20365%22%20app%20logins%20I'm%20seeing%20might%20correspond%20to%20something%20else.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-133993%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-133993%22%20slang%3D%22en-US%22%3E%3CP%3Einteresting%20so%20far%20in%20two%20separate%20tenants%20we%20are%20not%20seeing%20v1%20module%20legacy%20authentication%20attempts.%20%3CBR%20%2F%3ETo%20confirm%20we%20are%20looking%20in%20the%20same%20place%20as%20you%2C%20are%20you%20going%20into%20Azure%20Active%20Directory%20%26gt%3B%20Activity%20%26gt%3B%20SIgn-Ins%20%3F%20%3CBR%20%2F%3EWhen%20filtering%20on%20%22Office%20365%22%20as%20the%20application%20and%20sign-in%20status%20Failure%20for%20the%20past%2024%20hours%20we%20don't%20see%20any%20events.%20And%20when%20we%20broaden%20the%20filter%20for%20all%20failures%2C%20we%20don't%20see%20any%20of%20the%20legacy%20auth%20failures.%20Again%2C%20we%20are%20seeing%20this%20auditing%20gap%20exist%20in%20two%20separate%20Azure%20AD%20tenants.%20I'm%20about%20to%20check%20a%203rd.%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F25097iB67F3F848F39C3EF%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%222017-12-04_13-51-05.jpg%22%20title%3D%222017-12-04_13-51-05.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-133982%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-133982%22%20slang%3D%22en-US%22%3E%3CP%3ESo%20the%20V2%20module%20should%20be%20%22Azure%20Active%20Directory%20PowerShell%22%2C%20and%20it%20has%20the%20%22MFA%20Required%22%20set%20to%20true.%20Apart%20from%20that%2C%20I%20have%20been%20using%20the%20old%20MSOnline%20module%2C%20which%20gets%20reflected%20as%20just%26nbsp%3B%22Office%20365%22%20and%20as%20I'm%20using%20the%20-credentials%20parameter%20with%20it%2C%20it's%20definitely%20legacy%20auth%20(%22MFA%20requires%22%20says%20false).%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ENot%20sure%20about%20the%20delay%2C%20but%20I%20do%20have%20some%20logins%20from%20today%2C%20so%20should%20be%20relatively%20fast.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-133967%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-133967%22%20slang%3D%22en-US%22%3EThanks.%20can%20you%20confirm%20your%20events%20are%20from%20Azure%20Active%20Directory%20Powershell%20Module%20version%201.0%20using%20Legacy%20Authentication%3F%20How%20long%20is%20the%20delay%20before%20you%20see%20them%20in%20the%20logs%3F%20To%20be%20clear%20we%20are%20not%20referring%20to%20Exchange%20legacy%20authentication...%20it%20is%20speciic%20to%20Azure%20AD%20PowerShell%20using%20legacy%20auth.%3CBR%20%2F%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-133962%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Activity%20Log%20missing%20legacy%20auth%20failed%20attempts%20or%20account%20lockouts%20for%20AAD%20Powershell%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-133962%22%20slang%3D%22en-US%22%3E%3CP%3EI%20seem%20to%20have%20some%20in%20the%20logs%2C%20they%20are%20marked%20as%20%22Office%20365%22%20for%20the%20application%20name%20though.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Frequent Contributor

In my testing, I am not seeing any logging of failed attempts or account lockouts in the Azure Active Directory Activity Sign-In Logs when the legacy module of Azure Active Directory is used.

Modern Authentication clients are logged for failed sign-ins and account lockouts, but not when legacy authentication is used in the Azure Active Directory powershell module.

The risk / concern here is that attackers can go undetected in their brute force attempts.

I realize the risk is small, since accounts are locked out after 10 invalid attempts, however, it would still be nice to have visibility.

According to the documentation, it can take up to 8 hours for legacy apps to show up in the logs, however, I have waited 12 hours and I still see no sign of my simulated brute force activity.

"For some sign-ins activity data coming from legacy office applications, it can take to 8 hours for the reporting data to show up"

https://docs.microsoft.com/en-us/azure/active-directory/active-directory-reporting-latencies-azure-p...

The note above says "legacy office applications" so I assume that Azure AD Powershell would fit into that category rather than not logging any activity at all from it.

 

 

9 Replies

I seem to have some in the logs, they are marked as "Office 365" for the application name though.

Thanks. can you confirm your events are from Azure Active Directory Powershell Module version 1.0 using Legacy Authentication? How long is the delay before you see them in the logs? To be clear we are not referring to Exchange legacy authentication... it is speciic to Azure AD PowerShell using legacy auth.

So the V2 module should be "Azure Active Directory PowerShell", and it has the "MFA Required" set to true. Apart from that, I have been using the old MSOnline module, which gets reflected as just "Office 365" and as I'm using the -credentials parameter with it, it's definitely legacy auth ("MFA requires" says false).

 

Not sure about the delay, but I do have some logins from today, so should be relatively fast.

interesting so far in two separate tenants we are not seeing v1 module legacy authentication attempts.
To confirm we are looking in the same place as you, are you going into Azure Active Directory > Activity > SIgn-Ins ?
When filtering on "Office 365" as the application and sign-in status Failure for the past 24 hours we don't see any events. And when we broaden the filter for all failures, we don't see any of the legacy auth failures. Again, we are seeing this auditing gap exist in two separate Azure AD tenants. I'm about to check a 3rd.

2017-12-04_13-51-05.jpg

The UI is pretty crappy if you ask me, simply download the report and work with it. Then again, I might simply be wrong and the "Office 365" app logins I'm seeing might correspond to something else.

The downloaded report is also missing legacy authentication from Azure AD Powershell.

After simulating brute force against three separate AAD tenants using legacy powershell and waiting more than 12 hours after each attempt, I have yet to find a case where it is a logged event. I'll open a ticket with MSFT Support to see if they are aware of this.

Looking at the tests I did, it definitely does not log every single failed attempt. I did get one additional entry from today, which seems to correspond to the dozen or so attempts I made. And I'm starting to think that you are correct here - I was expecting at least two different types of "applications" reported as I tried both the MSOnline module and (legacy auth) ExO PowerShell, but only one of them is visible - which I guess is the ExO failure, not the MSOnline one. Let us know if you get an official answer please.

Solution
Update 12/11/2017 - Microsoft Premier Support said this is working "by design" and will either provide a public facing article that states this and/or will open a "Design Change Request" to log these legacy authentication failure events.

Thanks Joe. I guess the events I'm seeing are ExO PowerShell then, or something else.

 

And Microsoft, really?!

Related Conversations
Extentions Synchronization
ChirmyRam in Discussions on
3 Replies
Tabs and Dark Mode
cjc2112 in Discussions on
35 Replies
flashing a white screen while open new tab
Deleted in Discussions on
14 Replies
Stable version of Edge insider browser
HotCakeX in Discussions on
35 Replies