12-04-2017 08:45 AM
12-04-2017 08:45 AM
In my testing, I am not seeing any logging of failed attempts or account lockouts in the Azure Active Directory Activity Sign-In Logs when the legacy module of Azure Active Directory is used.
Modern Authentication clients are logged for failed sign-ins and account lockouts, but not when legacy authentication is used in the Azure Active Directory powershell module.
The risk / concern here is that attackers can go undetected in their brute force attempts.
I realize the risk is small, since accounts are locked out after 10 invalid attempts, however, it would still be nice to have visibility.
According to the documentation, it can take up to 8 hours for legacy apps to show up in the logs, however, I have waited 12 hours and I still see no sign of my simulated brute force activity.
"For some sign-ins activity data coming from legacy office applications, it can take to 8 hours for the reporting data to show up"
The note above says "legacy office applications" so I assume that Azure AD Powershell would fit into that category rather than not logging any activity at all from it.
12-04-2017 01:11 PM
I seem to have some in the logs, they are marked as "Office 365" for the application name though.
12-04-2017 01:17 PM
12-04-2017 01:38 PM
So the V2 module should be "Azure Active Directory PowerShell", and it has the "MFA Required" set to true. Apart from that, I have been using the old MSOnline module, which gets reflected as just "Office 365" and as I'm using the -credentials parameter with it, it's definitely legacy auth ("MFA requires" says false).
Not sure about the delay, but I do have some logins from today, so should be relatively fast.
12-04-2017 01:53 PM
interesting so far in two separate tenants we are not seeing v1 module legacy authentication attempts.
To confirm we are looking in the same place as you, are you going into Azure Active Directory > Activity > SIgn-Ins ?
When filtering on "Office 365" as the application and sign-in status Failure for the past 24 hours we don't see any events. And when we broaden the filter for all failures, we don't see any of the legacy auth failures. Again, we are seeing this auditing gap exist in two separate Azure AD tenants. I'm about to check a 3rd.
12-05-2017 12:21 AM
The UI is pretty crappy if you ask me, simply download the report and work with it. Then again, I might simply be wrong and the "Office 365" app logins I'm seeing might correspond to something else.
12-05-2017 07:35 AM
The downloaded report is also missing legacy authentication from Azure AD Powershell.
After simulating brute force against three separate AAD tenants using legacy powershell and waiting more than 12 hours after each attempt, I have yet to find a case where it is a logged event. I'll open a ticket with MSFT Support to see if they are aware of this.
12-05-2017 11:23 AM
Looking at the tests I did, it definitely does not log every single failed attempt. I did get one additional entry from today, which seems to correspond to the dozen or so attempts I made. And I'm starting to think that you are correct here - I was expecting at least two different types of "applications" reported as I tried both the MSOnline module and (legacy auth) ExO PowerShell, but only one of them is visible - which I guess is the ExO failure, not the MSOnline one. Let us know if you get an official answer please.
12-11-2017 10:13 AMSolution
12-11-2017 11:12 AM
Thanks Joe. I guess the events I'm seeing are ExO PowerShell then, or something else.
And Microsoft, really?!