Greetings! I am Sue Bohn and I lead the Identity Customer and Partner Success Team. The mission of our worldwide team is to cut the distance between our customers and engineering so we all can work effectively at cloud speed. We are restarting the mailbag series to augment our docs with popular questions we get about Azure Active Directory and to share deployment insights. Mark and I often commiserate good-naturedly that we have a 'night job' as well as a 'day job' in order to get everything done. One of Mark’s favorite night jobs is writing tips and tricks to make our customers’ lives easier. Read on to hone your skills to extract the most value from Azure AD log files.
Hey, y’all Mark here. I am finally back, in blog form. To get us back into the swing of things we thought we’d cover a topic that would apply to everyone, Azure AD logs! If you haven’t spent much time in this space, you are missing out on a gold mine of information about your environment. Let’s dig in and see what we can find.
Q1: I may have not been the most “diligent” when it comes to Azure AD logs, can you catch me up really quick.
A1: First off, this isn’t going to be really quick and for good reason. While this isn’t the most exciting stuff, it’s super important. Seriously. You’ll need this stuff for your day-to-day troubleshooting but more importantly for compliance, investigating a security event, data changes affecting dynamic group memberships. And when you want to see the rubber really hit the road, you’ll need this for due diligence if you must disclose a compromise. Being able to show that only a handful accounts were accessed versus the entire population will be a huge, huge deal. You can thank me later.
There are two kinds of Azure AD activity reports in: Sign-In logs and Audit logs. Sign-In logs show the interactive sign-ins into Azure AD. You’ll see what application they were trying to access, if it was a success or a failure, the User Display Name, UPN, some MFA info, some other session conditions and some info about the client–like device ID, browser, and OS. These fields that can be selected under Columns.
We will now channel our inner greatest detective (Sherlock, but probably Batman). Sign-In logs help us answer the following questions.
- Who signed in?
- What did they access?
- When did they access?
- Where did they access it from?
- Why did something happen? or not?
- What type of authentication did users use?
- Which apps do not have any conditional access policies associated with them?
You can also see:
- Some common tasks the Sign-In log was used for.
- Sign-in pattern of user/users.
- Top sign-in errors with failure reasons.
- Conditional access policies and Grant controls triggered by sign-ins.
- Application usage.
- Common browser and OS used.
Finally you can:
- Investigate security incidents.
- Investigate sign-in failures.
Audit logs have pretty much everything else. You can think of these as things that change the state of a resource. That would include things such as Password Resets, PIM elevations, TOU acceptance, B2B being redeemed. These are all selected under Category.
Putting our detective hat back on we can determine:
- Who performed an action?
- What changes were made?
- When were the changes made?
- Was action successful?
- Which object was the target of the action?
This is a good spot to look if you want to see the SSPR reset events, Audit PIM events, or check on User/App Provisioning.
Q2: Who can access these reports in Azure AD?
A2: Global Admins, Security Admins, Security Readers, and Reports Readers can all access these reports. There is no difference in the data scope between these roles. What I mean by that is that Global Admin will see the same thing as the Security Reader. Users can access their own sign-in data.
Q3: I have a federated domain and I think something is wrong, I only see successful sign-in events, no failures. What gives?
A3: This is expected if you think about it. The federated IDP is responsible for authenticating the user. If they fail to authenticate, the claim never makes it to Azure AD so it would never make it into the Azure AD logs. This is super important when doing an investigation. You’ll need to correlate across the Azure AD logs and your IDP. For AD FS, Azure AD Connect Health can really help. There are also some great tools to aid in searching and aggregating AD FS logs on the AD FS Help site.
Q4: Speaking of which, where are Security Reports in all this?
A4: They got their own spot, depending if you have Azure AD P1 or P2. For those with Azure AD P1, in the Azure AD blade, under Security you’ll see this menu:.
If you have Azure Active Directory P2, you can see this menu in Identity Protection under Investigate:
The difference between P1 and P2 is that P2 will have the most detailed information about all underlying risk events and enables you to configure security policies that automatically respond to configured risk levels. Read Azure Active Directory risk events to learn more. These events are also found in the Azure Security Center.
Q5: How do these reports differ from the Office 365 reports? These seem like they have similar data, but it looks different.
A5: They’re actually the same as they’re being collected from the same source but we each are displaying things differently as you’re observing. Not ideal, we know and we’re working to unify this so this type of question won't have to be asked. In the meantime, for Sign-In or Auditing events, use Azure AD. For Office 365 specific, use their logs.
Q6: There is a lot of data in these, do you have something that is “manager friendly” aka has graphs and colors.
A6: That would be the Azure AD Content Pack. Look, it even has a map!
This is a quick win. You can get this up and running in a few minutes, really. It shows App Usage, sign-ins by location, SSPR usage and Device info. Then you can start customizing it as you see fit.
Q7: What is the best way to integrate all of this data into my SIEM tool?
A7: This used to be really painful. Now, this is achievable through a single configuration change. You’ll need to send the logs to an Azure Event Hub. Then we have several SIEMs that have pre-built integration to pull these logs. As of this writing, Splunk, IBM QRadar and Sumo Logic. If your SIEM doesn’t support this please tell them, we’re more than happy to help get them going. If not, you can also use the Event Hub APIs to pull the info down. We have it all documented. For security events, these come from the Microsoft Intelligent Security Graph. You'll want to setup Azure Monitor to send the alerts to the same Event Hub. You can follow this guide.
Q8: My SIEM is a little bit like Hotel California, events check-in but they do not check out, do you have any key events I should be taking action on.
A8: You’re not alone. First, any High-Risk events from the security logs like Leaked Credentials or Users at high risk should be acted on immediately. For Medium Risk events such as TOR Browser logins, Unfamiliar Locations, Atypical Travel or Suspicious IP you will want to investigate and correlate with other data you have about that user.
Next, you will want to check your legacy authentications in the Sign-In logs. Using the Client App field, you can sort by different protocols such as IMAP and POP. We actually covered some of this at Ignite 2018.
Some other things that should make someone sit up and do something include promotion of accounts to an admin account, creation of accounts that look like service accounts/high-value employees, updates to Service Principals, consent grants made by admins, removal of MFA requirements, and any changes or disablement of audit configuration.
From the Office 365 side, creation of mail forwarding rules in a mailbox or transport rule to an external domain, addition of mail forward permissions or mailbox delegates, creation of custom email forms, and changes to external sharing polices to name a few. The good folks over in Office 365 security have an EXCELLENT post on this.
Q9: What if I don't already have an SIEM system to store and report on the Azure AD activity logs?
A9: Through the integration of Azure Security Monitor, you can emit your logs to an instance of Azure Log Analytics, where we even provide some sample views to help you analyze that data.
Q10: Do you have a list of things everyone should do?
A10: If you’ve been following along on this post you should be in better shape than when you started but let’s recap.
- First, you want to make sure you enable audit logging but just clicking on the logs in the Azure AD Portal and selecting yes.
- Next, the Azure AD Content Pack in Question 6 is a no brainer and really takes minutes to get working.
- Then you’ll want to get all this data flowing into your SIEM system by following Question 7
- Another thing that you’ll want to do is enable all the auditing in Office 365 and read through the finding bad behavior in Securing Office 365
- Next use Q8 to make sure your SIEM is configured to alert your SOC or someone when those key events.
Finally, a topic that is much too large for this post, does your company have an Incident Response plan? And does that plan include your PR team? You are probably thinking why would PR need to be a part of this? As I said in the intro, where you’ll really be glad you’ve spent your time on this when something truly bad has happened and PR must be involved. It will be much better for everyone if this isn’t their first foray into this arena when everything is at its worst possible.
For any questions you can reach us at AskAzureADBlog@microsoft.com, Tech Communities or on Twitter @AzureAD, @MarkMorow and @Alex_A_Simons
—Mark Morowczynski, Dhanyah Krishnamoorthy, and Ryen Macababbad
Microsoft
