SOLVED

Conditional Access Reporting

%3CLINGO-SUB%20id%3D%22lingo-sub-1379906%22%20slang%3D%22en-US%22%3ERe%3A%20Conditional%20Access%20Reporting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1379906%22%20slang%3D%22en-US%22%3E%3CP%3EAny%20changes%20are%20visible%20in%20the%20Audit%20log%3A%26nbsp%3B%3CFONT%20style%3D%22background-color%3A%20%23ffffff%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fportal.azure.com%2F%23blade%2FMicrosoft_AAD_IAM%2FActiveDirectoryMenuBlade%2FAudit%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fportal.azure.com%2F%23blade%2FMicrosoft_AAD_IAM%2FActiveDirectoryMenuBlade%2FAudit%3C%2FA%3E%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%3EYou%20can%20also%20access%20it%20programmatically%20via%20the%20Graph%20API.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1379973%22%20slang%3D%22en-US%22%3ERe%3A%20Conditional%20Access%20Reporting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1379973%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F58%22%20target%3D%22_blank%22%3E%40Vasil%20Michev%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ei%20set%20it%20that%20to%2030%20days%20and%20tried%20reviewing%20logs%20with%20and%20without%20the%20filter%20for%20service%3Dconditional%20access%20with%20no%20results.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ewe%20send%20our%20logs%20to%20splunk%2C%20and%20i%20do%20see%20some%20data%20but%20it%20looks%20like%20it%20comes%20from%20o365%20management%20logs.%20but%20that%20also%20only%20fetches%20at%20some%20frequency%2C%20i'd%20prefer%20to%20alert%20from%20azure%20so%20its%20more%20realtime.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1382526%22%20slang%3D%22en-US%22%3ERe%3A%20Conditional%20Access%20Reporting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1382526%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F355862%22%20target%3D%22_blank%22%3E%40gd-29%3C%2FA%3E%26nbsp%3BYou%20can%20use%20Log%20Analytics%20to%20create%20your%20own%20alerts%20I've%20found%20following%20article%20how%20to%20implement%20your%20custom%20alerts%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftech.nicolonsky.ch%2Fconditional-access-and-azure-log-analytics-in-harmony%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Ftech.nicolonsky.ch%2Fconditional-access-and-azure-log-analytics-in-harmony%2F%3C%2FA%3E%3C%2FP%3E%3CP%3EThey%20also%20requested%20this%20feature%20on%20uservoice%20but%20it's%20still%20not%20implemented%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ffeedback.azure.com%2Fforums%2F169401-azure-active-directory%2Fsuggestions%2F19331617-change-tracking-for-conditional-access-policies%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Ffeedback.azure.com%2Fforums%2F169401-azure-active-directory%2Fsuggestions%2F19331617-change-tracking-for-conditional-access-policies%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EYou%20can%20also%20use%20Azure%20Sentinel.%20connect%20your%20Azure%20AD%20Data%20Connector%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fsentinel%2Fconnect-azure-active-directory%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fsentinel%2Fconnect-azure-active-directory%3C%2FA%3E%3C%2FP%3E%3CP%3EAnd%20implement%20your%20own%20Rules%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1384501%22%20slang%3D%22en-US%22%3ERe%3A%20Conditional%20Access%20Reporting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1384501%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F664679%22%20target%3D%22_blank%22%3E%40JordyBlommaert%3C%2FA%3E%26nbsp%3Bthanks!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethis%20is%20a%20huge%20improvement.%20whats%20interesting%20is%20the%20querys%20from%20azure%20audit%20logs%20are%20way%20easier%20to%20see%20who%20made%20a%20change%20to%20the%20policy%2C%20but%20doesn't%20show%20what%20the%20change%20was%20(even%20though%20there%20is%20a%20new%20and%20old%20value%20field%2C%20its%20not%20accurate).%26nbsp%3B%3C%2FP%3E%3CP%3Ethe%20data%20we%20collect%20in%20splunk%20shows%20the%20policy%20as%20its%20changed%20(not%20the%20old%20values)%2C%20but%20doesn't%20seem%20to%20have%20the%20account%20that%20changed%20it%2C%20it%20shows%20some%20random%20API%20accounts.%20i'm%20assuming%20these%20are%20log%20entries%20from%20a%20backend%20process%20that%20we%20are%20capturing%20once%20you%20make%20the%20change%20in%20the%20website.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ei'll%20post%20back%20if%20i%20get%20this%20in%20a%20better%20place.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1379594%22%20slang%3D%22en-US%22%3EConditional%20Access%20Reporting%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1379594%22%20slang%3D%22en-US%22%3E%3CP%3Eare%20there%20any%20built%20in%20conditional%20access%20reporting%20for%20changes%20to%20the%20policy%3F%26nbsp%3B%3C%2FP%3E%3CP%3Eany%20way%20to%20alert%20or%20report%20when%20the%20policy%20changes%20and%20who%20made%20the%20change%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1379594%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EConditional%20Access%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Contributor

are there any built in conditional access reporting for changes to the policy? 

any way to alert or report when the policy changes and who made the change?

4 Replies

Any changes are visible in the Audit log: https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/Audit

You can also access it programmatically via the Graph API.

@Vasil Michev 

i set it that to 30 days and tried reviewing logs with and without the filter for service=conditional access with no results.

 

we send our logs to splunk, and i do see some data but it looks like it comes from o365 management logs. but that also only fetches at some frequency, i'd prefer to alert from azure so its more realtime.

 

 

best response confirmed by gd-29 (Contributor)
Solution

@gd-29 You can use Log Analytics to create your own alerts I've found following article how to implement your custom alerts: https://tech.nicolonsky.ch/conditional-access-and-azure-log-analytics-in-harmony/

They also requested this feature on uservoice but it's still not implemented: https://feedback.azure.com/forums/169401-azure-active-directory/suggestions/19331617-change-tracking...

 

You can also use Azure Sentinel. connect your Azure AD Data Connector: https://docs.microsoft.com/en-us/azure/sentinel/connect-azure-active-directory

And implement your own Rules

@JordyBlommaert thanks!

 

this is a huge improvement. whats interesting is the querys from azure audit logs are way easier to see who made a change to the policy, but doesn't show what the change was (even though there is a new and old value field, its not accurate). 

the data we collect in splunk shows the policy as its changed (not the old values), but doesn't seem to have the account that changed it, it shows some random API accounts. i'm assuming these are log entries from a backend process that we are capturing once you make the change in the website.

 

i'll post back if i get this in a better place.