Azure Policy Exemption Validation

Q: I’m working on Azure Policy for a large environment, how do I do some basic validation that I didn’t make common mistakes? 

A: There are a few ways I’ve validated my environment using PowerShell that I’ll walk through here. 

I enjoy using Azure Policy to enforce compliance and prevent mistakes that could make my clients non-compliant and cause all sorts of security problems. My clients often like to follow stricter standards than required by some regulations, while having more relaxed special exceptions to regulation for other resources. These special exceptions have the risk accepted and have exceptions logged as waivers. There are still other items where the client has met the compliance objective through different ways than Azure is able to measure, so we put in an exception stating that the issue has been mitigated. Azure Policy and Defender for Cloud make this process far less painful than it was before the cloud, however how do you know that someone on your team didn’t make a mistake while logging an exemption? People will always make mistakes, but I like to do some basic analysis through some scripting to validate that what my team did is mistake-free, and that regulators won’t find any paperwork errors. 

Before we get into good fun code, we have to see what the lawyers have for us. 

Disclaimer 

The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages. 

To analyze all exemptions and find problematic ones, we first need to collect all the exemptions. It is easy to have dozens of subscriptions across many tenants, and running a script in each one gets to be a pain. Fortunately, I documented an easy way to execute code against all subscriptions in a blog a couple of years ago here. Since I was going to be scanning several different environments, I made my life easier and wrote a cmdlet available in a module here and here for finding all policy exemptions. 

To get all policy exemptions I simply run: 

$allExemptions = Get-AHAllPolicyExemptions

 If you don’t get data, or not much data back, then make sure you have permissions to query all subscriptions and management groups of concern. You may need to PIM into additional roles. If you only have permission to some subscriptions but not others, then you will get a warning when running this command but it will collect whatever you have permission to. 

Now that I have all exceptions, I can look through them to find anomalous ones. 

To track down which exemptions are unusual, I decided on the following criteria. This is what worked for me, but you can modify these according to your needs. 

• Exempt an entire policy initiative instead of just one or a few policies – a resource may have a special exemption like having a storage account publicly accessible and have an exemption in place for it, however it should only be exempt from that single policy, not all regulations for your industry. If policies are applied incorrectly, then exemptions can be put in place to exclude a resource from all regulations in an initiative. 
• Exemptions that do not expire – exemptions should be validated at regular intervals to validate that they are still correct and necessary, so any exemption without an expiration warrants investigation. 
• Exemptions without a description – Exemptions should have a meaningful explanation of why it was put in place. For a waiver, there should be a description of why the risk was accepted; for a mitigation, the description should explain how the threat was mitigated. If there is no description, then it is unclear to auditors, so I want these to be re-visited and fixed. 

Now that we have all exemptions stored in a variable and we know what to look for, we can output some pretty data. 

List all exemptions where an entire initiative is excluded instead of individual policies: 

$allExemptions | Where{$_.Properties.PolicyDefinitionReferenceIds.count -eq 0} 

I want to make it prettier and easier to work with the output though, so here is a prettier version: 

$allSubs = Get-AzSubscription | %{@{$_.SubscriptionId = $_.Name}} #Hashtable of subscriptions 
$allExemptions | Where{$_.Properties.PolicyDefinitionReferenceIds.count -eq 0} | Select Name, SubscriptionId -ExpandProperty Properties | Select DisplayName, Description, Name, @{N='NumberOfPoliciesExempted';E={$_.PolicyDefinitionReferenceIds.count}}, ExemptionCategory, @{N='SubscriptionName';E={$allSubs.$($_.SubscriptionId)}} | FT 

The names of some exemptions are pseudo-random strings, so why do I output them? I’m lazy, when I want to look at an exemption in the portal, I want it to be easy to find, so I copy the Name into the search field which narrows down hundreds of exemptions to a single exemption. This is especially helpful if you have many exemptions with the same display name. 

 List all exemptions that do not expire: 

$allExemptions | Where{$Null -eq $_.Properties.ExpiresOn} 

Yay, that is pretty quick, now we can add the same stuff to make it prettier. 

$allExemptions | Where{$Null -eq $_.Properties.ExpiresOn} | Select Name, SubscriptionId -ExpandProperty Properties | Select DisplayName, Description, Name, @{N='NumberOfPoliciesExempted';E={$_.PolicyDefinitionReferenceIds.count}}, ExemptionCategory, @{N='SubscriptionName';E={$allSubs.$($_.SubscriptionId)}} | FT 

List all exemptions without a description: 

$allExemptions | Where{$Null -eq $_.Properties.Description} 

Again, I’ll pretty it up: 

$allExemptions | Where{$Null -eq $_.Properties.Description} | Select Name, SubscriptionId -ExpandProperty Properties | Select DisplayName, Description, Name, @{N='NumberOfPoliciesExempted';E={$_.PolicyDefinitionReferenceIds.count}}, ExemptionCategory, @{N='SubscriptionName';E={$allSubs.$($_.SubscriptionId)}} | FT 

Another option is to list all exemptions sorted by how many policies are exempted in each exemption. Exemptions listed as 0 policies exempted means all policies are exempted, a non-zero number is how many policies are exempted. To understand why 0 means everything, you can read more about it here. 

$allExemptions | Select Name, SubscriptionId -ExpandProperty Properties | Select DisplayName, Description, Name, @{N='NumberOfPoliciesExempted';E={$_.PolicyDefinitionReferenceIds.count}}, ExemptionCategory, @{N='SubscriptionName';E={$allSubs.$($_.SubscriptionId)}} | sort NumberOfPoliciesExempted | FT 

I hope you have enjoyed a glimpse into the kind of fun only attainable with fine attention to regulatory compliance. 

Have fun scripting! 

References: 

• Azure Policy Exemption Structure: https://learn.microsoft.com/en-us/azure/governance/policy/concepts/exemption-structure
• More on parsing JSON with PowerShell: https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/parsing-json-with-powershell/ba-p/2768721 
• Azure Helper GitHub: https://github.com/PaulHCode/AzureHelper 
• Azure Helper PowerShell Gallery: https://www.powershellgallery.com/packages/AzureHelper/