Managing Azure Policies through Python SDK
Published Oct 19 2023 12:58 AM 3,450 Views
Microsoft

Managing Azure Policies through Python SDK

 

Azure Policy helps to enforce organizational standards and to assess compliance at-scale. It also helps to bring your resources to compliance through bulk remediation for existing resources and automatic remediation for added resources.

 

Common use cases for Azure Policy include implementing governance for resource consistency, regulatory compliance, security, cost, and management. Policy definitions for these common use cases are already available in your Azure environment as built-ins to help you get started.

Specifically, some useful governance actions you can enforce with Azure Policy include:

  • Ensuring your team deploys Azure resources only to allowed regions.
  • Enforcing the consistent application of taxonomic tags

Requiring resources to send diagnostic logs to a Log Analytics workspace

 

References: https://learn.microsoft.com/en-us/azure/governance/policy/overview

                        https://learn.microsoft.com/en-us/azure/governance/policy/policy-glossary

https://learn.microsoft.com/en-us/azure/developer/python/sdk/azure-sdk-overview

 

We can manage Azure policies through portal, PowerShell, CLI, REST API, Bicep, ARM Templates, Terraform and SDKs.

 

This blog will cover the policy management through Python SDK and we can use any IDE that supports Python SDK for Azure. We are using Visual Studio Code here. (https://azure.microsoft.com/en-in/products/visual-studio-code)

 

Azure Python SDK Authentication Reference : https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-overview

 

 

Import Libraries

 

 

 

 

import os

from azure.identity import ClientSecretCredential
from azure.identity import AzureCliCredential
from azure.mgmt.authorization import AuthorizationManagementClient
from azure.mgmt.resource import PolicyClient, ResourceManagementClient
from azure.mgmt.resource.subscriptions import SubscriptionClient
from azure.mgmt.policyinsights import PolicyInsightsClient

 

 

 

 

Define and Assign Variables

 

 

 

 

subscription_id = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad"        # your subscription ID
tenant_id = "xyzxyz-abc-cd-avc-48ebcd07d17c"  #  Your tenant ID
client_id = "123abc-cdd-4f09-ad9f-abcdef"     # Your Client ID
client_secret = "wiufhuiw24874946497fff"      # Your Client Secret

POLICY_NAME = "KeyVaultDIAGDINEpolicy"
SUBSCRIPTION_ID = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad"
GROUP_NAME = "resourceGroupName"
MANAGEMENTGROUP_ID= "abcd-123-4e7b-a446-233cvd"
POLICY_ASSIGNMENT_NAME = "KVDiagDINEpolicy"

 

 

 

 

Creation of objects

 

 

 

resource_client = ResourceManagementClient(
        credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
        subscription_id=subscription_id
    )
 
   policyinsights_client = PolicyInsightsClient(
        credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
        subscription_id=subscription_id
    )
    
   policy_client = PolicyClient(
        credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
        subscription_id=subscription_id
    )

 

 

 

Creation of Policy Definition at the Subcription Scope 

 

 

 

definitionatsubscription = policy_client.policy_definitions.create_or_update(policy_definition_name=POLICY_NAME,
                                                                                parameters= {
    "displayName": "KVDiag",
    "policyType": "Custom",
    "mode": "Indexed",

    "parameters": {
      "effect": {
        "type": "String",
        "metadata": {
          "displayName": "Effect",
          "description": "Enable or disable the execution of the policy"
        },
        "allowedValues": [
          "DeployIfNotExists",
          "Disabled"
        ],
        "defaultValue": "DeployIfNotExists"
      },
      "profileName": {
        "type": "String",
        "metadata": {
          "displayName": "Profile name",
          "description": "The diagnostic settings profile name"
        }
      },
      "logAnalytics": {
        "type": "String",
        "metadata": {
          "displayName": "Log Analytics workspace",
          "description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.",
          "strongType": "omsWorkspace",
          "assignPermissions": "true"
        }
      },
      "azureRegions": {
        "type": "Array",
        "metadata": {
          "displayName": "Allowed Locations",
          "description": "The list of locations that can be specified when deploying resources",
          "strongType": "location"
        }
      },
      "metricsEnabled": {
        "type": "String",
        "metadata": {
          "displayName": "Enable metrics",
          "description": "Whether to enable metrics stream to the Log Analytics workspace - True or False"
        },
        "allowedValues": [
          "True",
          "False"
        ],
        "defaultValue": "True"
      },
      "logsEnabled": {
        "type": "String",
        "metadata": {
          "displayName": "Enable logs",
          "description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
        },
        "allowedValues": [
          "True",
          "False"
        ],
        "defaultValue": "True"
      }
    },
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.KeyVault/vaults"
          },
          {
            "field": "location",
            "in": "[parameters('azureRegions')]"
          }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]",
        "details": {
          "type": "Microsoft.Insights/diagnosticSettings",
          "existenceCondition": {
            "allOf": [
              {
                "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
                "equals": "[parameters('logsEnabled')]"
              },
              {
                "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
                "equals": "[parameters('metricsEnabled')]"
              },
              {
                "field": "Microsoft.Insights/diagnosticSettings/workspaceId",
                "equals": "[parameters('logAnalytics')]"
              }
            ]
          },
          "roleDefinitionIds": [
            "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
            "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
          ],
          "deployment": {
            "properties": {
              "mode": "incremental",
              "template": {
                "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                  "resourceName": {
                    "type": "string"
                  },
                  "location": {
                    "type": "string"
                  },
                  "logAnalytics": {
                    "type": "string"
                  },
                  "metricsEnabled": {
                    "type": "string"
                  },
                  "logsEnabled": {
                    "type": "string"
                  },
                  "profileName": {
                    "type": "string"
                  }
                },
                "variables": {},
                "resources": [
                  {
                    "type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings",
                    "apiVersion": "2017-05-01-preview",
                    "name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
                    "location": "[parameters('location')]",
                    "dependsOn": [],
                    "properties": {
                      "workspaceId": "[parameters('logAnalytics')]",
                      "metrics": [
                        {
                          "category": "AllMetrics",
                          "enabled": "[parameters('metricsEnabled')]",
                          "retentionPolicy": {
                            "enabled": "false",
                            "days": 0
                          }
                        }
                      ],
                      "logs": [
                        {
                          "category": "AuditEvent",
                          "enabled": "[parameters('logsEnabled')]"
                        }
                      ]
                    }
                  }
                ],
                "outputs": {}
              },
              "parameters": {
                "location": {
                  "value": "[field('location')]"
                },
                "resourceName": {
                  "value": "[field('name')]"
                },
                "logAnalytics": {
                  "value": "[parameters('logAnalytics')]"
                },
                "metricsEnabled": {
                  "value": "[parameters('metricsEnabled')]"
                },
                "logsEnabled": {
                  "value": "[parameters('logsEnabled')]"
                },
                "profileName": {
                  "value": "[parameters('profileName')]"
                }
              }
            }
          }
        }
      }
    }})
    
   print("Created policy definition ID : {}".format(definitionatsubscription.id))
   print("Created policy definition Name : {} \n".format(definitionatsubscription.name))
   print (" *********  ******** ******** ********** ******** ******* \n")

 

 

 

Creation of Policy Definition at Management Group Scope 

 

 

 

definitionatmanagementgroup = policy_client.policy_definitions.create_or_update_at_management_group(policy_definition_name=POLICY_NAME,
                                                                                parameters= {
    "displayName": "KVDiag",
    "policyType": "Custom",
    "mode": "Indexed",

    "parameters": {
      "effect": {
        "type": "String",
        "metadata": {
          "displayName": "Effect",
          "description": "Enable or disable the execution of the policy"
        },
        "allowedValues": [
          "DeployIfNotExists",
          "Disabled"
        ],
        "defaultValue": "DeployIfNotExists"
      },
      "profileName": {
        "type": "String",
        "metadata": {
          "displayName": "Profile name",
          "description": "The diagnostic settings profile name"
        }
      },
      "logAnalytics": {
        "type": "String",
        "metadata": {
          "displayName": "Log Analytics workspace",
          "description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.",
          "strongType": "omsWorkspace",
          "assignPermissions": "true"
        }
      },
      "azureRegions": {
        "type": "Array",
        "metadata": {
          "displayName": "Allowed Locations",
          "description": "The list of locations that can be specified when deploying resources",
          "strongType": "location"
        }
      },
      "metricsEnabled": {
        "type": "String",
        "metadata": {
          "displayName": "Enable metrics",
          "description": "Whether to enable metrics stream to the Log Analytics workspace - True or False"
        },
        "allowedValues": [
          "True",
          "False"
        ],
        "defaultValue": "True"
      },
      "logsEnabled": {
        "type": "String",
        "metadata": {
          "displayName": "Enable logs",
          "description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
        },
        "allowedValues": [
          "True",
          "False"
        ],
        "defaultValue": "True"
      }
    },
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.KeyVault/vaults"
          },
          {
            "field": "location",
            "in": "[parameters('azureRegions')]"
          }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]",
        "details": {
          "type": "Microsoft.Insights/diagnosticSettings",
          "existenceCondition": {
            "allOf": [
              {
                "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
                "equals": "[parameters('logsEnabled')]"
              },
              {
                "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
                "equals": "[parameters('metricsEnabled')]"
              },
              {
                "field": "Microsoft.Insights/diagnosticSettings/workspaceId",
                "equals": "[parameters('logAnalytics')]"
              }
            ]
          },
          "roleDefinitionIds": [
            "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
            "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
          ],
          "deployment": {
            "properties": {
              "mode": "incremental",
              "template": {
                "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                  "resourceName": {
                    "type": "string"
                  },
                  "location": {
                    "type": "string"
                  },
                  "logAnalytics": {
                    "type": "string"
                  },
                  "metricsEnabled": {
                    "type": "string"
                  },
                  "logsEnabled": {
                    "type": "string"
                  },
                  "profileName": {
                    "type": "string"
                  }
                },
                "variables": {},
                "resources": [
                  {
                    "type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings",
                    "apiVersion": "2017-05-01-preview",
                    "name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
                    "location": "[parameters('location')]",
                    "dependsOn": [],
                    "properties": {
                      "workspaceId": "[parameters('logAnalytics')]",
                      "metrics": [
                        {
                          "category": "AllMetrics",
                          "enabled": "[parameters('metricsEnabled')]",
                          "retentionPolicy": {
                            "enabled": "false",
                            "days": 0
                          }
                        }
                      ],
                      "logs": [
                        {
                          "category": "AuditEvent",
                          "enabled": "[parameters('logsEnabled')]"
                        }
                      ]
                    }
                  }
                ],
                "outputs": {}
              },
              "parameters": {
                "location": {
                  "value": "[field('location')]"
                },
                "resourceName": {
                  "value": "[field('name')]"
                },
                "logAnalytics": {
                  "value": "[parameters('logAnalytics')]"
                },
                "metricsEnabled": {
                  "value": "[parameters('metricsEnabled')]"
                },
                "logsEnabled": {
                  "value": "[parameters('logsEnabled')]"
                },
                "profileName": {
                  "value": "[parameters('profileName')]"
                }
              }
            }
          }
        }
      }
    }},management_group_id= 'abcd-233df-4e7b-a446-2333xcvc'
    
    )
    
   print("Created policy definition ID : {}".format(definitionatmanagementgroup.id))
   print("Created policy definition Name : {} \n".format(definitionatmanagementgroup.name))
   print (" *********  ******** ******** ********** ******** ******* \n")

 

 

 

Defining Scopes for Policy Assignment 

 

 

 

scopeRG = '/subscriptions/{}/resourceGroups/{}'.format(
        SUBSCRIPTION_ID,
        GROUP_NAME
    )
   scopeSubscription = '/subscriptions/{}'.format(
        SUBSCRIPTION_ID
    )
    
   scopeMG = '/providers/Microsoft.Management/managementGroups/{}'.format(
        MANAGEMENTGROUP_ID
        
    )

 

 

 

Creating Policy Assignments at different Scope - Resource Group, Subscription and Management Group

 

 

 

assignmentatRG = policy_client.policy_assignments.create(
        scopeRG,
        POLICY_ASSIGNMENT_NAME + " at RG", 
     
        {
            'policy_definition_id': definitionatmanagementgroup.id,
            "identity": {"type": "SystemAssigned"},
            "location" :"eastus",
                        'parameters' : {
      "profileName": {
        "value": "KV_DIAG_Settings1"
      },
      "logAnalytics": {
        "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
      },
        "azureRegions": {
        "value": ["westus"]
      }
            },
            
        }
    )
        
   print("Createed policy assignment: {}".format(assignmentatRG.id)) 
   print("Created policy assignment: {}\n".format(assignmentatRG.name)) 
   print (" *********  ******** ******** ********** ******** ******* \n")

   assignmentatsubscription = policy_client.policy_assignments.create(
        scopeSubscription,
        POLICY_ASSIGNMENT_NAME + "at Subscription",
        {
            'policy_definition_id': definitionatsubscription.id,
            "identity": {"type": "SystemAssigned"},
              "location" :"eastus",
                        'parameters' : {
      "profileName": {
        "value": "KV_DIAG_Settings2"
      },
      "logAnalytics": {
        "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
      },
        "azureRegions": {
        "value": ["westus"]
      }
            }
        }
    )
        
   print("Created policy assignment: {}".format(assignmentatsubscription.id)) 
   print("Created policy assignment: {} \n".format(assignmentatsubscription.name)) 
   print (" *********  ******** ******** ********** ******** ******* \n")

   assignmentatmanagementgroup = policy_client.policy_assignments.create(
        scopeMG,
        POLICY_ASSIGNMENT_NAME + "at MG",
        {
            'policy_definition_id': definitionatmanagementgroup.id,
            "identity": {"type": "SystemAssigned"},
             "location" :"eastus",
            'parameters' : {
      "profileName": {
        "value": "KV_DIAG_Settings3"
      },
      "logAnalytics": {
        "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
      },
        "azureRegions": {
        "value": ["westus"]
      }
            }
        
        }
    )
        
   print("Created policy assignment: {}".format(assignmentatmanagementgroup.id)) 
   print("Created policy assignment: {}\n".format(assignmentatmanagementgroup.name)) 
   print (" *********  ******** ******** ********** ******** ******* \n")

 

 

Trigger manual policy evaluation and get the resource compliance/non-compliance count : 

Refernce Article : https://learn.microsoft.com/en-us/powershell/module/az.policyinsights/start-azpolicycompliancescan?v...

 

 

   triggerevaluation = policyinsights_client.policy_states.begin_trigger_subscription_evaluation( 
     subscription_id='abc1233-e44e-4a99-a89c-123dfg')
     
   print("Manual policy evaluation triggered :\n{}".format(triggerevaluation.status))
   
   
   policyevaluationresult = policyinsights_client.policy_states.list_query_results_for_subscription_level_policy_assignment(
      subscription_id='abc123-e44e-4a99-a89c-12333df',
      policy_states_resource="latest",
      policy_assignment_name= '621dc2655dd74ed68a37286d'
   )

 
   for item in policyevaluationresult:
        print(' Resource ID : '+ item.resource_id)
        print(' Policy Assignment ID : '+item.policy_assignment_id)
        print(' Policy Assignment Scope : '+item.policy_assignment_scope)
        print(' Compliance State : '+item.compliance_state)
        print('\n')

        if item.compliance_state == 'Compliant':
            Compliant= int(Compliant +1)
        elif item.compliance_state == 'NonCompliant':
            NonCompliant = int(NonCompliant +1)
        else:
            1==1
   print('Number of Resources')     
   print('Compliant :' + str(Compliant))
   print('NonCompliant :' + str(NonCompliant))

 

 

Creation of Remediation Task at different Scopes:

 

 


   remediation = policyinsights_client.remediations.create_or_update_at_management_group("1233-dfdf-4e7b-a446-223233",
    "mymgremediation" ,
      {
        "policy_assignment_id": "/providers/microsoft.management/managementgroups/6cf4d5f3-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
      ##  "policy_definition_reference_id" : "4037797753452518688" ## (optional) Needed when we need to remediate the non-compliant resources of policy initiative assignment
     
      })
 
   print("Create remediation:\n{}".format(remediation))

   remediationRG = policyinsights_client.remediations.create_or_update_at_resource_group(
    "myremediationatRG", "aro-rererer",
      { 
        "policy_assignment_id": "/providers/microsoft.management/managementgroups/12345-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
        "resource_discovery_mode" : 'ExistingNonCompliant' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant.
        ##ExistingNonCompliant - Remediate resources that are already known to be non-compliant.
    

      })
   
   print("Create remediation:\n{}".format(remediationRG))

   remediationsubscription = policyinsights_client.remediations.create_or_update_at_subscription(
    "mymgremediationatsubsc" ,
      {
        "policy_assignment_id": "/providers/microsoft.management/managementgroups/12333-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
        "resource_discovery_mode" : 'ReEvaluateCompliance' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant.
        ##ExistingNonCompliant - Remediate resources that are already known to be non-compliant.
       ## "policy_definition_reference_id" : "4037797753452518688"

      })
   
   print("Create remediation:\n{}".format(remediationsubscription))

Note - resource_discovery_mode is only for the policy assignments at the subscription and the resource group scope.

 

 

 

Co-Authors
Version history
Last update:
‎Oct 18 2023 08:20 AM
Updated by: