Case Details:
With the move to the cloud, many teams have adopted agile development methods. You can automate deployments and use the practice of infrastructure as code. To implement infrastructure as code for your Azure solutions, use Azure Resource Manager templates (ARM templates).
ARM Templates are JSON files which uses declarative syntax, that lets you state what you intend to deploy without having to write the sequence of programming commands to create it.
Azure Policy helps to enforce organizational standards and to assess compliance at-scale. To achieve this we use either Built-In Policies or we would need to frame our own Custom Policies.
Often we are asked to frame policies to enable certain properties for a resource or provide default value for a property, etc. These properties can easily be enabled or added a value using PowerShell but while trying to incorporate these within ARM Policy its quite a challenging and time-consuming job.
For example, enabling Auditing for a synapse workspace, enable soft-delete for a Storage Blob, etc.
Challenges using ARM Templates / Policies ?:
- ARM templates are sometimes challenging with respect to debugging when an error occurs.
- It takes time for a deployment to complete, especially when multiple resources are involved or when resources like Networking Interface or APIM, etc are involved which takes more than 30 minutes to complete the deployment
- The declarative syntax used for writing a ARM Template/policy is not understood by majority of people unless you work regularly with it.
- Templates are initially written by someone but is maintained by someone else who might face challenges when some editing is required especially if size of template is too large.
Whenever the developer or coder encounters such situations, they could easily panic and also it might take days or weeks to edit incorporate new changes into the existing templates.
Our Solution:
To avoid all hassles with respect to ARM templates / Policies and achieve your desired result in no time, Microsoft has integrated PowerShell with ARM templates.
PowerShell is not a new language for a developer/coder. It’s a very popular scripting language and has wide-scale usage among the developer community. Most of the time a feature is available through PowerShell than other formats like ARM template, Azure Portal, etc.
PowerShell commands are easy to debug and can be run locally before you push it to server’s. You can literally do anything and achieve everything in Azure using PowerShell.
Due to all these advantages, PowerShell comes handy and preferred choice while editing ARM templates or Policies.
Pre-requisites:
- Create a Git-Hub account
- Create a repository in Git-Hub
- Make repository public.
You can do so by going to the “Settings” tab of Repository and scroll down to “Danger Zone” in “Settings”.
Click “Change Visibility” for “Change repository visibility”
- Upload your PowerShell script (script that has content for doing any changes you wanted to achieve using ARM template or Policy) to the above created public repository.
- Collect the raw URL by clicking on the “Raw” tab as shown below.
- The next step is to just integrate this URL into the ARM template.
Sample code:
{
"type": "Microsoft.Resources/deploymentScripts",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('str4Apr2022Name'))]"
],
"apiVersion": "2020-10-01",
"name": "pshell",
"location": "[resourceGroup().location]",
"kind": "AzurePowerShell",
"properties": {
"forceUpdateTag": "1",
"azPowerShellVersion": "6.2",
"arguments": "[concat(' -pClientId ','your client Id',' -pKey ','your client secret',' -pTenantId ','your tenant Id', ' -pSubscriptionId ', 'your subscription Id', ' -pStorageAccountName ',variables('str4Apr2022Name'))]",
"primaryScriptUri": "https://raw.githubusercontent.com/thakurmishra21/test/main/storageps.ps1",
"supportingScriptUris": [],
"timeout": "PT30M",
"cleanupPreference": "OnSuccess",
"retentionInterval": "P1D"
}
}
Example:
Let’s assume a scenario where you are trying to create a Storage Account and you want to ensure team follows the same. As part of your organization standards and meet compliance you need to ensure that the new storage account created should have Soft Delete Enabled on Blobs with a Retention Policy of 7 days.
You provide them with an ARM template as below:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"str4Apr2022Type": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_ZRS",
"Standard_GRS",
"Standard_RAGRS",
"Premium_LRS"
]
}
},
"resources": [
{
"name": "[variables('str4Apr2022Name')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [],
"tags": {
"displayName": "str4Apr2022"
},
"properties": {
"accountType": "[parameters('str4Apr2022Type')]"
}
}
],
"variables": {
"str4Apr2022Name": "[concat('str4Apr2022', uniqueString(resourceGroup().id))]"
}
}
You can enable soft delete on blobs with retention policy of 7 days using PowerShell by using the below simple command:
Enable-AzStorageBlobDeleteRetentionPolicy -ResourceGroupName <resource-group> `
-StorageAccountName <storage-account> `
-RetentionDays 7
To enforce Organization Standard and Compliance, all you need to do is just integrate the above PowerShell command into ARM template as below:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"str4Apr2022Type": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_ZRS",
"Standard_GRS",
"Standard_RAGRS",
"Premium_LRS"
]
}
},
"resources": [
{
"name": "[variables('str4Apr2022Name')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [],
"tags": {
"displayName": "str4Apr2022"
},
"properties": {
"accountType": "[parameters('str4Apr2022Type')]"
}
},
{
"type": "Microsoft.Resources/deploymentScripts",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('str4Apr2022Name'))]"
],
"apiVersion": "2020-10-01",
"name": "pshell",
"location": "[resourceGroup().location]",
"kind": "AzurePowerShell",
"properties": {
"forceUpdateTag": "1",
"azPowerShellVersion": "6.2",
"arguments": "[concat(' -pClientId ','your client Id',' -pKey ','your client secret',' -pTenantId ','your tenant Id', ' -pSubscriptionId ', 'your subscription Id', ' -pStorageAccountName ',variables('str4Apr2022Name'))]",
"primaryScriptUri": "https://raw.githubusercontent.com/thakurmishra21/test/main/storageps.ps1",
"supportingScriptUris": [],
"timeout": "PT30M",
"cleanupPreference": "OnSuccess",
"retentionInterval": "P1D"
}
}
],
"variables": {
"str4Apr2022Name": "[concat('str4Apr2022',uniqueString(resourceGroup().id))]"
}
}
Example-2:
Sometimes you are in a more complex situation than one described above.
For ex, let’s say you must enable “Audit Settings” for a Synapse workspace. You want to send logs to a storage account.
To achieve this you need to first create a storage account and once deployment of storage account is over you need to enable the “Audit settings”.
You can use the below Custom Policy to do so.
{
"mode": "All",
"policyRule": {
"if": {
"field": "type",
"equals": "Microsoft.Synapse/workspaces"
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Synapse/workspaces/auditingSettings",
"name": "Default",
"existenceCondition": {
"field": "Microsoft.Synapse/workspaces/auditingSettings/state",
"equals": "Enabled"
},
"roleDefinitionIds": [
"/providers/microsoft.authorization/roleDefinitions/056cd41c-7e88-42e1-933e-88ba6a50c9c3",
"/providers/microsoft.authorization/roleDefinitions/17d1049b-9a84-46fb-8f53-869881c3d3ab"
],
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string"
},
"auditRetentionDays": {
"type": "int"
},
"storageAccountsResourceGroup": {
"type": "string"
},
"location": {
"type": "string"
},
"clientId": {
"type": "string"
},
"clientSecret": {
"type": "string"
}
},
"variables": {
"retentionDays": "[parameters('auditRetentionDays')]",
"subscriptionId": "[subscription().subscriptionId]",
"uniqueStorage": "[uniqueString(parameters('location'), parameters('storageAccountsResourceGroup'))]",
"locationCode": "[substring(parameters('location'), 0, 3)]",
"storageName": "[tolower(concat('audit', variables('locationCode'), variables('uniqueStorage')))]",
"createStorageAccountDeploymentName": "[concat('Auditing-', uniqueString(variables('locationCode'), deployment().name))]"
},
"resources": [
{
"apiVersion": "2017-05-10",
"name": "[variables('createStorageAccountDeploymentName')]",
"type": "Microsoft.Resources/deployments",
"resourceGroup": "[parameters('storageAccountsResourceGroup')]",
"properties": {
"mode": "Incremental",
"expressionEvaluationOptions": {
"scope": "inner"
},
"parameters": {
"location": {
"value": "[parameters('location')]"
},
"storageName": {
"value": "[variables('storageName')]"
}
},
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string"
},
"storageName": {
"type": "string"
}
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2021-02-01",
"name": "[parameters('storageName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard_LRS"
},
"kind": "BlobStorage",
"tags": {
"createdBy": "Azure Policy - Configure Synapse workspaces to have auditing enabled"
},
"properties": {
"accessTier": "Hot",
"supportsHttpsTrafficOnly": true,
"allowBlobPublicAccess": false
}
}
],
"outputs": {
"storageAccountEndPoint": {
"type": "string",
"value": "[reference(parameters('storageName')).primaryEndpoints.blob]"
}
}
}
}
},
{
"type": "Microsoft.Resources/deploymentScripts",
"dependsOn": [
"[resourceId('Microsoft.Resources/deployments/', variables('createStorageAccountDeploymentName'))]"
],
"apiVersion": "2020-10-01",
"name": "pshell",
"location": "[resourceGroup().location]",
"kind": "AzurePowerShell",
"properties": {
"forceUpdateTag": "1",
"azPowerShellVersion": "6.2",
"arguments": "[concat(' -pClientId ',parameters('clientId'),' -pKey ',parameters('clientSecret'),' -pTenantId ',subscription().tenantId, ' -pSubscriptionId ', subscription().subscriptionId, ' -pWorkspaceName ',parameters('workspaceName') ,' -saPath ',resourceId('Microsoft.Storage/storageAccounts', variables('storageName')))]",
"primaryScriptUri": "https://raw.githubusercontent.com/thakurmishra21/SynapsePS/main/synapsePS.ps1",
"supportingScriptUris": [],
"timeout": "PT30M",
"cleanupPreference": "OnSuccess",
"retentionInterval": "P1D"
}
}
]
},
"parameters": {
"workspaceName": {
"value": "[field('name')]"
},
"auditRetentionDays": {
"value": "[parameters('retentionDays')]"
},
"storageAccountsResourceGroup": {
"value": "[parameters('storageAccountsResourceGroup')]"
},
"location": {
"value": "[field('location')]"
},
"clientId": {
"value": "[parameters('clientId')]"
},
"clientSecret": {
"value": "[parameters('clientSecret')]"
}
}
}
}
}
}
},
"parameters": {
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"retentionDays": {
"type": "Integer",
"metadata": {
"displayName": "Retention days (optional, 180 days if unspecified)",
"description": "The value in days of the retention period (0 indicates unlimited retention)"
},
"defaultValue": 180
},
"storageAccountsResourceGroup": {
"type": "String",
"metadata": {
"displayName": "Resource group name for storage accounts",
"description": "Auditing writes database events to an audit log in your Azure Storage account (a storage account will be created in each region where a Synapse workspace is created that will be shared by all Synapse workspaces in that region). Important - for proper operation of Auditing do not delete or rename the resource group or the storage accounts.",
"strongType": "existingResourceGroups",
"assignPermissions": true
}
},
"clientId": {
"type": "String",
"metadata": {
"displayName": "clientId",
"description": null
},
"defaultValue": "0"
},
"clientSecret": {
"type": "String",
"metadata": {
"displayName": "clientSecret",
"description": null
},
"defaultValue": "0"
}
}
}
A question might arise as to why I can’t achieve this using ARM template?
Reason: You need to enable “Auditing” only after Storage Account deployment is over. But even after deployment is completed there are some internal processes going on and the particular storage account is still cannot be used for several seconds after deployment (for this particular case we are generating the name of Storage Account dynamically and not explicitly passed by user, for ex: the deployment-id’s value could be used to form the storage account name) .
So ideally you need to wait for some time, make sure your processes sleep for some time (around 10-15 seconds). This cannot be achieved with the help of ARM templates.
So, all you have to do is invoke a PowerShell command which would enable Audit settings.
This PowerShell command would let you wait till Storage Account is created and available for use during enablement of Audit settings.
References:
Templates overview - Azure Resource Manager | Microsoft Docs