Monitoring Azure Policy compliance across your Azure Managed Application deployments

During our recent development work with a partner, they asked a question...

 

We want to deploy Azure Policies with our Managed application to ensure the solution stays compliant and security standards are enforced. Is there a way for us, as a publisher, to get alerted if the policies deployed with the Managed application are violated and then quickly engage with the customers to ensure that their deployment is secure? 

 

The short answer was no, so we built one! 

We are open-sourcing the code for this solution so that other publishers can adopt this approach to ensure customers' Managed applications are secure. For more context on the solution described here please read Microsoft documentation on planning a Managed application and Azure Policy.

 

Problem 

When creating a Managed application in the Azure Marketplace, it is recommended to add Azure Policies to make sure the solution stays compliant and security standards are enforced. As the publisher of the Managed application, customers are expecting you to manage the application for them (as the name suggests) and this should include security. However, monitoring compliance states at scale, across all your app deployments, is not an easy task.  

 

To address this, we explored alternatives including Azure Event Grid. However, this solution would not work for Managed applications, because Event Grid Subscriptions cannot be used cross tenant (from the customer's tenant to the publisher’s tenant). Even if this was possible, the publisher’s Event Grid would need to be re-configured each time a new customer deploys via the Azure Marketplace to include the new deployment which is currently cannot be dynamically achieved.

 

We also looked at the option of deploying the function as part of a Managed application solution. However, this approach presented several challenges, including the need to maintain it separately for each Managed application deployment and setup authentication for the outbound connectivity from the customer environment, which could pose a security risk. 

 

Solution overview 

The built-in functionality that allows publishers to access the managed resource group used by the Managed application is the foundation upon which the entire solution is built. 

 

We have built a time-triggered Azure Function app, called Policy States Collector, that uses Azure APIs to query the latest compliance state of Azure policies in the Managed application environment. The application retrieves the resource group name and subscription id from Azure Blob Storage to build the correct request. In the previous blog post  Capturing lifecycle events from your Azure Managed Application deployments, we covered how we would store these information to Azure table storage for each customer.

 

In terms of authentication to the customer environments, the application leverages the Service Principal configured in the authorizations section of the Managed application plan.  

 

After retrieving the data, the application filters it and sends it to a table in Log Analytics Workspace for real-time monitoring and analysis. The app filters out any policy which is not set up by the Managed application to ensure that only the relevant policies are processed. 

Finally, a Scheduled Query Rule Alert is configured to monitor non-compliant policies and trigger an Action Group for notification when an issue is detected. 

 

This solution offers automated monitoring of Azure policies for compliance, allowing for proactive identification and resolution of policy violations.   

 

The source code of the solution is available at the following GitHub repo: azure-marketplace-management-extras.

 

 

Implementation 

The following Azure resources are used in the implementation of the Policy States Collector:

1. Azure Log Analytics  
2. Azure Function with two function apps +  Data Collection Rule (DCR) + Data Collection Endpoint (DCE) 
3. Azure Key Vault
4. Azure Storage Table  
5. Scheduled Query Rule Alert + Action Group    

The implementation of this solution is divided into two main parts. The first part involves querying and building the latest state of policies for all Managed applications. The second part of the implementation involves ingesting the data into Log Analytics.  The full solution can be found here.

 

1. Querying and building the latest state of policies for all Managed applications.  

Getting Managed application info: 

 

 

async with ClientSecretCredential(
AZURE_TENANT_ID,
AZURE_CLIENT_ID, AZURE_CLIENT_SECRET
) as client_credential, TableClient.from_connection_string(
CONNECTION_STRING, TABLE_NAME
) as table_client:
managed_applications = table_client.list_entities()

 

 

Using each RG and subscription ID to query policies for all Managed applications. 

 

 

async for application in managed_applications:
            result = get_policies(
                client_credential, application["subscription_id"], application["mrg_name"])
            all_applications_policies_to_upload.append(result)

        policies_upload = await asyncio.gather(*all_applications_policies_to_upload, return_exceptions=True)

 

 

2. Ingest the data into Log Analytics using DCE and DCR 

 

 

async with ManagedIdentityCredential() as ingestion_credential, LogsIngestionClient(
            endpoint=DATA_COLLECTION_ENDPOINT, credential=ingestion_credential, logging_enable=True) as logs_client:
        try:
            await logs_client.upload(
                rule_id=DATA_COLLECTION_IMMUTABLE_ID, stream_name=STREAM_NAME, logs=policies_upload)
            logging.info(f'Uploaded {len(policies_upload)} policies')

        except HttpResponseError as e:
            logging.error(f"Upload failed: {e}")

 

 

Deployment

 

Set up your Managed application Service Principal 

From your Partner Center account, navigate to your Marketplace offer > select your managed application plan > Plan overview > Technical configuration

 

In the Authorizations section, use the ID of a Service Principal that will be assigned the Owner role on the Managed application managed resource group.   

 

 

Note down the Service Principal information from the Partner Center portal. We will use this later. 

 

Infrastructure and function apps  

 

The azure directory contains the Bicep templates that deploy the infrastructure components of the Policy States Collector monitor and Notification Endpoint. Additionally, a GitHub workflow has been created to automate the infrastructure deployment and the function app deployment. Read these instructions how to deploy the infrastructure and code. 

 

Scheduled Query Rule Alert usage

 

You can see all policies by using the PolicyComplianceStates_CL query in Azure Log Analytics. 

 

The Query which is used in Alert rule gets the latest policy state using the TimeGenerated value. Alert rule will rerun the query with the latest states of policies which will allow to resolve the alert if newly ingested data has all compliant the policies.  

 

The last configuration combines all policies by Policy_assignment_id .

 

 

'PolicyComplianceStates_CL | summarize arg_max(TimeGenerated,*) by Policy_assignment_id | where Is_compliant==false'

 

 

Conclusion

 

The Policy States Collector is a handy tool for Managed application publishers to monitor the policy compliance states across their customer deployments. It collects all policy states and stores them into a centralized location to further analyze them and raise alerts when they become non complaint. This allows the Managed application publishers to provide a better service, support, and ensure the application remains secure.