This post addresses a customer business case about using Azure Monitor for alerting on available VNet subnet IP usage. It explains the investigation process, including collaboration with the Center of Excellence for monitoring called ‘CoE for Monitoring’, and highlights the usage of an Azure resource query to collect the IP usage. The final solution involved adapting the query to use an Azure Monitor to alert with a ready to deploy example.
Table of Contents
Customer business case
"Using Azure Monitor I want to get alerted when we are running out of available IP addresses on any virtual network (vNet) subnets"
Investigation
Together with CoE for Monitoring member, Robert Seso, we found an Azure resource query that could do the trick. This query was created by a Microsoft CSA see Retrieve available IP addresses from Azure Resource Graph – Stefan Stranger's Blog
Enhancing the query
However we found out that this query was not 100% correct. It was lacking to count also any internal load balancer front end IP assigning's and needed some optimalizations. After this fix, the next step was to convert the Azure Resource Graph (ARG )query to let it run in a Log analytics workspace. This is done by simply adding arg(""). in front of any azure resource graph table used. Next, due to log analytics limitations we needed to let the used join operator know to do a local action by adding hint.remote=local. (See also the reference links at the end of this post for more learnings)
For all changes see the script code below.
|
To test this query, open a log Analytics Workspace (LAW) and execute the query. See an output example below:
Defining the alert
Once the query is tested and verified for accuracy in the Log Analytics Workspace (LAW), the next step involves integrating it into Azure Monitor. This includes setting up an alert mechanism that effectively tracks IP utilization and notifies relevant stakeholders. To ensure correct monitoring, the query must include the required columns, such as 'id', which associates the vNet resource ID with the alert dimensions. By leveraging dimensions, alerts can be configured to trigger specifically for each vNet or subnet, enabling granular insights. Following the selection of a 'Custom Log Search' query and defining the metric to alert on, the configuration of thresholds and actions ensures that the monitoring system is both proactive and responsive.
Create alert steps
Go to the Azure portal and open the Azure Monitor pane and select Add alert
Set the target, basically since it’s an Azure Resource Graph (ARG) query can be anything.
Alert condition
In our case we don't want to send out 1 alert if any of the subnets run out of IP space, but we want to generate an alert per vNet / subnet. This is done by using dimensions. Before we can use this, we need to be sure this option can be enabled. This can be done by adding a column named 'id' to the query output having the vNet 'resource id' value. Next, we configure the threshold.
- Select a ‘Custom Log Search’ query.
- Add the created query.
- Select the field name of metric we want to alert on. In this case we alert on AvailableIPaddresses but we can also go for the PercentageUsed for example.
- Enable the dimensions.
- Select the 2 dimensions.
- Configure the threshold for the monitor to fire.
Alert actions
(Optional step) When an alert if fired it could send out an email/SMS or trigger a logic app. To configure this, we need to add an action group. Or for better control use the Alert processing rules if possible. (See also the reference links at the end of this post for more learnings)
- Continue to the Actions pane.
- Select an existing or create an action group
Configure Email target
In case we create a new action group
- Provide a name
- Add an email notification type
- Enable and provide an email address
Alert details
The final step is to provide a name, servility and to resolve alerts and start the creation.
-
Configure the subscription and resource group to store this alert - Configure the alert severity
- Configure the identity that is running this alert rule. See details below.
- If you want to alert to resolve after the threshold is not hit enable this. If you want every time the threshold is hit a new alert generated, then disable this option.
Configure the Managed identity access
After deploying the alert rule, it's important to ensure proper role assignments and access levels to maintain functionality and security. Integrate the system or user-assigned identities to align with the operational needs of your application, taking care to select the least privilege necessary while enabling seamless monitoring. By doing so, the alert rule remains both effective and adaptable, perfectly positioned to react to specific thresholds or track resource conditions as dictated by the query structure and subscription scope.
In our case assigning a Managed identity access is needed because we are using the Azure Resource Graph (ARG) provider (see in the query the arg() statements). Go to the created Azure alert rule and enable the 'Identity' with at least a 'Monitoring Reader' role on for example, the subscription scope.
-
Navigate to the just created azure monitor rule - Access the Identity pane
- Select system assigned if you want the application principal created also be removed if the alert rule is deleted. Or user assigned if you want to reuse an application principal.
- Enable the identity
- Open the Assign role wizard
Assign role wizard
Once you've set up the identity, the next step is to narrow down the monitoring scope to match what you want the alerts to cover. Setting the subscription scope correctly makes sure the monitoring focuses on exactly what you need. Since in our example we want to report on all vNets in our subscription we configure the scope to ‘subscription’
- Apply the scope, in this use case we want to get all vNet IP data, so we select subscription
- Apply the least security role. That would be ‘Monitoring Reader’
The Result
Now the alert rule is created and activated you will see an alert when it’s fired like below.
Deployable example
As a starter see the created ARM template that is pasted below. This will create the Azure monitor that will trigger when the threshold PercentageUsed indicates that 95% of the IP address space is being used.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"scheduledqueryrules_ipsubnetmonitoring_name": {
"defaultValue": "ipsubnetmonitoring",
"type": "String"
},
"scheduledqueryrules_ipsubnetmonitoring_scope": {
"type": "string",
"defaultValue": "Full Resource ID of the resource emitting the metric that will be used for the comparison. For example scope subscription level /subscriptions/00000000-0000-0000-0000-0000-00000000",
"metadata": {
"description": "The scope for the Azure Monitor resources."
}
}
},
"variables": {},
"resources": [
{
"type": "microsoft.insights/scheduledqueryrules",
"apiVersion": "2024-01-01-preview",
"name": "[parameters('scheduledqueryrules_ipsubnetmonitoring_name')]",
"location": "westeurope",
"kind": "LogAlert",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"displayName": "[parameters('scheduledqueryrules_ipsubnetmonitoring_name')]",
"severity": 1,
"enabled": true,
"evaluationFrequency": "PT5M",
"scopes": [ "[parameters('scheduledqueryrules_ipsubnetmonitoring_scope')]" ],
"targetResourceTypes": [
],
"windowSize": "PT5M",
"criteria": {
"allOf": [
{
"query": "arg(\"\").resources\n| where type =~ 'Microsoft.Network/virtualNetworks'\n//| where id has 'SCOMSKYPE01'\n| extend addressPrefixes=array_length(properties.addressSpace.addressPrefixes)\n| extend vNetAddressSpace=properties.addressSpace.addressPrefixes\n| where (isnotnull(properties.subnets))\n| mv-expand subnet=properties.subnets limit 2000\n| extend subnetId = subnet.id\n| extend virtualNetwork = name\n| extend subnetPrefix = subnet.properties.addressPrefix\n| extend subnets = properties.subnets\n| extend subnetName = tostring(subnet.name)\n| extend prefixLength = toint(split(subnetPrefix, \"/\")[1])\n| extend addressPrefix = split(subnetPrefix, \"/\")[0]\n| extend numberOfIpAddresses = trim_end(\".0\", tostring(pow(2, 32 - prefixLength) - 5))\n| extend startIp = strcat(strcat_array((array_slice(split(addressPrefix, '.'), 0, 2)), \".\"), \".\", tostring(0))\n| extend endIp = strcat(strcat_array((array_slice(split(addressPrefix, '.'), 0, 2)), \".\"), \".\", trim_end(\".0\", tostring(pow(2, 32 - prefixLength) - 5)))\n| extend endIPNew = case(\n prefixLength == 23,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '1.255'),\n prefixLength == 22,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '3.255'),\n prefixLength == 21,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '7.255'),\n prefixLength == 20,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '15.255'),\n prefixLength == 19,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '31.255'),\n prefixLength == 18,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '63.255'),\n prefixLength == 17,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '127.255'),\n prefixLength == 16,\n strcat(strcat(strcat_array((array_slice(split(startIp, '.'), 0, 1)), \".\"), \".\"), '255.255'),\n 'unknown'\n )\n| extend finalendIPaddress = iff(endIPNew == \"unknown\", endIp, endIPNew) \n| join hint.remote=local kind=leftouter (\n arg(\"\").resources\n | where type =~ 'microsoft.network/networkinterfaces' or type =~ 'microsoft.network/loadbalancers'\n | project\n id,\n ipConfigurations = iff(isnull(properties.ipConfigurations), properties.frontendIPConfigurations, properties.ipConfigurations),\n subscriptionId\n | mvexpand ipConfigurations\n | project\n id,\n subnetId = tostring(iff(isnull(ipConfigurations.properties.subnet.id), ipConfigurations.properties.subnet.id, ipConfigurations.properties.subnet.id)),\n subscriptionId\n | parse kind=regex subnetId with '/virtualNetworks/' virtualNetwork '/subnets/' subnetName\n | summarize usedIPAddresses = count() by subnetName, virtualNetwork, subscriptionId \n )\n on subnetName, virtualNetwork, subscriptionId\n| extend usedIPAddresses_new = iff(isnull(usedIPAddresses), 0, usedIPAddresses)\n| extend AvailableIPAddresses = (toint(numberOfIpAddresses) - usedIPAddresses_new) \n| extend PercentageUsed = round(100.0 * (todouble(usedIPAddresses_new) / todouble(numberOfIpAddresses)), 2)\n| project\n id,\n subscriptionId,\n resourceGroup,\n virtualNetwork,\n subnetId,\n subnetName,\n IPRange = strcat(startIp, \" - \", finalendIPaddress),\n numberOfIpAddresses,\n usedIPAddresses,\n AvailableIPAddresses,\n PercentageUsed\n",
"timeAggregation": "Total",
"metricMeasureColumn": "PercentageUsed",
"dimensions": [
{
"name": "virtualNetwork",
"operator": "Include",
"values": [
"*"
]
},
{
"name": "subnetName",
"operator": "Include",
"values": [
"*"
]
}
],
"resourceIdColumn": "id",
"operator": "GreaterThanOrEqual",
"threshold": 95,
"failingPeriods": {
"numberOfEvaluationPeriods": 1,
"minFailingPeriodsToAlert": 1
}
}
]
},
"autoMitigate": true
}
}
]
}Deploy ARM template
To proceed, ensure the ARM template is correctly formatted and validated to avoid deployment errors. Once validated, move to the deployment interface, where you can configure necessary parameters such as alerts and resource properties. Follow the step-by-step process to align template configurations seamlessly, ensuring deployment compatibility with your specified environment.
- Open the ‘deploy a custom template’ pane
- Select ‘build your own template in the editor’
The cusomized template window will open
-
Paste the ARM template into the pane and save it.
The final step is to start the deployment.
Back in the main wizard
- Specify the resource group where the alert will be deployed
- Specify the alert name and the alert scope (in this example on subscription level)
Check the created alert rule.
- Go to the resource group that you specified in the template
- Search for the type ‘Log search alert rule’
Conclusion
By following these steps, you can ensure a streamlined deployment process to monitor the IP usage using Azure Monitor alerts. This approach not only enhances resource monitoring but also simplifies alert management for your infrastructure.
Related Resources
To learn more on this capability, refer to:
- How Azure Resource Graph uses alerts to monitor resources - Azure Resource Graph | Microsoft Learn
- Create Azure Monitor alert rules - Azure Monitor | Microsoft Learn
- Alert processing rules for Azure Monitor alerts - Azure Monitor | Microsoft Learn
- Cross-cluster join - Kusto | Microsoft Learn
- Troubleshoot Azure Resource Graph alerts - Azure Resource Graph | Microsoft Learn
- Managed identities for Azure resources | Microsoft Learn
Happy Monitoring!
Michel Kamp
TZ lead CoE for Monitoring at Microsoft SfMC (Support for Mission Critical)
Microsoft