At-Scale Failure Reporting for Azure Update Manager
Introduction Azure Update Manager simplifies patching across Azure virtual machines and Azure Arc-enabled servers by providing a centralized platform for patch assessment and installation. However, as environments scale, a key challenge emerges—efficiently identifying and troubleshooting patch failures across large fleets of machines. While Azure Update Manager surfaces detailed error messages in the Azure portal, this information is typically available only at an individual machine level. In enterprise environments managing hundreds or thousands of systems, drilling into each VM to find error details quickly becomes impractical. In this article, we walk through a real-world use case and demonstrate how to leverage Azure Resource Graph (ARG) to extract failed machines along with their error details for a specific maintenance run—using a single query. The Challenge: Scaling Patch Failure Visibility In a large enterprise deployment, Azure Update Manager was configured to manage patching across: Windows and Linux virtual machines Azure cloud VMs and Arc-enabled on‑premises servers Multiple regions and subscriptions While patching operations were largely successful, a subset of machines experienced failures. The key challenges faced by the operations team were: Error messages were visible only by drilling into each failed VM in the portal No built‑in way to aggregate failures across all machines Lack of a simple mechanism to export: Failed VMs Error codes Error messages The team needed a scalable, query‑driven approach to analyze failures across an entire maintenance run. Key Insight: Where Azure Update Manager Stores Data Azure Update Manager does not rely on Log Analytics to store operational results. Instead: Patch assessment and installation results are stored in Azure Resource Graph Azure Resource Graph acts as a centralized, queryable store for update operations This design enables powerful querying without requiring additional ingestion, configuration, or cost overhead. Understanding Maintenance Runs and Correlation IDs Each Azure Update Manager maintenance run generates a unique identifier: properties.correlationId represents the maintenance (schedule) run ID All machines involved in the same patch cycle share this ID This allows all machines within a single patch execution to be correlated and queried collectively. The Solution: Query Failed VMs with Error Messages Azure Resource Graph allows querying failures at scale using the maintenanceresources dataset. Core Query (Kusto Query Language) 1 maintenanceresources 2 | where type =~ "microsoft.maintenance/applyupdates" 3 | where tostring(properties.correlationId) contains "<YourMaintenanceRunID>" 4 | where tostring(properties.status) =~ "Failed" 5 | project properties.resourceId, properties.errorCode, properties.errorMessage What This Query Delivers All machines that failed in a specific maintenance run Error codes for troubleshooting Full error messages that are otherwise visible only in the Azure portal Note: Property names for error information can vary by environment. Validate available fields using Azure Resource Graph Explorer and adjust the project clause if required. Sample Output (Conceptual) Resource ID Error Code Error Message vm-01 0x80244007 Windows Update API failed vm-02 0x80072f8f Connectivity issue vm-03 1C WSUS configuration issue Advanced Scenario: Automatically Detecting the Latest Failed Maintenance Run In real-world scenarios, you may not always know the maintenance run ID. The following query dynamically identifies the most recent maintenance run that had failures, and then retrieves all failed machines from that run. 1 // Step 1: Identify the latest maintenance run ID with failures 2 let lastFailedRun = toscalar( 3 maintenanceresources 4 | extend runId = extract(@"applyupdates/(\d+)$", 1, properties.correlationId) 5 | where type =~ "microsoft.maintenance/applyupdates" 6 | where tostring(properties.status) =~ "Failed" 7 | order by tostring(properties.startDateTime) desc 8 | take 1 9 | project runId 10 ); 11 // Step 2: Query all failed VMs from that run 12 maintenanceresources 13 | where type =~ "microsoft.maintenance/applyupdates" 14 | where tostring(properties.correlationId) contains lastFailedRun 15 | where tostring(properties.status) =~ "Failed" 16 | project properties.resourceId, properties.errorCode, properties.errorMessage This approach is ideal for automation, scheduled reporting, and dashboard scenarios. Why This Approach Matters Operational Efficiency Eliminates manual portal navigation Provides consolidated failure insights in seconds Scalability Works across large, distributed environments Supports both Azure and hybrid (Arc‑enabled) machines Automation Ready Can be integrated into scripts, dashboards, and reporting pipelines Enables proactive monitoring and alerting scenarios Best Practices for Enterprise Patch Reporting To maximize the value of this approach: Capture and track maintenance run IDs Use Azure Resource Graph as the primary reporting layer Build reusable queries for different patch scenarios Export reports for compliance and auditing Correlate failures with root‑cause trends over time Conclusion As organizations scale patching operations with Azure Update Manager, visibility, speed, and automation become essential. While the Azure portal is effective for per‑machine troubleshooting, it is not optimized for fleet‑level analysis. Azure Resource Graph fills this gap by enabling a shift from manual troubleshooting to automated, query‑driven failure analysis at scale. By adopting this approach, teams can significantly improve operational efficiency, reduce mean time to resolution, and build a more mature patch management strategy. Final takeaway: Don’t rely only on the portal Leverage Azure Resource Graph to operationalize patch insights at enterprise scale References Azure Update Manager – Query resources with Azure Resource Graph https://learn.microsoft.com/azure/update-manager/query-logs Azure Update Manager – Troubleshooting guide https://learn.microsoft.com/azure/update-manager/troubleshoot Sample Azure Resource Graph queries for Azure Update Manager https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/update-manager/sample-query-logs.mdAutomating Windows Server Licensing Benefits with Azure Arc Policy
Introduction: Managing Windows Server benefits licensing across hybrid environments can be challenging. Azure Arc combined with Azure Policy simplifies this by automatically enforcing licensing compliance. This blog explains how the provided policy works and how to deploy it. Why implement this policy? Automating Windows Server Licensing Benefits with Azure Arc Policy ensures that all eligible machines are seamlessly enabled for essential management services, including Azure Update Manager, Best Practice Assessment, Change Tracking, Inventory, and Windows Admin Center integration. For organizations managing hundreds or thousands of servers, manual enablement can be time-consuming and error prone. This policy continuously monitors your environment, automatically identifying newly added machines and highlighting those missing the required benefits, so you can maintain compliance and streamline operations at scale This learn document detail the benefits available when Windows Server is connected via Azure Arc, especially for machines with Software Assurance or subscription licenses: https://learn.microsoft.com/en-us/azure/azure-arc/servers/windows-server-management-overview?tabs=portal Note – Ensure that your organization has the proper Software Assurance Benefits to cover the machines that are being assigned. Please reference this link for billing information Windows Server Management enabled by Azure Arc - Azure Arc | Microsoft Learn "Customers need to explicitly attest for their Azure Arc-enabled servers or enroll in Windows Server pay-as-you-go to be exempt from billing for these services. Eligibility isn't inferred directly from the enablement to Azure Arc. Eligibility is not inferred from licensing status for the Azure Arc-enabled SQL Server instances that may be connected to an Azure Arc-enabled." Policy Purpose and Logic The policy ensures Arc-enabled Windows Servers are licensed correctly. It evaluates machines based on OS type, license status, and conditions for Software Assurance or Pay-As-You-Go. If compliance is missing, a remediation policy deploys the appropriate license profile. Key Conditions Applies to resources of type Microsoft.HybridCompute/machines with osType = windows. Checks if licenseProfile.licenseStatus equals Licensed. Uses existenceCondition to determine if the machine should have SA or PAYG licensing based on osSku and licenseChannel. Deployment Details The policy uses DeployIfNotExists effect. It deploys licenseProfiles under the Arc machine resource. Two scenarios are handled: Pay-As-You-Go: If licenseChannel contains 'PGS', productProfile.subscriptionStatus is set to Enabled. Software Assurance: If licenseChannel does not contain 'PGS', softwareAssuranceCustomer is set to true. The Policy The policy is located in GitHub (Link) and AzPolicyAdvertiser (Link). Download the policy files to be used in the following steps. Policy Description For 2025 server, if license type is Pay-as-you-go, then this will check the Pay-as-you-go box in license menu. If 2025 and not Pay-as-you-go license or not 2025 server then check Software Assurance box. This policy only checks Windows Server resources and will NOT check unlicensed servers How to Deploy the Policy After downloading the policy file, use Az PowerShell to create and assign the policy: #Create policy definition New-AzPolicyDefinition ` -Name "activate-azure-benefits-for-windows-arc-machines" ` -DisplayName "Activate Azure Benefits for Windows Arc Machines" ` -Policy 'azurepolicy.json' ` -ManagementGroupName "<MyManagementGroup>" ` -Mode Indexed #Assign policy definition $Policy = Get-AzPolicyDefinition -Name 'activate-azure-benefits-for-windows-arc-machines' -ManagementGroupName "<ScopeOfDefinitionCreation>" New-AzPolicyAssignment ` -Name "activate-arc-benefits" ` -DisplayName "Activate Azure Benefits for Windows Arc Machines" ` -PolicyDefinition $Policy ` -Scope "/providers/Microsoft.Management/managementGroups/<MyManagementGroup>" ` -Location 'eastus' ` -IdentityType 'SystemAssigned' # Optional use subscriptions instead of management groups. # or "/subscriptions/<SubscriptionId>" You can also copy and paste the contents of the policy into the portal or use a policy-as-code solution of your choice. Compliance The compliance blade of the Azure Policy will show the machines that do not abide by the policy definition. In this example many of the machines are not enabled for the Windows Server Benefits. The next step will be to use remediation tasks to enable these machines. On the Policy Remediation blade, you can initiate a remediation task to add the machines to enable the Azure Arc Benefits. Choose between the two radio button options for remediating all the selected locations, a single location, or select specific resources to remediate. When the Remediate button is pressed, a task is summitted and a notification will be displaced when the task is completed. The process may take some time and a status of In Progress will be displayed until the status changes to Complete. After this is completed go back and look at the Azure Arc Benefits – Windows Server Blade and you will see the machines activated. Note on Pay-as-you-go enablement When a Windows machine is deployed using Pay-as-you-go, as an example a new Windows Server 2025 machine, the status of the license after creation will be “Unlicensed” as shown below. The policy is not evaluating Unlicensed machines. The machine will need to have the Pay-as-you-go with Azure check box checked at least one time to “License” the machine. After the machine is Licensed the License details will show: Now if the machine would have the benefits removed in the future by unchecking the box, the machine will be audited with the policy. As an example, the Arc machine would show that the License type is Pay-as-you-go, Licensed, Disabled (for the Azure Benefits). Summary This policy automates Windows Server licensing for Arc-enabled machines. It ensures compliance by deploying license profiles for Software Assurance or Pay-As-You-Go scenarios. Deploying this policy reduces manual effort and enforces consistent licensing across your hybrid environment.
Revolutionizing log collection with Azure Monitor Agent
The much awaited deprecation of the MMA agent is finally here. While still sunsetting, this blog post reviews the advantages of AMA, different deployment options and important updates to your favorite Windows, Syslog and CEF events via AMA data connectors.
Azure Arc and Defender for Servers: Connectivity and Monitoring Script
2. Overview of Defender for Servers Microsoft Defender for Servers is a plan within Microsoft Defender for Cloud that provides advanced threat protection for Windows and Linux servers, whether they are hosted in Azure, on-premises, or in other cloud environments. It includes capabilities such as endpoint detection and response (EDR), vulnerability assessment, file integrity monitoring, and adaptive application controls. Defender for Servers integrates with Microsoft Defender for Endpoint to provide unified security management and threat detection. For more information on Defender for servers visit documentation at the link below. https://learn.microsoft.com/en-us/azure/defender-for-cloud/tutorial-enable-servers-plan 3. Onboarding On-Premises Servers via Azure Arc To onboard on-premises servers to Defender for Servers, Azure Arc is used to project non-Azure machines into Azure. This enables the application of Azure policies, monitoring, and security configurations. The onboarding process involves: - Installing the Azure Connected Machine Agent on the server - Registering the server with Azure Arc - Enabling Defender for Servers in Microsoft Defender for Cloud - Ensuring the server is reporting and compliant with security policies. For more information on connecting on-premises servers to Azure Arc visit documentation in the link below. Connect hybrid machines to Azure using a deployment script - Azure Arc | Microsoft Learn 4. Script Purpose and Details This PowerShell script is designed to help infrastructure administrators verify the health of the HIMDS service (used by Microsoft Defender for Endpoint) and the connectivity status of the Azure Connected Machine Agent (Azure Arc) on multiple servers. It is especially useful in scenarios where administrators do not have access to the Azure portal but need to ensure that servers are properly onboarded and connected. Key functions of the script include: - Reading a list of computer names from a CSV file - Checking the status of the HIMDS service on each machine - Running the 'azcmagent show' command remotely to verify Azure Arc connectivity - Logging and displaying the results with color-coded output 5. PowerShell Script # Path to the CSV file $csvPath = "C:\Path\To\computers.csv" # Import computer names from CSV $computers = Import-Csv -Path $csvPath | Select-Object -ExpandProperty ComputerName # Array to store connected machines $connectedMachines = @() foreach ($computer in $computers) { Write-Host "Checking $computer..." -ForegroundColor Cyan try { # Check HIMDS service $himdsService = Get-Service -ComputerName $computer -Name "himds" -ErrorAction Stop $himdsStatus = $himdsService.Status # Run azcmagent show remotely and parse output $azcmOutput = Invoke-Command -ComputerName $computer -ScriptBlock { try { $output = azcmagent show | Out-String return $output } catch { Write-Error "Failed to run azcmagent: $_" return $null } } if ($azcmOutput -ne $null) { $statusLine = $azcmOutput -split "`n" | Where-Object { $_ -match "Agent Status\s*:\s*Connected" } if ($statusLine) { Write-Host "[$computer] HIMDS Service: $himdsStatus, Azure Arc Status: Connected" -ForegroundColor Green $connectedMachines += $computer } else { Write-Host "[$computer] HIMDS Service: $himdsStatus, Azure Arc Status: Not Connected" -ForegroundColor Yellow } } else { Write-Host "[$computer] HIMDS Service: $himdsStatus, Azure Arc Status: Unknown (command failed)" -ForegroundColor Red } } catch { Write-Host "[$computer] Error: $_" -ForegroundColor Red } } # Output connected machines Write-Host "`nConnected Machines:" -ForegroundColor Cyan $connectedMachines | ForEach-Object { Write-Host $_ -ForegroundColor Green } 6. How It Simplifies Administrative Tasks This script streamlines the process of verifying Azure Arc connectivity across multiple servers. Instead of manually logging into each server and running individual checks, administrators can execute this script to: - Quickly identify which machines are connected to Azure Arc - Detect issues with the HIMDS service - Generate a list of healthy and connected machines - Save time and reduce the risk of human error
Leveraging CEF with Azure Monitor Agent (AMA) for GCP-Hosted Fortinet Firewall and Syslog Forwarder,
Blog Title: "Leveraging CEF with Azure Monitor Agent (AMA) for GCP-Hosted Fortinet Firewall and Syslog Forwarder, connected via Azure Arc to Stream Fortinet logs to Microsoft Sentinel with Data Collection Rules." Who Will Benefit from This Blog? This blog is tailored to those seeking to set up an integrated Security and Monitoring system using Microsoft tools like Azure Monitor Agent, Azure Arc, and Microsoft Sentinel, with a focus on handling Fortinet logs in CEF format hosted in Other Cloud viz GCP in Blog context. The audience that will benefit from this blog includes IT professionals, cloud administrators, security experts, and anyone responsible for managing and monitoring cloud infrastructure, particularly in multi-cloud environments.
