Public Preview: Azure Monitor pipeline transformations
Overview The Azure Monitor pipeline extends the data collection capabilities of Azure Monitor to edge and multi-cloud environments. It enables at-scale data collection (data collection over 100k EPS), and routing of telemetry data before it's sent to the cloud. The pipeline can cache data locally and sync with the cloud when connectivity is restored and route telemetry to Azure Monitor in cases of intermittent connectivity. Learn more about this here - Configure Azure Monitor pipeline - Azure Monitor | Microsoft Learn Why transformations matter Lower Costs: Filter and aggregate before ingestion to reduce ingestion volume and in turn lower ingestion costs Better Analytics: Standardized schemas mean faster queries and cleaner dashboards. Future-Proof: Built-in schema validation prevents surprises during deployment. Azure Monitor pipeline solves the challenges of high ingestion costs and complex analytics by enabling transformations before ingestion, so your data is clean, structured, and optimized before it even hits your Log Analytics Workspace. Check out a quick demo here - If the player doesn’t load, open the video in a new window: Open video Key features in public preview 1. Schema change detection One of the most exciting additions is schema validation for Syslog and CEF : Integrated into the “Check KQL Syntax” button in the Strato UI. Detects if your transformation introduces schema changes that break compatibility with standard tables. Provides actionable guidance: Option 1: Remove schema-changing transformations like aggregations. Option 2: Send data to a custom tables that support custom schemas. This ensures your pipeline remains robust and compliant with analytics requirements. For example, in the picture below, extending to new columns that don't match the schema of the syslog table throws an error during validation and asks the user to send to a custom table or remove the transformations. While in the case of the example below, filtering does not modify the schema of the data at all and so no validation error is thrown, and the user is able to send it to a standard table directly. 2. Pre-built KQL templates Apply ready-to-use templates for common transformations. Save time and minimize errors when writing queries. 3. Automatic schema standardization for syslog and CEF Automatically schematize CEF and syslog data to fit standard tables without any added transformations to convert raw data to syslog/CEF from the user. 4. Advanced filtering Drop unwanted events based on attributes like: Syslog: Facility, ProcessName, SeverityLevel. CEF: DeviceVendor, DestinationPort. Reduce noise and optimize ingestion costs. 5. Aggregation for high-volume logs Group events by key fields (e.g., DestinationIP, DeviceVendor) into 1-minute intervals. Summarize high-frequency logs for actionable insights. 6. Drop unnecessary fields Remove redundant columns to streamline data and reduce storage overhead. Supported KQL sunctions 1. Aggregation summarize (by), sum, max, min, avg, count, bin 2. Filtering where, contains, has, in, and, or, equality (==, !=), comparison (>, >=, <, <=) 3. Schematization extend, project, project-away, project-rename, project-keep, iif, case, coalesce, parse_json 4. Variables for Expressions or Functions let 5. Other Functions String: strlen, replace_string, substring, strcat, strcat_delim, extract Conversion: tostring, toint, tobool, tofloat, tolong, toreal, todouble, todatetime, totimespan Get started today Head to the Azure Portal and explore the new Azure Monitor pipeline transformations UI. Apply templates, validate your KQL, and experience the power of Azure Monitor pipeline transformations. Find more information on the public docs here - Configure Azure Monitor pipeline transformations - Azure Monitor | Microsoft Learn
Accelerating SCOM to Azure Monitor Migrations with Automated Analysis and ARM Template Generation
Accelerating SCOM to Azure Monitor Migrations with Automated Analysis and ARM Template Generation Azure Monitor has become the foundation for modern, cloud-scale monitoring on Azure. Built to handle massive volumes of telemetry across infrastructure, applications, and services, it provides a unified platform for metrics, logs, alerts, dashboards, and automation. As organizations continue to modernize their environments, Azure Monitor is increasingly the target state for enterprise monitoring strategies. With Azure Monitor increasingly becoming the destination platform, many organizations face a familiar challenge: migrating from System Center Operations Manager (SCOM). While both platforms serve the same fundamental purpose—keeping your infrastructure healthy and alerting you to problems—the migration path isn’t always straightforward. SCOM Management Packs contain years of accumulated monitoring logic: performance thresholds, event correlation rules, service discoveries, and custom scripts. Translating all of this into Azure Monitor’s paradigm of Log Analytics queries, alert rules, and Data Collection Rules can be a significant undertaking. To help with this challenge, members of the community have built and shared a tool that automates much of the analysis and artifact generation. The community-driven SCOM to Azure Monitor Migration Tool accepts Management Pack XML files and produces several outputs designed to accelerate migration planning and execution. The tool parses the Management Pack structure and identifies all monitors, rules, discoveries, and classes. Each component is analyzed for migration complexity: some translate directly to Azure Monitor equivalents, while others require custom implementation or may not have a direct equivalent. Results are organized into two clear categories: Auto-Migrated Components – Covered by the generated templates and ready for deployment Requires Manual Migration – Components that need custom implementation or review Instead of manually authoring Azure Resource Manager templates, the tool generates deployable infrastructure-as-code artifacts, including: Scheduled Query Alert rules mapped from SCOM monitors and rules Data Collection Rules for performance counters and Windows Events Custom Log DCRs for collecting script-generated log files Action Groups for notification routing Log Analytics workspace configuration (for new environments) For streamlined deployment, the tool offers a combined ARM template that deploys all resources in a single operation: Log Analytics workspace (create new or connect to an existing workspace) Action Groups with email notification All alert rules Data Collection Rules Monitoring Workbook One download, one deployment command — with configurable parameters for workspace settings, notification recipients, and custom log paths. The tool generates an Azure Monitor Workbook dashboard tailored to the Management Pack, including: Performance counter trends over time Event monitoring by severity with drill-down tables Service health overview (stopped services) Active alerts summary from Azure Resource Graph This provides immediate operational visibility once the monitoring configuration is deployed. Each migrated component includes the Kusto Query Language (KQL) equivalent of the original SCOM monitoring logic. These queries can be used as-is or refined to match environment-specific requirements. The workflow is designed to reduce the manual effort involved in migration planning: Export your Management Pack XML from SCOM Upload it to the tool Review the analysis — components are separated into auto-migrated and requires manual work Download the All-in-One ARM template (or individual templates) Customize parameters such as workspace name and action group recipients Deploy to your Azure subscription For a typical Management Pack, such as Windows Server Active Directory monitoring, you may see 120+ components that can be migrated directly, with an additional 15–20 components requiring manual review due to complex script logic or SCOM-specific functionality. The tool handles straightforward translations well: Performance threshold monitors become metric alerts or log-based alerts Windows Event collection rules become Data Collection Rule configurations Service monitors become scheduled query alerts against Heartbeat or Event tables Components that typically require manual attention: Complex PowerShell or VBScript probe actions Monitors that depend on SCOM-specific data sources Correlation rules spanning multiple data sources Custom workflows with proprietary logic The tool clearly identifies which category each component falls into, allowing teams to plan their migration effort with confidence. A Note on Validation This is a community tool, not an officially supported Microsoft product. Generated artifacts should always be reviewed and tested in a non-production environment before deployment. Every environment is different, and the tool makes reasonable assumptions that may require adjustment. Even so, starting with structured ARM templates and working KQL queries can significantly reduce time to deployment. Try It Out The tool is available at https://tinyurl.com/Scom2Azure.Upload a Management Pack, review the analysis, and see what your migration path looks like.Announcing Application Insights SDK 3.x for .NET
Microsoft remains committed to making OpenTelemetry the foundation of modern observability on Azure. Today, we’re excited to take the next step on that journey with a major release of the Application Insights SDK 3.x for .NET. Migrate to OpenTelemetry with a Major Version Bump With Application Insights SDK 3.x, developers can migrate to OpenTelemetry-based instrumentation with dramatically less effort. Until now, migrating from classic Application Insights SDK to the Azure Monitor OpenTelemetry Distro required a clean install and code updates. With this release, most customers can adopt OpenTelemetry simply by upgrading their SDK version. The new SDK automatically routes your classic Application Insights Track* APIs calls through a new mapping layer that emits OpenTelemetry signals under the hood. Why This Matters By upgrading, you gain: ✔ Vendor‑neutral OpenTelemetry APIs going forward You can immediately begin writing new code using OpenTelemetry APIs, ensuring future portability and alignment with industry standards. ✔ Access to the full OpenTelemetry ecosystem You can now easily plug in community instrumentation libraries and exporters. For example, collecting Redis Cache dependency data—previously not supported with Application Insights 2.x—becomes straightforward. ✔ Multi‑exporter support Export to Azure Monitor and another system (e.g., a SIEM or backend of your choice) simultaneously if your scenario requires it. What Still Requires Attention: Initializers and Processors One area where automatic migration is not possible is telemetry processors and telemetry initializers. These Application Insights extensibility points were extremely flexible, allowing custom property injection, filtering, or deletion logic. OpenTelemetry supports similar behavior, but through more structured concepts such as span processors. See here for a full list of breaking changes. On a positive note, these OpenTelemetry components generally deliver better performance and clearer behavior. Our documentation assists with migration, and we plan to release an MCP with guardrails to assist LLM in accurate coding. Keeping the essence of Azure Monitor Application Insights While OpenTelemetry encourages the use of the OpenTelemetry-Collector, we remain committed to preserving the simplicity that customers love about Azure Monitor Application Insights. The Azure Monitor OpenTelemetry Distro is all that’s required to get started. It’s just a single NuGet package and you configure it with a Connection String. Telemetry flows in minutes. No Collector is required unless you explicitly want one. We are able to achieve this with extensive built‑in sampling to manage cost and a trace‑preservation algorithm, so you see complete traces. This keeps the “just works” spirit of Azure Monitor Application Insights intact, while aligning with OpenTelemetry standards. Feedback If you encounter issues during the upgrade, please open a support ticket—we want the migration to be smooth. If you’d like to share feedback or engage directly with the product team, email us at otel@microsoft.com. This is not an official support channel, but we read every email and appreciate hearing feedback directly from you!
Announcing the Public Preview of Azure Monitor health models
Troubleshooting modern cloud-native workloads has become increasingly complex. As applications scale across distributed services and regions, pinpointing the root cause of performance degradation or outages often requires navigating a maze of disconnected signals, metrics, and alerts. This fragmented experience slows down troubleshooting and burdens engineering teams with manual correlation work. We address these challenges by introducing a unified, intelligent concept of workload health that’s enriched with application context. Health models streamline how you monitor, assess, and respond to issues affecting your workloads. Built on Azure service groups, they provide an out-of-the-box model tailored to your environment, consolidate signals to reduce alert noise, and surface actionable insights — all designed to accelerate detection, diagnosis, and resolution across your Azure landscape. Overview Azure Monitor health models enable customers to monitor the health of their applications with ease and confidence. These models use the Azure-wide workload concept of service groups to infer the scope of workloads and provide out-of-the-box health criteria based on platform metrics for Azure resources. Key Capabilities Out-of-the-Box Health Model Customers often struggle with defining and monitoring the health of their workloads due to the variability of metrics across different Azure resources. Azure Monitor health models provide a simplified out-of-the-box health experience built using Azure service group membership. Customers can define the scope of their workload using service groups and receive default health criteria based on platform metrics. This includes recommended alert rules for various Azure resources, ensuring comprehensive monitoring coverage. Improved Detection of Workload Issues Isolating the root cause of workload issues can be time-consuming and challenging, especially when dealing with multiple signals from various resources. The health model aggregates health signals across the model to generate a single health notification, helping customers isolate the type of signal that became unhealthy. This enables quick identification of whether the issue is related to backend services or user-centric signals. Quick Impact Assessment Assessing the impact of workload issues across different regions and resources can be complex and slow, leading to delayed responses and prolonged downtime. The health model provides insights into which Azure resources or components have become unhealthy, which regions are affected, and the duration of the impact based on health history. This allows customers to quickly assess the scope and severity of issues within the workload. Localize the Issue Identifying the specific signals and resources that triggered a health state change can be difficult, leading to inefficient troubleshooting and resolution processes. Health models inform customers which signals triggered the health state change, and which service group members were affected. This enables quick isolation of the trouble source and notifies the relevant team, streamlining the troubleshooting process. Customizable Health Criteria for Bespoke Workloads Many organizations operate complex, bespoke workloads that require their own specific health definitions. Relying solely on default platform metrics can lead to blind spots or false positives, making it difficult to accurately assess the true health of these custom applications. Azure Monitor health models allow customers to tailor health assessments by adding custom health signals. These signals can be sourced from Azure Monitor data such as Application Insights, Managed Prometheus, and Log Analytics. This flexibility empowers teams to tune the health model to reflect the unique characteristics and performance indicators of their workloads, ensuring more precise and actionable health insights. Getting Started Ready to simplify and accelerate how you monitor the health of your workloads? Getting started with Azure Monitor health models is easy — and during the public preview, it’s completely free to use. Pricing details will be shared ahead of general availability (GA), so you can plan with confidence. Start Monitoring in Minutes Define Your Service Group Create your service group and add the relevant resources as members to the service group. If you don’t yet have access to service groups, you can join here. Create Your Health Model In the Azure Portal navigate to Health Models and create your first model. You’ll get out-of-the-box health criteria automatically applied. Customize to Fit Your Needs In many cases the default health signals may suit your needs, but we support customization as well. Investigate and Act Use the health timeline and our alerting integration to quickly assess impact, isolate issues, and take action — all from a single pane of glass. You can access health models today in the Azure portal! For more details on how to get started with health models, please refer to our documentation. We Want to Hear From You Azure Monitor health models are built with our customers in mind — and your feedback is essential to shaping the future of this experience. Whether you're using the out-of-the-box health model or customizing it to fit your unique workloads, we want to know what’s working well and where we can improve. Share Your Feedback Use the “Give Feedback” feature directly within the Azure Monitor health models experience to send us your thoughts in context. Post your ideas in the Azure Monitor community. Prefer email? Reach out to us at azmonhealthmodels@service.microsoft.com — we’re listening. Your insights help us prioritize features, improve usability, and ensure Azure Monitor continues to meet the evolving needs of modern cloud-native operations.
Data Collection Rule : XPath queries to filter 7036 without WMI etc
Hi, In PowerShell on server I’m trying to filter out some events from Event Id 7036 Service Control Manager Start stop services. I’m trying to filter out WMI Performance Adapter, so I don’t want to have those events imported in log analytic workspace with data collection rule. Can you help me what I’m doing wrong ? $XPath = 'System!*[System[(EventID="7036")]] and [EventData[Data[@Name="param1"]!="WMI Performance Adapter"]]' Get-WinEvent -FilterXPath $XPath Get-WinEvent : Could not retrieve information about the Security log. Error: Attempted to perform an unauthorized operation.. At line:3 char:1 + Get-WinEvent -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Get-WinEvent], Exception + FullyQualifiedErrorId : LogInfoUnavailable,Microsoft.PowerShell.Commands.GetWinEventCommand Get-WinEvent : No events were found that match the specified selection criteria. At line:3 char:1 + Get-WinEvent -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (:) [Get-WinEvent], Exception + FullyQualifiedErrorId : NoMatchingEventsFound,Microsoft.PowerShell.Commands.GetWinEventCommand $XPath = 'System!*[System[(EventID="7036")]] and [EventData[Data[@Name="param1"]!="WMI Performance Adapter"]]' Get-WinEvent -LogName 'System' -FilterXPath $XPath Get-WinEvent : The specified query is invalid At line:2 char:1 + Get-WinEvent -LogName 'System' -FilterXPath $XPath + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Get-WinEvent], EventLogException + FullyQualifiedErrorId : System.Diagnostics.Eventing.Reader.EventLogException,Microsoft.PowerShell.Commands.GetWinEventCommand
