Azure Observability Blog articles

Take command of your Microsoft Foundry AI agents with the Azure Copilot Observability Agent

Ron Frenkel — Tue, 21 Jul 2026 16:28:48 GMT

Generative AI (GenAI) agents are moving from prototypes to production, running as part of real distributed applications. When a Microsoft Foundry agent slows down, returns errors, or spikes in token usage, teams need to know why - fast, and with evidence. The Azure Copilot Observability Agent, generally available and powered by Azure Monitor, makes Foundry and GenAI agent investigations a first-class scenario. You can investigate agent failures, latency, tool-call errors, and backend saturation through the same signals that run your Azure estate.

Why AI observability matters in production

Building an agent is one thing. Running a fleet of them at scale, in front of real users, is another. AI applications also introduce behaviors that traditional monitoring wasn't built to explain:

Non-deterministic outputs - the same prompt can take a different path through the agent's logic.
New failure modes - hallucinations, tool-call failures, and reasoning that silently goes off track.
Cost and latency that shift with behavior - tokens and response time depend on what the agent decides to do, which models are used and so on.
Fleet-scale complexity - many agents, across frameworks and hosting, each part of a larger system.

AI observability closes this gap. You need to observe your AI workflows, understand their health, and investigate when an agent is failing, slow, or behaving in ways users notice.

From telemetry to analysis and investigation

Microsoft Foundry is your home for building and evaluating agents, while Azure Monitor makes those agents a first-class item you can observe through traces, tool calls, dependencies, token usage, and evaluation signals - across frameworks and hosting environments. Once you instrument your agents, their telemetry flows into Azure Monitor through Application Insights, and the Observability Agent reasons over it there. Setting this up is a straightforward process: instrument the agent, confirm telemetry is landing, then work with it in Azure Monitor. The Monitor AI Agents with Application Insights walks you through the experience and lets you jump directly from an agent's Monitoring tab in Foundry to Azure Monitor, and New capabilities to observe agents in Azure Monitor covers the wider telemetry pipeline behind it.

Telemetry tells you what happened, Analysis helps you make sense of it, and investigations tell you why. Most days start with analytics: you can chat with your observability data in natural language - ask for an error summary, a latency overview, or dependency health, then follow up to explore a pattern and decide whether something warrants a deeper look. So, the Observability Agent isn't only for investigations; it's also how you analyze and explore your agent telemetry day to day. When that exploration surfaces a real problem and you need the why, it escalates into deep investigations and root cause analysis for GenAI scenarios. The Azure Copilot Observability Agent reasons across your telemetry - logs, metrics, traces, alerts, dependencies, Foundry telemetry, and changes - together with Azure resource context, discovered topology, and relevant instructions. It frames hypotheses, gathers evidence, rules out weak explanations, and shows the reasoning behind its findings. Because it runs inside Azure Monitor, it inherits the access controls you already use - chat and investigations run with the signed-in user's identity and Azure role-based access control (RBAC), and your prompts and responses aren't used to train foundation models.

For Foundry and GenAI agents, you can investigate issues including agent failures and errors, latency, tool-call failures and failing Model Context Protocol (MCP) or backend dependencies, token spikes, and regressions after deployments - prioritizing the failure, latency, and throttling patterns that matter most.

It's powered by the same investigation model the Observability Agent applies to your applications, Azure Kubernetes Service (AKS), and virtual machines (VMs). Autonomous operations, in public preview, can analyze alerts in the background, correlate related alerts, and create Azure Monitor issues with findings and next steps - preparing context and reducing triage work, with humans in control.

Deep investigation through the Azure stack

One of the Observability Agent's key strengths is that it doesn't stop at the agent boundary. It investigates through the Azure stack - from the agent to its dependencies and tool calls, down to the infrastructure behind them.

Take a common Foundry pattern: agent failures caused by backend SQL saturation. When a Foundry GenAI agent starts failing, the root cause can go beyond the model or the prompt. The agent scopes to the affected agent, its dependencies and tool calls, and the SQL backend; correlates the failures with SQL timeouts and a spike in failed dependency calls; rules out alternatives like cache issues or a broader outage; and identifies a short SQL saturation window often driven by a high-cost query as the cause. For more patterns, read the deep investigation examples for the Azure Copilot Observability Agent.

Watch it in action:

A closer look: diagnosing an agent over a time range

To see how this plays out, point the Observability Agent at a specific agent and a time range say, the last 24 hours, and ask it to diagnose operation and performance. Instead of a single answer, it works through the problem the way an experienced operator would, across three lines of reasoning.

Failure patterns. It first establishes whether failures trace back to one dominant cause or several unrelated ones, and identifies which tools and dependencies are failing most and with what error and GenAI error types. It separates expected, app-level failures - validation errors, 400s and 404s - from platform and reliability failures like 5xx responses, timeouts, and dropped connections, so you don't spend time on noise. For each top failure type, it points to the likely locus: Main agent, sub-agent, tool, or a downstream service shows the supporting evidence, and proposes the most actionable fix.

Latency and slow runs. When a run is slow, it traces the end-to-end flow - entry point, agent steps, model and tool calls, and downstream dependencies - and shows where the time actually goes. If you give it a specific run, it walks that transaction step by step; if you don't, it looks across runs to find where latency concentrates: The large language model (LLM) call, a tool call, orchestration, or a downstream dependency, and tells you whether the slowness is systemic or isolated.

Throttling and quota pressure. It confirms whether you're seeing throttling signals - 429s, 503s, timeouts, and backoff patterns - and attributes them to the most likely source: the model or provider, the Foundry runtime, or another dependency. It also checks whether retries and backoff are helping or actually amplifying the problem.

The result isn't a dashboard you have to interpret. It's a prioritized, evidence-backed read on what's failing, what's slow, and what's being throttled - with the reasoning shown at each step, so you can act on it or dig further.

Get started

The Azure Copilot Observability Agent is generally available in Azure Monitor, and autonomous operations are available in public preview.

Instrument your Foundry and GenAI agents, then explore them in the agents view in Azure Monitor.
See what's possible once agents are instrumented in New capabilities to observe agents in Azure Monitor.
Learn how to use the Azure Copilot Observability Agent.
Explore how deep investigations in the Azure Copilot Observability Agent work.
Read the announcement: Azure Copilot Observability Agent is generally available, with autonomous operations in preview.

What's next for Foundry and GenAI agents

The Observability Agent already reasons about the scenarios that matter most for GenAI agents today: It investigates agent failures, latency, and performance, and it also understands token economy and works with evaluation signals. Here's where we're investing next:

Advanced token analysis - clearer visibility into where tokens go, which runs are most expensive, and how model and prompt choices affect cost.
Deeper failure attribution - pinpoint whether an issue comes from agent logic, model behavior, or a backend dependency, so you troubleshoot the right layer.
Richer evaluation correlation - easier ways to tie quality regressions to the specific runs, prompts, and changes behind them.

These additions extend the Observability Agent's existing strengths while keeping the same governed, evidence-first, human-in-control model you rely on today.

Stay connected

Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next.
Live webinar - a walkthrough of real Observability Agent scenarios, best practices, and what's available today - along with a look at what's coming next, and live Q&A with the product team. Register for the Observability Agent webinar

We'd love your feedback

The Observability Agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience or reach us at rofrenke@microsoft.com.

Understanding billing for the Azure Copilot Observability Agent

Noa Kuperberg — Thu, 16 Jul 2026 13:15:47 GMT

The Azure Copilot Observability Agent brings an agentic investigation experience directly into Azure Monitor. Teams can chat with their observability data, run deep investigations across application and infrastructure signals, and, in preview, use autonomous operations to correlate alerts and create issues for review. The common thread across these experiences is that the agent performs AI work on behalf of the user or configured workflow, and that work has a cost. Azure Copilot Observability Agent billing went into effect July 1, 2026.

This post explains the billing model at a practical level: what is measured, which agent operations are billable today, how usage appears to users, and how this differs from the standard Azure Monitor costs customers already manage for telemetry ingestion, retention, alerting, and other monitoring capabilities. For current list pricing and the most detailed billing guidance, always refer to the official billing documentation and the Azure Monitor pricing page.

A consumption-based model for agentic work

The Observability Agent uses a consumption-based pricing model: customers pay for the AI work the agent performs. This consumption is measured in Azure Agent Credits, or AAC. AAC provides a consistent unit for agent work across models and tokens used.

AAC is designed to reflect the amount of agentic processing required to complete a task. Simple questions, such as "what was the maximum latency of this app yesterday?", typically use few tokens. A deep investigation consumes more agent and tool work, and therefore typically incurs higher cost.

Note that a single agent operation - be it a chat question or a deep investigation - is currently capped at 500 AACs.

Charges are scoped to the Azure subscription of the monitored resource, or to the subscription of the named agent instance if one is used (required for autonomous operations). This keeps the cost associated with the environment where the agent is being used, and lets teams review agent consumption alongside other subscription-level Azure costs.

What is billable today

There are three main usage patterns to understand.

Chat - the agent's chat allows users to explore and analyze their observability data through natural-language questions about their Azure resources and their logs, metrics, traces, or related telemetry, and the agent performs the work needed to answer. This is typically the lowest-cost pattern because the scope is often focused and iterative.
Deep investigation - can be initiated through a number of entry points in the Azure Portal, and also through the chat (users can tell the agent to run a deep investigation). A deep investigation performs a broader analysis - it gathers signals, correlates findings, reasons across application, infrastructure, and Azure platform context, and produces an investigation report. Because this workflow runs multiple agent and tool steps, it typically consumes more AAC than chat.
Autonomous operations are currently in preview. Autonomous alert processing, triage and optional correlation can run in the background to group related alerts and reduce noise. Alert correlation itself isn’t billed during preview. If autonomous operations automatically run a deep investigation on an agent-created issue, that deep investigation is billable. This is an important distinction: preview correlation and issue creation are different from the investigation work that may be triggered as part of that flow.

How users see usage in the product

Cost transparency is part of the experience. After the agent returns a response in chat, users can open the usage indicator (hexagon-shaped icon) located next to the thumbs-up/down icons, to see how many AACs were used to generate that response. This makes consumption visible at the point where the user sees the value of the answer, rather than only later in a billing report.

This is especially useful because not all agent interactions are equal. A short question that summarizes a recent trend can require much less agent work than a long-running investigation that reviews multiple signals and hypotheses. Showing AAC usage per response helps users understand that relationship and adjust how they use the agent when needed.

How costs appear in Azure Cost Management

Teams can review the overall agent cost in subscription Cost Management. The product name appears as Azure Monitor Observability Agent, and the meter name appears as Observability Agent Azure Agent Credits.

This gives admins a familiar place to monitor consumption. The agent cost is not a replacement for standard Azure Monitor charges. Existing Azure Monitor costs — such as logs ingestion, retention, alerting, web tests, and other metered monitoring capabilities — continue to follow their own billing models.

Through Cost Analysis smart views, such as Services, users can select the Azure Monitor service and review specific entries of the Azure Monitor Observability Agent.

Practical guidance for teams

Start by using chat for focused exploration: ask about trends, errors, performance, anomalies, or a specific resource. Use deep investigations when you need a broader, multi-signal analysis of an incident or suspected root cause.

Review the AAC usage shown after agent responses so users can build intuition about which prompts stay lightweight and which workflows require deeper analysis.

Use Azure Cost Management to monitor the subscription-level cost of agent usage, and keep the Observability Agent cost distinct from standard Azure Monitor telemetry costs such as logs ingestion, retention, alerting, and web tests.

For current pricing details, billable behavior, and any updates to what is billed in preview or GA experiences, use the official billing documentation as the source of truth.

Coming up...

Looking ahead, we plan to introduce billing caps for the Observability Agent, giving customers greater control over monthly token consumption, capacity usage, and overall costs.

Learn more

We’d love your feedback

The Observability Agent continues to evolve based on real-world usage and operator feedback. Share feedback through the Give Feedback option in the product, or reach us at noakuper@microsoft.com.

Public Preview: Advanced platform metrics in Azure Monitor

alyssaschimm — Wed, 15 Jul 2026 14:43:18 GMT

We are excited to announce the Public Preview of advanced platform metrics for Azure Monitor, delivering more granular telemetry to help customers monitor and optimize their workloads more effectively. This new capability builds on Azure Monitor platform metrics, which continue to provide broad insight into the health, activity, and consumption of Azure resources. Advanced platform metrics add finer-grained signals, helping customers pinpoint changes and trends within resources more quickly and accurately.

Azure Storage is the first Azure resource to provide advanced platform metrics to customers. Today, Azure Storage users rely on platform metrics to understand overall storage account trends, but this account-level telemetry does not always show what is driving change. For example, a storage account may show steady capacity growth without revealing which specific container is responsible. That growth could be coming from one container used for backups, another storing application logs, or a staging container used by a data pipeline. Advanced platform metrics for Azure Monitor address this scenario by providing container-level visibility, helping customers quickly identify where growth is occurring, investigate unexpected consumption increases, and make more informed cost and capacity planning decisions.

What is available in Public Preview?

In Public Preview, the following advanced platform metrics are available for Azure Storage across all Azure public cloud regions:

Container Blob Capacity: The amount of storage used by a specific container in a storage account.
Container Blob Count: The number of blob objects in a specific container in a storage account.

Pricing and billing

Advanced platform metrics for Azure Monitor are offered as a paid capability during Public Preview. For the latest pricing details, see Azure Monitor pricing.

Getting started

Advanced platform metrics can be enabled per storage account through PowerShell or Azure CLI. For instructions on enabling, managing, and viewing Azure Storage advanced platform metrics, see Azure Platform Metrics for Azure Blob Storage (preview).

After advanced platform metrics are enabled for a storage account, they can be queried, visualized, and used for alerts through the same existing platform metrics experiences. Container Blob Capacity and Container Blob Count will appear in the Metric dropdown menu in Metrics Explorer, alongside all existing Azure Storage platform metrics.

Advanced platform metrics in Azure Monitor Metrics Explorer

Users can then select Apply splitting and choose Container name to view metrics for individual containers.

Apply splitting for advanced platform metrics

The chart below shows container-level capacity data for three containers.

Visualization of advanced platform metrics in Azure Monitor Metrics Explorer.

Azure Storage scenarios enabled by advanced platform metrics

Standard Azure Monitor platform metrics provide visibility into the storage account as a whole, such as total blob capacity or total object count. With the addition of advanced platform metrics for Azure Storage, customers can understand which individual containers are contributing to growth, object count increases, or operational issues, enabling scenarios such as:

Cost analysis and internal attribution: In shared storage accounts, identify which containers are consuming the most storage so teams can better understand which applications, environments, or business functions are driving usage without building custom reporting pipelines for this scenario.
Capacity planning and growth forecasting: Track storage growth at the container level to see which workloads are driving overall account growth and make more informed planning and budgeting decisions.
Runaway storage growth detection: Quickly isolate individual containers experiencing unexpected increases in capacity or object count, reducing investigation time when usage changes unexpectedly.

What's next?

As the feature moves toward General Availability (GA) and beyond, customers can expect to see more advanced platform metrics for Azure Storage, as well as new advanced platform metrics for other Azure resources. Continue to follow the Azure Observability Blog for the latest updates.

Feedback

We would love to hear your feedback on advanced platform metrics, including how your teams are using the feature to optimize workloads and additional advanced platform metrics that you would like to see onboarded to Azure Monitor. Please fill out this Azure Monitor advanced platform metrics feedback form, or email advancedplatformmetrics@microsoft.com.

Azure Monitor Observability Agent goes autonomous (preview)

Efrat_Ben_Porat — Wed, 08 Jul 2026 16:16:16 GMT

Autonomous operations for the Azure Copilot Observability Agent are now in public preview, alongside the agent's general availability. With autonomous operations enabled, the Observability Agent listens to your alerts as they fire, triages them in the background, and runs deep investigations on the issues it creates. Along the way, it correlates related alerts into a single issue - so your team starts from a small set of explained, investigated issues instead of a stream of raw alerts.

Until now, teams invoked the agent when they needed it - an interactive assistant, ready to investigate when you pointed it at a problem. Now it also prepares triage context continuously, on its own, while people stay responsible for decisions and any change to the environment.

See autonomous operations in action. The Observability Agent triages incoming alerts, correlates related ones into a single Azure Monitor issue, and runs a deep investigation automatically, with no human trigger.

From alerts to answers

Azure Monitor already gives you strong signals when something is wrong - across both metric and log alerts. Dynamic thresholds learn normal behavior and flag anomalies automatically, and that same anomaly detection now extends to log search alerts and, in preview, to Prometheus and OpenTelemetry metrics. Smart detection in Application Insights surfaces failures and performance anomalies without manual rules.

The hard part is what happens next: connecting dozens of alerts, working out what they share, and figuring out what's actually going on - before anyone can act. That's the work that still lands on a person, often in the middle of the night. It's exactly where the Observability Agent comes in.

What's in the public preview

In public preview, you can enable the Observability Agent to:

Promote individual prominent alerts into issues when you configure that with custom instructions.
Run a deep investigation automatically on every issue it creates.
Correlate related alerts into a single Azure Monitor issue, with a natural-language explanation of why they belong together.

You provision the agent once as a resource in your Azure environment - a dedicated identity to scope, govern, and assign autonomous tasks to - then turn on autonomous operations and it gets to work.

What it changes for your team

The outcome is fewer things to look at and faster triage:

Your team works from a short queue of meaningful issues, not a constant stream of alerts.
Each issue arrives with context, reasoning, and an investigation already attached.
Low-priority issues can be reviewed and dismissed in seconds.
The assembly work that used to come first now happens before anyone is paged.

People still make every decision and every change. The agent just makes sure they start with full context.

How it works

Your own instructions. Topology shows how services connect, but your team knows which boundaries matter: ownership, escalation paths, and the alerts that should always become issues. Custom instructions let you capture that in plain language and apply it going forward. For example:
"The billing service is owned by a different team with a separate on-call rotation. Even when billing alerts fire alongside clinical service alerts, treat them as separate issues."
Instructions shape how the agent correlates and creates issues. They don't grant permissions, bypass Azure RBAC, or change resources.
Automatic topology discovery. Point the agent at your Application Insights resource and it maps services, dependencies, and how they relate. That map becomes persisted knowledge the agent builds and reuses - the same context that grounds both its correlation decisions and its deep investigations, so reasoning reflects your real architecture instead of starting from scratch each time.
Deeper investigations. When the agent investigates a correlated issue, it starts from the whole picture: every related alert, every impacted resource, and the reasoning correlation already produced. The result is sharper root-cause hypotheses and recommendations that account for the full scope of impact.

In practice

A database latency spike triggers alerts across checkout, billing, and recommendation services. Without autonomous operations, each alert is triaged on its own.

With autonomous operations enabled, the Observability Agent groups the related alerts into one issue, explains the shared timeline, and starts investigating automatically. Because your custom instructions define billing as a separate ownership boundary, its alerts become a distinct issue routed to that team's rotation.

Responders start from two clear, ownership-aligned issues - each already investigated - instead of dozens of isolated alerts.

What's next

Autonomous operations mark the next step for the Observability Agent: from user-invoked analysis to continuous preparation. The agent assembles the context, explains the issue, and runs the investigation; your team reviews the evidence and decides what to do.

And once issues are created, you can act on them. Azure Monitor issues connect to Action Groups, so approved actions can flow into your existing workflows - more on that in a future post.

Next steps

Learn how to get started with the Azure Copilot Observability Agent.
Review the preview details in Autonomous operations in the Observability Agent.
Explore how investigations work in Deep investigations in the Observability Agent.
Learn how teams preserve context with Azure Monitor issues.

Stay connected

Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next.
Live webinar
A walkthrough of real Observability Agent scenarios, best practices, and what's available today, along with a look at what's coming next and live Q&A with the product team. Register for the Observability Agent webinar

We'd love your feedback

The Observability Agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience or reach us at azureobsagent@microsoft.com.

Export historical data from Log Analytics workspace with Export Job (preview)

YossiY — Tue, 07 Jul 2026 09:53:09 GMT

Log Analytics Export Job is now available in public preview. It gives you a straightforward way to export historical log data from your workspace to Azure Blob Storage, without writing custom scripts or disrupting live operations. You submit a job including a query, time range, and the service handles the rest asynchronously.

Historical data had no built-in exit path

Your Log Analytics workspace accumulates months, sometimes years, of telemetry. That data has real value beyond the workspace: training security models, satisfying compliance requirements, supporting forensic investigations with external tools, or migrating to a new analytics platform. The challenge has always been getting it out.

Log Analytics supports continuous data export for ongoing ingestion, but that doesn’t help with data that already exists. Teams that needed to export historical data had to build their own solutions: scripted query loops, Logic Apps, or Azure Functions calling the query API in batches and stitching results into storage. These approaches were slow, brittle, and hard to operationalize at scale.

Export Job closes that gap.

One job per table, across Analytics and Basic tiers

You target a specific table, define a KQL filter on table, set a time range, and the job exports that data, whether it sits in Analytics or Basic tier, writing the results directly to your storage account as Parquet files.

End-to-end flow of a Log Analytics Export Job

You can filter with KQL to scope the export to exactly the columns and records you need, reducing cost and downstream processing time.
Output is gzip-compressed Parquet, the standard columnar format for data lakes, Spark, Azure Data Explorer, and most ML frameworks, with no conversion step required.
Export data in hourly folders to your blob storage.
Billing is based on two existing meters: data scanned, using existing Log Analytics scan rates, and data volume exported as measured in your storage account.

Resilient execution

Large exports can be interrupted by network issues, transient storage errors, or downtime. Export Job includes a built-in retry mechanism to overcome these interruptions automatically. The service splits the job into hourly bins, each tracked and written independently to your storage container. Transient failures are retried without any action on your part. If a bin fails after retry exhaustion or job 7-days' timeout, you can retry it manually within 7 days of job completion, without restarting the entire job or re-exporting data that already completed successfully. Before a retry writes new data, any partial output from the failed bin is automatically cleaned up, so there is no risk of duplicates in your storage account.

Getting started

Log Analytics Export Job is available in public preview today. Configuration is programmatic through the Azure Monitor REST API, letting you create, check status, cancel, and retry jobs. Before your first job:

Enable the workspace Managed Identity in your Log Analytics workspace settings.
Assign the Storage Blob Data Contributor and Log Analytics Reader roles to the workspace Managed Identity on your destination storage account.
Ensure the destination storage account is in the same Azure region as the workspace (cross-region support is on the roadmap).
Enable the Jobs category in your workspace’s diagnostic settings, to route job execution records to the LAJobLogs table. This gives you creation time, job parameters, and bin-level status for every job you run.
Assess that export volume and run duration using suggested query in export job article.
Consider export job bounderies:
1. The maximum time range per job is one year
2. The maximum run duration per job is seven days. When reached due to volume, you can retry to continue export from where it stopped.
3. Five concurrent jobs are supported

Once prerequisites are in place, create a job with a single API call:

POST https://api.loganalytics.azure.com/v2/subscriptions/{subscriptionId}/resourcegroups/{resourcegroup}/providers/Microsoft.OperationalInsights/workspaces/{workspace}/jobs/export?api-version=2023-09-01-preview
Authorization: {credential}

content-type: application/json

{

"startTime": "2025-01-01T00:00:00Z",

"endTime": "2025-06-30T23:59:59Z",

"query": "{query}",

"destinationStorageAccounts": [

"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{storageAccountName}"

"containerName": "{containerName}",

"outputDataFormat": "Parquet",

"dateTimeFormat": "yyyy-MM-ddTHH"

}

Copy the job ID returned in the response, which can be used to poll status, cancel, or retry individual failed bins.

Learn more: https://aka.ms/LogsExportJob

Share your feedback as we continue to improve the feature.

Find anomalies in Prometheus and OpenTelemetry metrics with Dynamic Thresholds (Preview)

yairgil — Thu, 02 Jul 2026 13:10:54 GMT

Dynamic thresholds are extended to query-based metric alerts in Azure Monitor, allowing to detect and alert on anomalies in Azure Monitor managed Prometheus metrics and OpenTelemetry metrics stored in an Azure Monitor Workspace.

This follows the introduction of Dynamic Thresholds for Log search alerts — Azure Monitor now offers consistent Dynamic Thresholds support across logs and metrics — platform metrics, log search queries, and now query-based metric alerts. A consistent anomaly-detection approach, wherever your signals live.

Dynamic thresholds are not a single static formula. They apply a range of machine-learning models and algorithms to historical query results, learn each series’ normal rhythm — including hourly, daily, and weekly seasonality — and automatically fit the most appropriate baseline separately to every time series. This way, a single alert rule can monitor many resources or dimensions while each one gets its own independent, self-refining baseline.

Why Dynamic Thresholds Matter

Simpler configuration: Reduce the need to define, maintain, and continuously tune static thresholds inside PromQL alert logic.

Adaptive monitoring: Let alert thresholds adjust to changing workload behavior, recurring traffic peaks, and seasonal usage patterns.

At-scale intelligence: Monitor multiple time series with a single alert rule, while Azure Monitor learns an independent baseline for each resource or dimension combination.

Example 1 — Spot CPU anomalies in AKS workloads

Scenario: Monitor container CPU utilization across pods or deployments in AKS with a query-based metric alert built on Prometheus metrics.

Example query:

sum by (microsoft_resource_id, namespace, deployment, container) (rate(container_cpu_usage_seconds_total[5m])) / sum by (microsoft_resource_id, namespace, deployment, container) (container_spec_cpu_quota / container_spec_cpu_period)

Why dynamic thresholds help: CPU usage of a Kubernetes workload changes with workload mix, deployment timing, scaling activity, and traffic patterns. Static thresholds can be difficult to tune across namespaces, deployments, and containers. Dynamic thresholds learn a separate baseline for each monitored time series — in this example, for every pod, deployment, and container combination — so genuine CPU spikes stand out while expected variation from autoscaling and traffic mix stays quiet.

Example 2 — Catch application latency regressions sooner

Scenario: Detect abnormal latency patterns in an application by alerting on custom OpenTelemetry metrics stored in an Azure Monitor Workspace.

Example query:

histogram_quantile(0.95, sum by (le, service_name, http_route, http_method) (rate(http_server_duration_seconds_bucket[5m])))

Why dynamic thresholds help: Application latency naturally changes with traffic, user behavior, and release cadence. Fixed thresholds can be noisy during peak periods and too loose during quiet ones. Dynamic thresholds learn a separate baseline for each time series — here, for every service, route, and method — so real p95 latency regressions surface even as traffic and release cadence shift throughout the day.

Best practices for better results

To get the best results from dynamic thresholds for PromQL-based alerts, design your query so Azure Monitor can learn a clear, stable signal over time:

Keep the expression numeric. Dynamic thresholds work best when the query returns a continuous numeric signal rather than a Boolean true/false result. For example, use an expression that calculates CPU usage, not a Boolean comparison like CPU > 0.8.
Use meaningful dimensions. Split by dimensions such as namespace, deployment, service, or route when you want separate baselines for different workloads or endpoints.
Prefer stable entities. Use longer-lived dimensions or aggregate across short-lived entities so the model has enough consistent history to learn from. In Kubernetes, for example, deployment is usually a better baseline dimension than individual pod ID.
Choose the right threshold behavior. Decide whether the alert should trigger on values above the learned upper bound, below the lower bound, or both.
Start with medium sensitivity. Use Medium as a balanced default, then tune up or down based on noise and missed anomalies.
Allow enough historical data. Dynamic thresholds improve as more history is collected. Initial seasonal patterns use recent history, and weekly seasonality becomes more effective after several weeks of data.

Get started

Ready to try it? Create a query-based metric alert with dynamic thresholds on your metrics in Azure Monitor Workspace. You can create such rules in the Azure portal, where the built-in preview chart shows when your dynamic threshold alert would have fired based on historical baseline analysis. Use the preview chart to tune both the PromQL query and the dynamic threshold sensitivity before enabling the rule. You can also create query-based metric alert rules using programmatic interfaces or resource templates.

Figure 1. Dynamic thresholds preview chart showing the learned baseline and the points where an alert would have fired.

Dynamic thresholds cut alert noise where it starts — at detection. The alerts that do fire connect into Azure Monitor’s broader AIOps experience, where the Azure Copilot Observability Agent can help correlate signals into investigated issues with explainable reasoning — with humans in control.

Next steps

Stay connected

Follow the Azure Observability Blog for more updates on Azure Monitor, Prometheus-based monitoring, alerting, and troubleshooting experiences. We’ll continue sharing product updates, practical guidance, and examples to help you improve observability across your Azure environments.

Feedback

We’d love to hear how dynamic thresholds for query-based metric alerts work for your scenarios. Share your feedback through your Microsoft account team, Azure support channels, or the feedback options in the Azure portal so we can continue improving the experience.

Azure Copilot Observability Agent is generally available, with autonomous operations in preview

EfratNauerman — Tue, 23 Jun 2026 19:38:23 GMT

Complex cloud environments have outpaced manual operations. Agentic cloud operations connect people, tools, and data to streamline investigation workflows and move teams from scattered signals to evidence-backed next steps. With unified observability, teams can investigate Azure-monitored applications, Azure Kubernetes Service (AKS) environments, VMs, Foundry telemetry, infrastructure, and platform signals with greater context and control.

Powered by Azure Monitor, the Azure Copilot Observability Agent is now generally available. It helps engineering, SRE, DevOps, and operations teams move from telemetry and alert noise to investigated issues, explainable reasoning, and recommended next steps that can reduce Time-To-Mitigate (TTM).

Autonomous operations are also available in public preview. They help prepare context and reduce triage work while people remain responsible for mitigation decisions and any changes to the environment.

From alert noise to investigated issues

The Observability Agent helps teams reduce the effort required to understand operational problems. Instead of starting every investigation from a dashboard, query editor, or alert payload, teams can work with an AI companion that reasons across telemetry, Azure resource context, discovered topology, and custom instructions to identify what changed, what is correlated, and what evidence supports the conclusion.

Teams can start with natural-language exploration and continue into deeper investigations when an issue requires more evidence. That light-to-deep workflow helps responders move from broad questions to a structured investigation without losing the reasoning trail.

Here's what this looks like in practice: after a deployment, several alerts might fire across an app, database dependency, and compute resource. The Observability Agent can group those signals around the affected service, identify when the regression started, compare related dependencies and infrastructure metrics, and capture the findings in an Azure Monitor issue. The responder can then validate the evidence, add team context, route work to the right owner, and decide whether a rollback, configuration change, or code fix is appropriate.

Explainable investigations across Azure-monitored signals

Operations teams need more than a chatbot that answers questions. The Observability Agent follows an investigation workflow: it frames hypotheses, gathers evidence, compares signals by time, scope, and type, rules out weak explanations, and shows the reasoning path behind its findings.

The Observability Agent can help teams:

Investigate incidents and alerts across Azure-monitored applications, Azure Kubernetes Service (AKS) environments, VMs, Foundry telemetry, infrastructure, and platform signals
Correlate related signals to reduce noise and surface higher-signal issues with context
Explore telemetry using natural language while preserving transparency into the supporting data
Compare signals by time, scope, and type to separate likely causes from coincidental changes
Provide a reasoning trail that shows what the agent found, what it ruled out, and why
Recommend next steps that engineers can review before deciding how to act

This same investigation model applies to specialized skills and issue types, including customer's application, Azure Kubernetes Service (AKS), Foundry, VMs, and GenAI issues. When the relevant telemetry is available, the Observability Agent can correlate logs, metrics, traces, alerts, dependencies, resource graph, resource health, activity logs, Foundry telemetry, and changes. This helps teams investigate customer-visible issues with evidence, including latency, token spikes, tool-call failures, agent errors, hallucinations, deployments, API failures, performance regressions, infrastructure dependencies, and platform incidents.

This explainability is central to the product. In production operations, trust is earned through evidence. The Observability agent is built to support human judgment, not bypass it.

Azure expertise, with context from your environment

Context matters in every investigation. The same symptom can mean different things depending on application architecture, recent deployments, dependencies, historical incidents, and team practices.

The Observability Agent brings Microsoft and Azure operational knowledge into the investigation experience. It can use discovered topology, Azure resource context, logs, metrics, traces, and custom instructions to ground investigations in signals that are more relevant to your environment.

Native to Azure Monitor, with humans in control

Because the Observability Agent is built into Azure Monitor, teams can use it close to the telemetry, alerts, and workflows they already rely on. Investigations can also be captured as Azure Monitor issues, creating a shared case file for humans and agents to collaborate on evidence, reasoning, and next steps.

The Observability Agent is designed for governed AI operations inside Azure Monitor. Interactive chat and investigations use the signed-in user's identity and Azure role-based access control (RBAC). Prompts and responses are not used to train foundation models, and the agent doesn't restart resources, change configuration, or resolve issues on its own.

Autonomous operations in public preview

Alongside general availability, autonomous operations for the Observability Agent are available in public preview. When enabled, the agent can analyze alerts in the background, correlate related alerts when they likely represent the same incident, create Azure Monitor issues automatically, and run deep investigations on agent-created issues.

This automatic triage helps reduce alert noise by turning streams of individual alerts into higher-signal issues with context, findings, and recommended next steps. Teams can review the issue, continue the investigation, and decide what action to take.

Autonomous operations are designed to prepare context and reduce triage work, not to remove human control. Engineers remain responsible for decisions, approvals, and any changes to the environment.

Next steps

Check out our latest announcements and related blogs: Azure Blog and OMB Blog.
Learn how to use the Observability Agent in Azure Copilot Observability Agent.
Explore how investigations work in Deep investigations in the Azure Copilot Observability Agent.
Learn more on how to Chat with your observability data
Learn how teams preserve context in Azure Monitor issues.
Review preview details in Autonomous operations in the Azure Copilot Observability Agent.

Stay connected

Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next.
Live webinar - a walkthrough of real Observability Agent scenarios, best practices, and what's available today - along with a look at what's coming next, and live Q&A with the product team. Register for the Observability Agent webinar.

We'd love your feedback

The Observability agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience, or reach us at enauerman@microsoft.com.

Accelerating AKS troubleshooting with the Azure Copilot Observability Agent

yairgil — Wed, 17 Jun 2026 16:27:27 GMT

AKS incidents rarely stay within one Kubernetes object, signal, or tool. A latency spike might first appear in application telemetry, but the root cause may sit elsewhere: pod restarts, node pressure, scheduling failures, or a recent configuration change. The Azure Copilot Observability Agent in Azure Monitor helps connect these signals into an explainable investigation, so teams can move from symptoms to evidence-backed next steps.

Why AKS troubleshooting is complex

Troubleshooting Azure Kubernetes Service (AKS) is complex because failures can originate in workloads, platform components, infrastructure, or the application code running on the cluster. For example, pods stuck in Pending may indicate capacity or scheduling issues, while application latency may be caused by throttling, failed probes, pod restarts, or node pressure below the app.

During an incident, simply having more telemetry is not enough. Teams need a way to test likely causes, rule out unrelated signals, and keep the investigation tied to the affected workload and time window.

From signal to root cause: the investigation flow

The Observability Agent follows a consistent investigation pipeline:

Scope the problem by identifying the most likely infrastructure resources involved, plus connected dependencies.
Collect data across metrics, logs, traces, change history, and related signals.
Detect anomalies using learned baselines (for metrics) and log analysis.
Correlate across resources spanning infrastructure and application layers.
Run deep diagnostics by invoking resource-specific tools when needed to pinpoint root cause.
Summarize findings in a structured format: what happened, why it happened, and what to do next.

AKS investigation data sources

The agent works with telemetry already available in your Azure Monitor environment. Investigation depth improves as more relevant signals are enabled, including Container insights logs, Kubernetes events and state, Azure managed service for Prometheus, container and pod logs, Application Insights telemetry for AKS-hosted workloads, Azure Activity Log changes, control plane logs routed through diagnostic settings, and resource metadata for the cluster, node pools, workloads, and related Azure resources.

Figure 1. AKS investigation data sources

You don’t need to enable every telemetry source to get started. The Observability Agent uses the data already available in Azure Monitor, and its findings become more complete as more AKS and application signals are collected.

Example 1: AKS infrastructure — explaining why new pods never start

Consider a workload rollout on AKS where replacement pods remain stuck in Pending state. What looks like a failed release may stem from the workload definition, cluster state, or underlying infrastructure.

Investigation walkthrough

Symptom: rollout is blocked
Replacement pods remain in Pending during rollout, and Kubernetes events show repeated scheduling failures. This indicates that the rollout is blocked before new pods can start.
Workload evidence: scheduling, not startup
Pod state identifies the affected workload, while Kubernetes events show repeated placement failures. The issue is therefore tied to scheduling rather than application startup or container crash behavior.
Cluster evidence: capacity pressure
When enabled, Prometheus node metrics show CPU and memory utilization near capacity. Cluster-level trends show resource pressure increasing at the same time as pending pods and scheduling failures.
Likely cause: insufficient schedulable capacity
The scheduler cannot place new pods because the relevant node pool does not have enough available capacity. The failed rollout is best explained by capacity pressure in the target node pool rather than an application crash or image startup failure.
Recommended action
Scale out the affected node pool or adjust workload resource requests, then retry the rollout once schedulable capacity is restored.

Figure 2. AKS investigation flow

The Observability Agent connects pod state, scheduling events, and node pressure to explain why the rollout is blocked and which capacity action to consider next.

Example 2: Joint app-AKS investigation — tracing application latency to pod restarts

Now consider a customer-facing application where users see increased latency and intermittent HTTP 5xx errors after deployment. The first symptom appears in application telemetry, but the unhealthy requests are served by pods that are repeatedly restarting in AKS.

Investigation walkthrough

Symptom: customer-facing service degradation
After deployment, application telemetry shows increased latency and HTTP 5xx errors. The first visible impact appears at the application layer.
AKS evidence: unstable pods
Affected pods enter CrashLoopBackOff, restart counts increase, and Kubernetes events show back-off restarts, probe failures, or image or command errors. Container logs point to startup exceptions, missing configuration, or crash details.
Resource evidence: workload-specific pressure
Container memory usage approaches configured limits before restarts, while node metrics show no broad node pressure. This suggests the issue is workload-specific rather than cluster-wide capacity related.
Change evidence: deployment correlation
Deployment history shows a new image or configuration change shortly before restarts began, with no matching platform health event. The timing points to the latest deployment or configuration change.
Recommended action
Review the latest image or configuration change, inspect container logs, adjust memory limits, or roll back if needed. Focus remediation on the workload change rather than node pool scaling.

This pattern shows how an application symptom can map back to AKS workload behavior. Application telemetry establishes the user impact, while Kubernetes events, container logs, and resource metrics help explain why the affected pods keep failing.

Operational impact

For site reliability engineers, platform teams, and IT professionals, the Observability Agent reduces the time spent moving between application and AKS telemetry. It brings relevant signals into one investigation, surfaces supporting evidence, and applies Azure Monitor and AKS context so your team can review the findings, validate the recommended path, and decide which production changes to make.

Figure 3. AKS investigation results

Using the Observability Agent

You can start using the Observability Agent from the Azure portal in two common AKS troubleshooting flows:

Investigation mode: Start an investigation from an Azure Monitor alert on an AKS resource or from an Application Insights alert for an AKS-hosted workload. The agent uses the alert context to scope the incident, correlate application and cluster telemetry, and summarize the likely cause with recommended next steps.
Chat-based exploration: Open the Monitor experience in AKS and select the Observability Agent button to chat with your telemetry. Use natural language to ask follow-up questions, explore logs and metrics, detect and inspect anomalies, and narrow down likely causes.

Figure 4. Starting Observability Agent from AKS Monitor experience

Next steps

Stay connected

Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next.
Live webinar — A walkthrough of real Observability Agent scenarios, best practices, and what's available today, along with a look at what's coming next and live Q&A with the product team. Register for the Observability Agent webinar.

We'd love your feedback

The Observability Agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience, or reach us at: azureobsagent@microsoft.com

Anomaly detection made easy with Dynamic thresholds for Log search alerts

Efrat_Ben_Porat — Tue, 16 Jun 2026 17:33:52 GMT

We’re excited to announce the General Availability of dynamic thresholds for log search alerts in Azure Monitor. Dynamic thresholds make anomaly detection easier by using machine learning to learn normal behavior from your historical log query results, automatically account for patterns such as hourly, daily, and weekly seasonality, and adapt as your environment changes. Instead of manually choosing static limits that can quickly become outdated, you can let Azure Monitor automatically determine the right threshold for each alert rule. Dynamic thresholds for Log search alerts are available at no extra charge - you pay the standard log search alert rule rate.

Dynamic threshold preview chart displayed when setting up a new alert rule.

Why it matters

Simplified configuration: No need to fine-tune thresholds manually.

Adaptive monitoring: Alerts automatically adapt to changing usage patterns and trends.

At-scale intelligence: For multi-dimensional monitoring, thresholds are calculated per dimension combination.

Example use cases

AKS Pod restart spike anomaly detection

Scenario: Monitor Kubernetes Pod logs for spikes in pod restarts across clusters.
Why dynamic thresholds help:

AKS workloads often scale dynamically; static thresholds can’t account for autoscaling patterns.
Dynamic thresholds adapt to normal fluctuations in node/pod counts and alert only on true anomalies.

Example query:

KubePodInventory | summarize restartCount = sum(PodRestartCount) by bin(TimeGenerated, 10m), ClusterName, Namespace, Name

Dynamic threshold settings:
- Namespace (for workload-level baselines).
- Name (for per-pod granularity if needed).
- Measure: restartCount (the aggregated column from the query).
- Split by dimensions (optional):
  - Namespace (for workload-level baselines).
  - Name (for per-pod granularity if needed).

Resource Inventory Drift Detection (Azure Resource Graph)

Scenario: Detect sudden spikes in resource creation or deletion across subscriptions or management groups utilizing Log search alerts integration with Azure Resource Graph that may indicate runaway deployments.
Why dynamic thresholds help:

Large organizations often have thousands of resources with varying deployment patterns.
Static thresholds can’t account for seasonal changes (e.g., monthly deployments, scaling events).
Dynamic thresholds adapt per subscription or resource type, reducing false positives.

Example query:

arg("").Resources | summarize resourceCount = count() by type, subscriptionId

Dynamic threshold settings:
- type (for specific resource type changes).
- subscriptionId (for per-subscription granularity).
- Measure: resourceCount (the aggregated column from the query).
- Split by dimensions (optional):
  - type (for specific resource type changes).
  - subscriptionId (for per-subscription granularity).

Getting Started

Learn more about Log search alerts with dynamic thresholds and how to set up alert rules in Azure Monitor.

General Availability: Simple log alerts in Azure Monitor

Efrat_Ben_Porat — Tue, 16 Jun 2026 17:33:31 GMT

We are excited to announce the General Availability of Simple log alerts in Azure Monitor! This feature is designed to provide a simplified and more intuitive experience for monitoring and alerting, enhancing your ability to detect and respond to problems in near real-time.

Simple log alerts are a type of Log search alerts in Azure Monitor, designed to provide a simpler alternative to traditional Log search alerts. Unlike Log search alerts that aggregate rows over a defined period, Simple Log Alerts evaluate each row individually.

Simple Log Alerts are supported using Basic logs as well. Before, choosing Basic logs for cost optimization - for example, configuring the traces table in Application Insights with Basic logs plan - meant giving up the ability to alert on that data. Simple log alerts close that gap, so you can keep the cost savings and alert on telemetry stored in Basic Logs.

🌐 When to use

Simple Log Alerts are ideal for monitoring applications or network traffic where unaggregated, real-time detection and quick incident response are critical.

Example scenarios:

Failed automation jobs - get notified the moment a backup job, scheduled task, or any automated process fails, rather than waiting for an aggregation window.
Windows events affecting storage or security - alert on individual event log entries that signal disk failures, security breaches, or service disruptions.

🔁 Flexible Trigger Recurrence

By default, Simple log alerts fire on every matching row, but you can tune this to reduce noise. Choose to alert only when the condition is met at least once, twice, three times, or a custom number of times within a minute - giving you control over sensitivity without sacrificing the low-latency.

💰 Pricing Information

Simple log alert rules evaluate your data every minute, so billing is the same as 1-minute frequency alert rules. For detailed pricing information, refer to Pricing - Azure Monitor | Microsoft Azure. You will see these rules in your billing statement tagged with kind:simplelogalert.

📚 Documentation and Links

Inside the Observability Agent: How Deep Investigations and Reasoning Work

yalavi — Mon, 15 Jun 2026 15:57:54 GMT

Incidents are rarely explained by one chart. A spike in request failures might be caused by a dependency timeout. A dependency timeout might come from a connection pool issue. And that connection pool issue might trace back to a deployment or a resource health event that started minutes earlier.

From the outside, Azure Copilot observability agent presents this as a focused troubleshooting experience that correlates signals such as metrics, logs, alerts (and alert context), tracing data, and resource and service health.

Under the hood, the implementation is more than a chat prompt with a few queries attached. It is a dedicated investigation runtime designed to build hypotheses, gather data across layers, and return an explanation that is meant to be understandable and data-backed.

Deep Investigation Is a Dedicated Investigation Run

A deep investigation runs as its own scoped investigation session anchored to a clear goal, a fixed incident time, a set of target resources, and the user’s Azure context, so it can query the right telemetry and platform data without losing the problem frame. Beyond goal, time, and scope, the agent also folds in the prior conversation, such as symptoms shared, suspected causes, and alerts that came up, so the run starts already aware of context.

That isolation matters for two reasons:

Stable problem frame: Anchored to a specific incident window and resource scope, which keeps analysis from drifting into generic commentary.
Room to do real work: Can take multiple steps to collect data, explore alternatives, and stream progress while the focused run continues.

In other words, the product feels conversational, but the runtime behaves like an agentic investigation engine.

Learning the Resource Context

Today the agent builds and persists a resource context per resource, learning each resource’s configuration, normal behavior, and relationships and reusing that context across investigations, so it does not start from scratch every time.

The Core Method: Think, Act, Review

Deep investigation is organized around a think-act-review loop.

The agent starts by generating multiple plausible leads instead of jumping to a single answer. It is explicitly instructed to consider several hypotheses, investigate each one, and rule out alternatives with evidence. That matters in observability, because the first suspicious signal is often a symptom rather than the cause.

The loop works like this:

Think: Form hypotheses about what could explain the incident.
Act: Use the available tools to gather telemetry and platform data.
Review: Critique progress, identify gaps, and decide whether to continue digging.
Repeat: Refine the next set of questions until the root cause is well supported, or uncertainty is clearly stated.

This is a meaningful design choice. Many AI troubleshooting experiences fail because they are optimized for fluency instead of investigation discipline. Here, the agent is pushed toward structured reasoning: identify leads, test them, reject weak ones, and only then conclude.

Success is defined as more than a plausible answer. The investigation aims to produce:

Incident framing: onset time, impact window, and blast radius.
Affected components: services, operations/endpoints, and dependencies.
Alternative hypotheses: what was considered and why it was ruled out.
Mitigation steps: recommended actions and how to validate recovery.
Support data: metric anomalies, resource health, configuration changes, and alerts as standard inputs.

Signals from Each Layer

The most important thing deep investigation does is correlate signals across layers that usually live in different tools.

Application layer

At the application layer, the agent can query Application Insights for requests, exceptions, dependencies, traces, custom events, and performance counters. That gives it the immediate symptom picture:

What failed?
Which operations or endpoints were affected?
Which dependencies slowed down or timed out?
How many users were impacted?
Did the issue hit one role or several?

This is where the incident becomes concrete. Instead of saying “latency increased,” the system can identify the deployment version, operation, the failing dependency, the time window, and the scale of impact.

Infrastructure & Platform layer

At the infrastructure & platform layer, the agent looks at Azure Monitor metrics, Log Analytics, resource changes, alerts, Azure Resource Graph metadata, and Azure Resource Health.

This is where the investigation shifts from “what broke” to “what changed” and “what else moved with it.”

A quick ML-based anomaly screen for resource metrics.
Deeper metric inspection using discovered metric definitions, metric values, and dimension values.
Resource change analysis to identify configuration, deployment, scale, or network changes.
Alerts as supporting data.
Azure Resource Graph queries for topology and configuration context.
Azure Resource Health and Azure Service Health to detect resource-level degradations and broader service incidents.

Together, these checks help answer not just “what moved,” but whether that movement plausibly explains the symptom.

That is an important characteristic of a mature investigation system: it does not confuse “not flagged” with “healthy.”

Correlating Signals Across Layers

The differentiator is not any single tool. It is the way the system combines signals from different layers into a single causal story.

Deep investigation is designed to align data along three axes:

Time: Did the signal move at the same time as the incident?
Scope: Did it affect the same resource, dependency, operation, or region?
Type: Does the signal match the observed symptoms?

That may sound simple, but it is what turns observability data into an explanation.

Application telemetry: Dependency timeout pattern.
Infrastructure logs: Connection pool exhaustion.
Platform data: Deployment or configuration change shortly before failures began.
Resource Health: No degradation reported.

Individually, each signal is suggestive. Together, they can support a much stronger conclusion: the problem likely came from an application or configuration change, not from a broader Azure platform outage.

The same logic works in the other direction. If application failures start at the same time a resource becomes degraded in Resource Health, metrics show a sharp availability drop, and alerts fire on the same target, the agent has a much stronger basis for identifying a platform-level contributing factor.

This is also why alerts are treated as supporting data instead of root cause by themselves: they indicate a threshold breach, but not why it happened. Deep investigation uses alerts as one input into a larger causal picture.

Explainability Is Built into the Runtime

The observability agent is built to explain its reasoning.

Layer 1: Live progress. As the deep investigation runs, the runtime captures reasoning updates and tool outputs and streams lightweight progress steps back into the parent conversation, where users can follow them in the Chain-of-Thought (CoT) view as the investigation unfolds.

Layer 2: Final narrative. The investigation produces a customer-facing report that separates What happened, Analysis, and What can be done next, which keeps symptoms, causes, and mitigations distinct.

Layer 3: Dashboard contract. The narrative is transformed into a structured dashboard object (summary, impact time, severity, findings, supporting data). Data cards are expected to be telemetry-backed artifacts that explain timeline, blast radius, dependency health, error mix, saturation signals, change correlation, and mitigation validation.

This structure matters because it keeps symptoms, causes, and mitigations distinct, making the outcome easier to read, easier to validate, and easier to act on.

In other words, explainability is not just “show the answer.” It is “show the data structure behind the answer.”

A Concrete Example

Imagine an incident where an API’s latency jumps from hundreds of milliseconds to several seconds.

Deep investigation reconstructs the chain of events:

Application layer: Latency spike concentrated in one operation, coinciding with dependency timeouts to a downstream service.
Infrastructure layer: Downstream logs show connection pool exhaustion messages or restart behavior.
Platform layer: A configuration or deployment change appears a few minutes before the incident; detailed metrics and Resource Health checks help confirm timing and rule out broader platform degradation.

Now the system can do something more useful than saying “latency is high.” It can tell a connected story:

The incident began at a specific time.
It affected a specific set of operations and users.
The API slowdown was downstream-driven.
The downstream failures started after a configuration change reduced available capacity.
No platform health event explains the timing, which makes an application or config-level cause more likely.
The recommended next step is to revert the change, validate recovery, and monitor the affected metrics.

That is the difference between telemetry retrieval and root cause analysis.

What Makes This Approach Valuable

The interesting part of deep investigation is not that it can query logs, metrics, or alerts. Plenty of systems can do that.

The interesting part is that it treats investigation as a control loop with data gates:

Form multiple leads.
Check across layers.
Correlate by time and scope.
Rule out healthy resources.
Treat weak data as weak.
Turn the result into an explainable output.

That combination is what makes the experience useful for SREs and developers under pressure. During an incident, teams do not just need more telemetry. They need a system that can reduce ambiguity without hiding data.

That is what the deep investigation runtime in the observability agent is trying to deliver.

Learn More

Modern VM monitoring, powered by OpenTelemetry

viviandiec — Thu, 11 Jun 2026 18:57:11 GMT

At Build 2026, we're announcing the general availability of OpenTelemetry (OTel) Guest OS metrics for Azure VMs and Arc-enabled Servers. OTel provides a standards-based foundation for VM monitoring with consistent metrics across Windows and Linux, richer Guest OS and per-process visibility, and streamlined integration with open-source and cloud-native observability tools.

Alongside the GA, we're introducing an enhanced VM monitoring experience, recommended alerts, and out-of-the-box Grafana dashboards, all powered by OTel Guest OS metrics. We're also sharing upcoming VM troubleshooting capabilities in the Azure Copilot observability agent enriched by OTel Guest OS metrics.

What are OpenTelemetry Guest OS metrics

OTel Guest OS metrics are collected from inside a VM. Today's coverage includes a curated set of CPU, memory, disk I/O, networking, and per-process metrics including CPU utilization, memory usage, uptime, and thread count. The supported set is point-in-time and will continue to expand as the OTel Host Metrics Receiver evolves upstream.

This level of visibility helps customers diagnose operating system and application issues without manually signing into individual VMs.

Why they matter

1. Lower cost and faster queries

Default OTel Guest OS metrics are available at no additional cost. They are stored in Azure Monitor Workspace using metric-optimized storage and pricing, providing lower cost and faster query performance compared to LA-based metrics.

2. Per-process visibility for deeper troubleshooting

Customers can optionally enable per-process metrics for deeper visibility into VM resource consumption. This helps identify noisy processes, memory leaks, runaway jobs, or resource-intensive applications without manually signing into the VM.

3. Consistent metrics across Windows and Linux

Use the same metric names, dashboards, and alerts across operating systems without maintaining separate monitoring configurations.

4. Native PromQL support

Use PromQL with the scale and managed experience of Azure Monitor Workspace.

5. OpenTelemetry-based standardization

Use the same metrics across Azure Monitor, existing OTel pipelines, or other compatible observability backends.

Log Analytics (LA)‑based metrics vs OTel‑based metrics

Customers running workloads on Azure VMs and Arc-enabled Servers have long relied on Log Analytics (LA)-based metrics for fleet visibility. That experience continues to be generally available and trusted by thousands of customers.

We recommend evaluating your requirements to determine which approach best suits your needs. LA-based metrics remain the foundation for customers who need advanced analytics and correlation, while OTel-based metrics open new possibilities for modern VM observability. Learn more.

New Capabilities Powered by OpenTelemetry

VM monitoring experience powered by OpenTelemetry (GA)

We're excited to announce the general availability of the enhanced monitoring experience for Azure VMs and Arc servers. This experience brings comprehensive monitoring capabilities in a single, streamlined view, helping you more efficiently observe, diagnose, and optimize your virtual machines.

The new experience offers two levels of insight within one unified interface:

Basic view (Host OS-based): Available for all Azure VMs with no configuration required. This view surfaces key host-level metrics including CPU, disk, and network performance for quick health checks.
Detailed view (Guest OS-based): Requires simple onboarding. Azure Monitor continues to support the GA detailed view powered by Log Analytics-based metrics. Customers can now choose to power the experience using OTel Guest OS metrics, which enable recommended alerts and provide expanded visibility into Guest OS and process-level resource consumption, including CPU, memory, disk I/O, and networking.

Dashboards with Grafana for VMs

For deeper analysis and customization, customers can leverage Azure Monitor dashboards with Grafana powered by OTel Guest OS metrics and PromQL at no additional cost.

Built-in dashboards provide out-of-the-box visualizations for at-scale monitoring, host-level monitoring, Guest OS monitoring, and per-process monitoring, while still allowing teams to:

Customize panels and dashboards
Run ad hoc investigations
Import dashboards from the Grafana community
Share dashboards using Azure RBAC and ARM/Bicep deployment support

Together, the enhanced VM monitoring experience and Grafana dashboards provide both streamlined day-to-day monitoring and flexible deep troubleshooting capabilities for modern VM environments.

Query metrics in the context of your resources (GA)

We’re also announcing the general availability of resource-scope querying for Azure Monitor Workspace (AMW) metrics, including OTel Guest OS metrics. With resource-scope query, you can query metrics directly from the context of a resource, resource group, or subscription, without needing to know which workspace stores the data. This simplifies troubleshooting, aligns with Azure-native workflows, and enforces least-privilege access using Azure RBAC.

This capability powers scenarios like querying OTel Guest OS metrics directly from the Virtual Machine resource in Azure Portal, or resources can be scoped as a dedicated data source in Grafana to query with PromQL, making it easier for application and infrastructure teams to monitor and troubleshoot in the context of their workloads.

Coming soon: Observability Agent Troubleshooting for VMs (Public Preview)

Today, the Observability Agent helps customers investigate issues by correlating applications, infrastructure signals, LA-based metrics, logs, alerts, health information, and recent changes into a guided investigation narrative.

Support for OTel Guest OS metrics is coming soon, extending investigations with richer Guest OS and per-process visibility. With OTel Guest OS metrics, the Observability Agent will be able to incorporate finer-grained operating system and process-level insights into its analysis, helping customers more quickly identify resource bottlenecks and understand their impact on application performance.

Instead of manually piecing signals together across multiple tools and timelines, customers will receive a guided investigation summary with likely causes and recommended next steps. Combined with the new VM monitoring experience and Grafana dashboards, customers will have both AI-assisted investigations and powerful manual troubleshooting tools built on the same OTel foundation.

Onboarding VMs at scale to OpenTelemetry

Onboarding Azure VMs and Arc-enabled Servers to OTel Guest OS metrics is now simpler and more cost-efficient than ever. For teams getting started at scale, the easiest path is through the Monitoring Coverage experience in the Azure portal, where you can review recommended resources and onboard VMs through a guided workflow.

Customers that prefer infrastructure-as-code can use ARM and Bicep templates to apply the same monitoring configuration programmatically.

Azure Advisor recommendations provide another seamless entry point for onboarding, proactively identifying VMs that are not fully monitored and guiding customers to enable OTel -based monitoring with a few clicks. This helps teams continuously improve coverage across their fleet without needing to manually audit resources.

Customers can now also reuse an existing Data Collection Rule (DCR) during onboarding, making it easier to standardize monitoring across large VM fleets. After onboarding, teams can centrally evolve their monitoring configuration by updating that DCR to collect additional metrics and logs, with changes applying across all associated VMs.

Get Started

Explore the new OpenTelemetry-powered experiences today:

Azure Monitor SLIs now Generally Available

Sokuma — Thu, 11 Jun 2026 17:18:29 GMT

Azure Monitor SLIs are now generally available

Service Level Indicators (SLIs) and Service Level Objectives (SLOs) in Azure Monitor are now generally available. Teams can now measure reliability based on customer experience, not just infrastructure signals.

SLI: A quantitative measure of how well an application or service is performing from the customer’s point of view.

SLO: A defined target for an SLI that represents how good or bad the SLI is over a given time-period. This is also referred to as a baseline in Azure Monitor.

Traditional monitoring shows what is happening across your systems, but not always what customers are experiencing. A service can be technically available and still feel unreliable because of latency, partial failures, or dependency issues. SLIs help close that gap by measuring reliability from the customer’s point of view.

With GA, Azure Monitor now brings SLI authoring, SLO tracking, error budgets, and burn rate–based alerting into one experience, helping teams focus on whether they are meeting the reliability their customers expect.

What Azure Monitor SLI helps you do

Azure Monitor SLI lets you measure availability and latency with either request-based or window-based evaluation methods.

In Azure Monitor, SLIs are defined at the Service Group level, which provides a logical representation of your application across multiple resources.

This gives teams a clearer view of application health, customer impact, and the signals that matter most.

SLIs continuously evaluate your service by using existing Azure Monitor metrics and store the resulting evaluations in your Azure Monitor Workspace.

Azure Monitor uses these SLI evaluations to power error budgets, burn rate visualization, and alerting. This helps teams spot reliability issues earlier and make better release and incident response decisions.

Get started

To get started, you’ll need:

A Service Group.
Application metrics flowing into an Azure Monitor Workspace, for example through Managed Prometheus or OpenTelemetry Collect and analyze OpenTelemetry data with Azure Monitor (Preview) - Azure Monitor | Microsoft Learn

Learn more here.

Summary Azure Monitor SLI helps teams measure customer experience, track reliability against clear targets, and respond sooner with error budgets and burn rate–based alerting.

Learn more in the product documentation and start defining SLIs for your services in Azure Monitor today.

Azure Monitor Metrics Export Generally Available

Sokuma — Thu, 11 Jun 2026 17:17:35 GMT

Today, we’re excited to announce the general availability of Azure Monitor Metrics Export using data collection rules (DCRs). A scalable, flexible way to continuously export platform metrics with dimensional fidelity, lower latency, and more control over what you send downstream.

Azure Monitor Metrics Export is configured through data collection rules and can route platform metrics to Azure Storage accounts, Azure Event Hubs, or Azure Log Analytics workspaces. Compared to diagnostic settings, DCR-based metrics export supports multidimensional metrics, metric-name filtering, and improved scalability for large environments.

Here are some of the key benefits of Azure Monitor Metrics Export:

Control what you export: You can export all supported metrics for a resource type or filter to specific metric names, helping reduce downstream volume and manage cost.
Preserve dimensional fidelity: The DCR-based metric export supports multidimensional metrics, making downstream analysis and correlation more meaningful.
Get faster export latency: End-to-end export latency is typically within about 3 minutes, improving time to insight for operational and analytics workflows.

With Azure Monitor Metrics Export, organizations can build more scalable observability pipelines, route metrics to the destinations that fit their architecture, and unlock richer analysis for operations, reporting, and integration scenarios.

What’s new in GA

With general availability, Azure Monitor Metrics Export offers a production-ready path to continuously stream supported platform metrics using data collection rules. Azure Monitor Metrics Export now covers 44 Azure regions, up from 12 regions previously. This expanded footprint helps more customers adopt DCR-based metrics export closer to where their resources run, improving rollout flexibility for global deployments.

Customers can export metrics to Azure Storage, Azure Event Hubs, or Azure Log Analytics, preserve metric dimensions, and filter by metric name to better control downstream volume and cost.

Learn more about metrics export using data collection rules.

We’re excited to make Azure Monitor Metrics Export generally available and look forward to seeing how customers use it to build more reliable, cost-conscious, and extensible monitoring solutions on Azure.

PUBLIC PREVIEW - Azure Monitor - Collect Azure Resource Platform Logs at Scale with DCRs

Mahesh_Sundaram — Wed, 10 Jun 2026 19:34:21 GMT

The Challenge of Platform Log Collection at Scale

For many platform teams, Azure resource platform log collection starts simple—but becomes difficult to manage at enterprise scale. Imagine a team responsible for more than 8,000 Azure resources across 14 subscriptions. Every new resource needs a diagnostic setting that must be:

Configured consistently
Audited continuously
Monitored for configuration drift
Re-enabled when settings are changed or disabled

That complexity grows quickly when the same process must scale across:

Multiple engineering teams
Dozens of Azure regions
Ongoing compliance and security audits

That is the reality of collecting Azure resource platform logs today with diagnostic settings.

What should be a centralized observability strategy can become a fragmented, time-consuming operational burden—driven by resource-by-resource configuration at massive scale.

Today, we’re changing that. We’re excited to announce the public preview of Data Collection Rules (DCRs) for Azure Resource Platform Logs—a centralized, secure, policy-driven way to collect platform telemetry across thousands of resources with a single rule. DCRs provide a standard model for defining data collection across Azure Monitor, including platform telemetry, metrics, logs, agents, and PaaS data sources.

If you’ve ever stared at a sprawl of Diagnostic Settings and wished for a simpler way to manage platform logs at scale, this preview is for you.

Configure once. Govern centrally. Scale infinitely

Here's how the customer pain shows up today—and how DCRs change the operating model:

Area	Pain with Diagnostic Settings today	How DCRs change it
Configuration	Configure each resource one at a time. Inconsistent coverage. Drift goes unnoticed.	Define one DCR. Associate it with thousands of resources via DCRA. Standards stay consistent.
Cost	No way to filter or shape log data before ingestion.	Filter and transform at ingestion time (Log Analytics destination). Send only what matters.
Security	No modern managed-identity-first pattern across all destinations. Security/Audit gaps.	Managed identity (system- or user-assigned) with least privilege RBAC for Storage and Event Hubs.
Operations	Slow onboarding. Manual repetitive work. Hard to audit and roll back.	Onboard at scale with ARM, Bicep, Terraform, or Azure Policy. Version-controlled like your IaC.
Governance	Fragmented workflows. Different patterns for different telemetry types.	One declarative model. Auditable associations. Aligned with your existing rollout mechanisms.

What can you do now

With DCRs for platform logs in public preview, your team can shift from per-resource toil to platform-scale operations:

Define once. Apply everywhere.

Author a single DCR for the log categories you care about, then associate it with thousands of resources using ARM, Bicep, Terraform, or Azure Policy. No more per-resource configuration.

Cut ingestion costs by filtering noise before it lands.

Shape and filter data at ingestion time when routing to Log Analytics. Only the signal you need reaches your workspace, everything else is dropped before billing.

ⓘ Transformations supported only for Log Analytics destination during preview.

Integrate new workloads using DCR policies

Create a new subscription, apply your usual DCR through policy, and observability will be set up automatically. There’s no need for manual diagnostic settings, or concerns about drift.

Route logs to the destination that fits each scenario.

Send analytics-grade logs to Log Analytics, archival to Storage, real-time streams to Event Hubs, and ecosystem flows to Partner Solutions—from the same management model.

Govern with modern identity & security by default.

Managed identity (system- or user-assigned) for Storage and Event Hubs—no shared keys. Azure Policy for enforcement. Full visibility into every association.

What's next

We're continuing to invest in the DCR-based platform logs experience. On the near-term roadmap:

More resource types expanding coverage across Azure services.
Additional destinations including more Partner Solutions in Marketplace.
Broader transformation support beyond Log Analytics.
Deeper IaC and Policy integration with built-in policy definitions and reference templates.

Set up once, manage centrally, and Scale infinitely.

That's the new model for Azure platform telemetry.

References

Platform log collection with data collection rules (Preview) - Azure Documentation
Platform logs supported resource types and regions (Preview) - Azure Documentation
Data collection rules in Azure Monitor — overview and concepts on Microsoft Learn
Diagnostic Settings in Azure Monitor — for context on the current per-resource model
Resource logs in Azure Monitor — definitions and categories

Connect Metrics to Traces with Exemplars in Azure Monitor

sunayanasingh — Mon, 08 Jun 2026 17:21:43 GMT

Following Microsoft’s recent GA announcement for OpenTelemetry (OTel) support, we are excited to announce support for Exemplars for customers instrumenting metrics with Prometheus or OpenTelemetry and traces using OpenTelemetry, enhancing Azure Monitor’s integrated observability experience for cloud-native applications.

Modern cloud-native applications generate enormous volumes of telemetry. Metrics help teams detect that something is wrong, but traces explain why. Exemplars bridge these two worlds by attaching trace references directly to metric data points, making it dramatically easier to pivot from a spike in latency or errors to the exact distributed trace responsible for the issue.

With Azure Monitor, customers can now ingest metrics with exemplars and visualize them in Azure Managed Grafana. This enables seamless correlation between metrics and traces, helping engineering teams troubleshoot issues faster and reduce mean time to resolution (MTTR).

Why Exemplars Matter

Traditional monitoring workflows often require users to manually correlate data across multiple systems. Exemplars simplify this workflow by embedding trace context directly into metric samples. For example, if a latency metric spikes at a specific timestamp, the exemplar associated with that data point can link directly to the distributed trace responsible for the outlier.

This provides several benefits:

Faster root cause analysis
Quicker transition from aggregate metrics to request-level details
Simplified debugging workflows for SRE and platform teams
Better observability experiences for microservices and distributed applications

Unified Observability with Azure Monitor

With Azure Monitor and Azure Managed Grafana, you can now:

Ingest OTLP or Prometheus metrics with exemplars into Azure Monitor Workspace
Store and analyze traces in Azure Monitor Application Insights
Visualize exemplar markers directly in Grafana charts
Navigate from a metric spike to the exact distributed trace associated with that data point

By combining these signals in a single observability platform, organizations can correlate infrastructure health, application behavior, and request traces without context switching between tooling.

How It Works

Once metrics, exemplars, and traces are ingested into Azure Monitor, Azure Managed Grafana can consume exemplar information from the configured Prometheus data source. When exemplars are enabled in Grafana dashboards, users will see markers associated with individual metric data points. Selecting an exemplar opens the associated trace in Azure Monitor, providing end-to-end diagnostic context.

Getting Started

Setup data ingestion:

Instrument your application to emit OpenTelemetry traces, OpenTelemetry or Prometheus metrics with exemplars, and enable ingestion of the same to Azure Monitor using OpenTelemetry Collector. Follow the instructions in Ingest OTLP Data into Azure Monitor with OTel Collector - Azure Monitor | Microsoft Learn. After this step, you will have the Log Analytics Workspace, Azure Monitor Workspace and Application Insights resources all set up to store the telemetry data.
Create an Azure Managed Grafana instance and connect it with the Azure Monitor Workspace by navigating to your Azure Monitor Workspace in the Azure portal and then clicking on “Linked Grafana workspaces”. To learn more, see Manage an Azure Monitor workspace - Azure Monitor | Microsoft Learn
Optionally, enable Azure Managed Prometheus on your AKS cluster or use remote-write and configure it to use the same Azure Monitor Workspace to centralize infrastructure and application metrics.

Enable Exemplars in Azure Managed Grafana:

After setting up the data ingestion, ensure that logs and traces are flowing into Log Analytics Workspace, and metrics are flowing into Azure Monitor Workspace.

Step 1: Enable Exemplars on Prometheus Data Source in Azure Managed Grafana

Navigate to Connections -> Data Sources in Azure Managed Grafana. Since you have connected Azure Managed Grafana to Azure Monitor Workspace, you will see the data source (Managed_Prometheus_<AMW-Name>) already configured. If the data source is not configured, follow the steps here to add your Azure Monitor Workspace as a data source.
Open the data source configuration.
Click Add Exemplars to enable exemplar support.

Step 2: Configure Trace Linking with Azure Monitor

In the exemplar configuration section, toggle Internal Link to On.
Select Azure Monitor as the data source.
In the Label Name, enter the name of the field in the labels object that should be used to get the trace id, eg. trace_id.
Click Save & Test.

This configuration enables direct navigation from exemplar markers in Grafana charts to the associated traces stored in Azure Monitor. Azure Managed Grafana also supports trace correlation from other solutions like Jaeger etc. To use your trace solution, use the appropriate links.

Step 3: Enable Exemplars in Dashboards

Navigate to a Grafana dashboard that uses your configured Prometheus data source.
Open the panel options for a metrics chart.
Toggle Exemplars to On.

Once enabled, exemplar markers will appear on supported metric visualizations. Clicking on it will show exemplar details along with an option to open the corresponding distributed trace in Azure Monitor.

To learn more, visit https://aka.ms/azmon-exemplars

Azure Monitor Health Model (Preview): What's New!

shijain13 — Fri, 05 Jun 2026 01:30:56 GMT

Azure Monitor Health Model is a modern observability capability that brings together telemetry, architecture, and business context of your workloads to generate health insights. It continuously aggregates signals across dependencies, producing a single, actionable health state which reduces alert noise and shifts team toward proactive operations with cohesive system view, clearer insights, and faster troubleshooting.

It addresses the common operation question 'Is my system/service/app healthy?' and 'Which underlying unit / component is impacting health?'

This refresh introduces flexible, workload-centric discovery (use application insights topology, Azure resource graph queries in addition to designing user and system flows) and smarter, faster health signal creation (use recommended signals, import existing alert rules, set dynamic thresholds).

Expanded Discovery Scope

As customers began modeling increasingly complex applications, we identified an opportunity to make discovery more flexible and intuitive. Teams naturally reason about their systems differently; some at the application level, others through infrastructure fleets or telemetry views. By expanding discovery options, we enable customers to build health models using the constructs they already use, making it easier to evolve health models as applications and architectures change.

Azure Monitor health models now support multiple discovery mechanisms:

Application Insights–based discovery for application-centric modelling
Azure Resource Graph (ARG) discovery for scalable, query-based resource selection
Continued support for Service Groups, now including nested Service Groups, as part of a broader set of discovery options

This evolution reflects a shift toward loosely coupled modelling, enabling customers to define health based on application architecture rather than infrastructure-centric grouping. Learn more about Discovery

Discovery Rule

Extended Health Signals

Our goal has been to help customers achieve meaningful health insights faster with less manual effort. By introducing platform defaults and surfacing recommended signals, we make it easier to align health models with proven Azure best practices from day one. At the same time, we preserve support for existing alerting strategies and investments, ensuring customers can extend rather than replace what they already have. These enhancements balance simplicity, guidance, and flexibility as environments scale.

Health Models now supports the following health signal capabilities:

Resource Health as a default signal, ensuring every model starts with a reliable platform-provided baseline
Recommended signals, automatically surfaced based on Azure service best practices and enhanced through Azure Monitor Baseline Alerts (AMBA) integration
Reuse of existing signals, enabled by importing Azure Monitor alert rules as health signals

Learn more about Signals

Signals - Recommended and Import from Alert Rules for Azure Resources

Introducing Health Aggregation Rules

Modern cloud applications are built for resiliency, redundancy, and tolerance of partial failure. Health Models are designed to reflect this reality by enabling customers to define what “healthy” means for their architecture. Flexible aggregation rules allow teams to model intent rather than individual component states, producing health views that better align with operational priorities and business impact.

Health Models now supports advanced aggregation logic, enabling the following types of scenarios:

Regional resiliency aggregation using numeric thresholds (e.g., 2 out of 4 regions must remain healthy)
Cluster and fleet health aggregation using percentage thresholds (e.g., 60% of VMs in a cluster must be healthy)

This enables modelling resiliency patterns, partial failures, and graceful degradation, providing a more accurate view of real business impact.

Import Custom Signal

Health is most valuable when it reflects both system behavior and application context. By enabling custom health inputs, customers can incorporate signals that are closest to their business logic and application state. Contextual annotations further enrich analysis, making health timelines easier to interpret and correlate with change events.

To support this, Health Models now provides for:

Custom health report ingestion for external application and system health signals
Data annotations to overlay deployments, incidents, and configuration changes on health state

Alert Experience

To proactively learn about health state change, health models allow creating Alert rules and associated action group trigger automated responses sich as notifying user. It is now possible to view all the alerts on a Health Model and start troubleshooting.

Alerts in Health Model Note: To avail these new capabilities, upgrade your health models to the new API version using built-in migration wizard in Azure portal for a simple, guided experience.

Alerts in Health Model

Note: To avail these new capabilities, upgrade your health models to the new API version using built-in migration wizard in Azure portal for a simple, guided experience.

Is 94% of your syslog just noise? Now you can filter it out before ingestion.

Ivan_Varnitski — Wed, 03 Jun 2026 17:19:40 GMT

At Microsoft Build 2026, we are announcing the public preview of multi-stage transformations for Azure Monitor Data Collection Rules (DCRs). Multi-stage transformations let you filter, aggregate, parse, and map your logs at the point of collection, before data is ingested into your workspace. Processing happens in a defined sequence of steps called processors, and you can chain them together to build precise data pipelines that reduce ingestion volume, improve data quality, and lower monitoring costs.

Processors in orange run on the agent (client-side). The KQL transform in green runs in the ingestion pipeline. Data volume shrinks at each stage.

What are multi-stage transformations?

A Data Collection Rule defines how Azure Monitor collects, transforms, and routes telemetry data. Until now, DCRs supported a single KQL transformation step on the ingestion side. Multi-stage transformations extend this model by introducing a processor pipeline: an ordered sequence of processing steps that run on the agent (client-side) or at the ingestion endpoint (ingestion-side), or both.

Each processor performs one operation: filtering records, parsing structured fields from raw text, renaming or dropping columns, aggregating metrics, or running a KQL expression. Processors execute in order, and the output of one becomes the input to the next. This composable design replaces what previously required complex, monolithic KQL queries or external pre-processing scripts.

Client-side processors run on the Azure Monitor Agent before data leaves the source machine. This means filtered and aggregated data never crosses the network, reducing both egress and ingestion costs. Ingestion-side processors run in the Log Analytics ingestion pipeline and support KQL-based transformations for more complex logic.

Key applications

The most immediate use case is cost reduction. When you can filter records on the agent before they leave the machine, you stop paying for data you never query. Syslog is the classic example: in many environments, informational and debug messages make up the vast majority of volume, and none of it gets looked at unless something breaks. A single filter processor can cut that stream by 90% or more.

Aggregation is equally powerful for high-frequency telemetry. Performance counters sampled every 15 seconds produce millions of records per hour across a large fleet, but most dashboards and alert rules only need 5-minute granularity. Rolling up those samples on the agent, before they cross the network, dramatically reduces ingestion without losing the operational signal your team actually relies on.

Beyond cost, multi-stage transformations improve the quality of the data that does reach your workspace. Parsing structured fields out of raw text (JSON payloads, XML event data, CEF security logs) at collection time means downstream queries are simpler and faster. And because each processor handles one step in a readable sequence, maintaining the pipeline is far easier than debugging a single monolithic KQL expression that tries to do everything at once.

To make this concrete, let’s walk through the two highest-impact patterns we see with preview customers: filtering noisy syslog data and aggregating performance counters.

Filter data before ingestion

The filter processor evaluates each record against conditions you define and drops anything that does not match. Because filtering runs on the agent, dropped records are never serialized, transmitted, or ingested. This makes it the highest-impact processor for cost reduction.

You configure filters using simple field-level conditions: specify a column name, an operator (equals, not equals, greater than, contains, etc.), and a value. Conditions can be combined with AND/OR logic for precise control.

Scenario: Keep only warning-and-above syslog messages

A typical syslog stream generates thousands of informational and debug messages for every actionable warning or error. With a filter processor, you set a severity threshold, and the agent drops everything below it before transmission.

In this example, the filter keeps records where SeverityNumber >= 4 (Warning). The 57,000 debug and informational records per hour are dropped on the machine. Only the 3,250 actionable records are transmitted and ingested, a 94% reduction in syslog volume.

Filters also support compound conditions. For example, you can keep auth-facility errors OR any critical message regardless of facility, all in a single processor step. This kind of targeted filtering is especially useful for security teams that need specific event categories without paying for the full syslog firehose.

Aggregate logs before ingestion

The aggregate processor rolls up high-frequency records into time-windowed summaries on the agent. This is especially valuable for performance counters, heartbeat signals, and any telemetry where per-second granularity is not needed for operational decisions.

You configure the processor with a time window (for example, 5 minutes), the aggregation operators to apply (average, sum, min, max, count), and the dimension columns to group by (such as host name and counter name). The agent collects records within each window, computes the aggregates, and emits one summary record per group.

Scenario: Roll up performance counters into 5-minute summaries

A fleet of 500 VMs, each reporting 10 performance counters every 15 seconds, generates roughly 2 million raw records per hour. Most operational dashboards and alert rules use 5-minute granularity, making the per-sample detail redundant.

With the aggregate processor, each agent rolls up its local counter stream into 5-minute windows, grouped by counter name. Each summary record contains the average, maximum, and sample count for that window.

	Raw data	After aggregation (5-min windows)
Records per VM per hour	2,400 (10 counters x 4/min x 60 min)	120 (10 counters x 12 windows)
Records across 500 VMs per hour	1,200,000	60,000
Volume reduction		95%
Operational fidelity	Per-sample (15s)	Avg, max, and count per 5 min

Because the aggregation runs on the agent, the reduced data set is what gets transmitted and ingested. Dashboards and alerts that rely on 5-minute granularity work identically, but ingestion costs drop by 95%. Route the output to a custom table with columns that match the aggregate output (average, max, count, and your dimension columns).

Chain processors for complete pipelines

Processors are composable. A common pattern chains a header processor (to convert raw data into tabular format), a filter (to drop irrelevant records), a parse step (to extract fields from structured payloads), and a column drop (to remove fields not needed downstream).

Scenario: Parse, filter, and slim down Windows Event logs

Consider a security team that needs logon success and failure events (Event IDs 4624 and 4625) from the Windows Security log. The raw event stream contains hundreds of event types, each carrying a large XML payload. A four-step pipeline handles this:

Header processor converts the raw event stream into tabular rows
Parse processor extracts EventID and TargetUser from the XML payload into typed columns
Filter processor keeps only logon success (4624) and failure (4625) events, dropping everything else
Drop processor removes the bulky RawXml and RenderingInfo columns that are no longer needed

The result is a lean, security-focused data set containing only the events and fields the team actually queries. Each step is independent and can be modified without affecting the others.

Authoring multi-stage DCRs

Multi-stage transformations are available through the Azure portal and through the REST API (version 2025-05-11). The portal provides a visual editor for building processor pipelines, previewing the schema at each stage, and validating the configuration before deployment.

The Transform tab in the DCR data source configuration lets you add processors at each stage and preview the resulting schema.

For infrastructure-as-code workflows, the full DCR JSON can be authored and deployed via ARM templates, Bicep, or direct REST API calls.

To get started:

Open Azure Monitor in the Azure portal and navigate to Data Collection Rules
Create a new DCR or edit an existing one
In the data source configuration, select Edit transformation
Author your transformation logic across client and ingestion stages using the set of available processors
Preview the schema output at each stage to verify the pipeline produces the expected result
Save and associate the DCR with your target resources

Preview notes:

Multi-stage transformations are available in public preview starting June 3, 2026
Client-side processors require Azure Monitor Agent version 1.35 or later
Aggregation output must be routed to custom tables (standard table schemas do not match aggregate output)
Data collection, workspace ingestion, and alert rules may incur costs based on the settings you enable. Preview pricing may differ from general availability pricing. See Azure Monitor pricing for current rates

To learn more, see:

Data Collection Rules overview

Looking ahead

Multi-stage transformations are part of our continued investment in giving teams control over their data before it reaches the workspace. During the preview period, we plan to expand processor coverage, add support for additional data source types, and incorporate user feedback into the authoring and validation experience.

We are also exploring how multi-stage transformations can serve as the foundation for advanced scenarios such as data scrubbing, inline enrichment from external reference data, and AI-assisted pipeline authoring. These capabilities will build on the same processor model, so pipelines you create today will extend naturally as new processors become available.

We welcome your feedback as you try multi-stage transformations. Use the feedback options in the Azure portal, or reach out through your Microsoft account team.

This feature is currently in preview. Previews are provided "as-is," "with all faults," and "as available," and are excluded from the service level agreements and limited warranty. For more information, see Supplemental Terms of Use for Microsoft Azure Previews].

Statements in this post about future plans and capabilities represent our current intentions and are subject to change. They should not be relied upon when making purchasing decisions.

When Telemetry Volume Gets Real: Azure Monitor pipeline’s Performance Story!

susaraswat4 — Wed, 03 Jun 2026 17:19:17 GMT

What is Azure Monitor pipeline?

Azure Monitor pipeline provides centralized governance and a single point of control that runs close to your data sources, so you can filter, transform, aggregate, and route telemetry before it's sent to Azure Monitor. This approach helps you reduce ingestion volume, improve reliability in disconnected environments, and apply consistent data processing across hybrid and multi-cloud deployments. Built on OpenTelemetry technology, the pipeline supports standard ingestion protocols including Syslog and OTLP, enabling it to receive telemetry from a wide range of clients and environments. Read more about Azure Monitor pipeline here - Azure Monitor pipeline GA: Centralized, Secure Telemetry Ingestion

Azure Monitor pipeline Performance

A single replica on a stock 8-core node sustains ~200,000 Syslog messages per second end-to-end into Log Analytics — roughly 17 billion events or ~20 TB per day — using only ~2.8 GB of working-set memory.

That's ~2.5 TB/day of throughput per vCPU, on commodity hardware, with no special tuning. (Measured on pipeline v1.1.1, May 2026.) Find more detailed performance information in the table below -

vCPUs	Example node	Syslog Basic*	Syslog Fully Formed*	CEF Fully Formed*
2	Standard_D2as_v6	~50,000/sec	~35,000/sec	~17,000/sec
4	Standard_D4as_v6	~100,000/sec	~70,000/sec	~35,000/sec
8	Standard_D8as_v6	~200,000/sec	~150,000/sec	~65,000/sec
16	Standard_D16as_v6	~400,000/sec	~300,000/sec	~130,000/sec

Syslog Basic* – Azure Monitor pipeline ingesting raw syslog data into Azure Monitor custom table

Syslog Fully Formed* – Azure Monitor pipeline ingesting syslog data in Azure Monitor standard syslog table

CEF Fully Formed* – Azure Monitor pipeline ingesting CEF data in Azure Monitor standard CEF table

Further, adding replicas scales throughput linearly. Linear scaling is what makes the rest of the performance story credible in practice: if one 4-core node handles about 100,000 Syslog logs per second, eight replicas scale that to roughly 800,000 logs per second without changing the architecture. In other words, you do not hit an arbitrary throughput wall as volume grows—you add cores or replicas and get predictable capacity growth. We are continuously improving these numbers, and the latest guidance is documented here -- Azure Monitor pipeline performance and sizing - Azure Monitor | Microsoft Learn

Why this Performance Story Matters?

Zero-config core usage. The pipeline automatically uses every available CPU core. Move to a bigger node and it just goes faster — no tuning, no config.
Backpressure, not data loss. When you exceed capacity, the pipeline applies TCP backpressure to senders instead of dropping messages. Rising send latency is your scale-up signal.
Predictable sizing math. Pick your per-vCPU rate, divide your peak logs/sec, add 30% headroom, round up. Done.
Efficient memory usage. ~2.8 GB working-set to push 200,000 logs/sec means you're paying for throughput, not overhead.

One sizing tip worth knowing: make sure senders open at least as many concurrent TCP connections as there are cores on the pipeline node. The pipeline distributes traffic across cores by source connection, so too few connections leave cores idle.

How this Stacks Up?

Telemetry pipelines are usually sized per CPU core, making per-core throughput a practical way to reason about capacity and scaling.

Against that backdrop, ~2.5 TB/day per vCPU for Syslog Basic — and ~65,000–150,000 logs/sec, on 8 cores for fully formed records — highlights the per-core efficiency of Azure Monitor pipeline for edge log collection. Exact numbers will vary based on event size and processing applied, but the key point is consistency: you get substantial throughput per core, and it scales linearly as you add capacity.

Less hardware to move the same volume, efficient memory usage, backpressure instead of loss, and linear growth — that's the performance case for Azure Monitor pipeline.

Get started

Spin up a pipeline group on your Arc-enabled cluster, point your Syslog/CEF senders at it, and watch the throughput numbers above hold up in your own environment! Read more about getting started here -- What is Azure Monitor pipeline? - Azure Monitor | Microsoft Learn

New Capabilities to Observe Agents in Azure Monitor

MattMc — Fri, 05 Jun 2026 22:09:44 GMT

Over the last six months, we have been listening to you and building new capabilities to help you observe your agents. You’ve been sharing with us that quality issues are tricky and evaluation is critical, that agent reasoning needs to be understood, that humans must be in the loop to review select agent interactions, and that security and privacy are essential.

To address these concerns, we’re announcing several new capabilities that make agents a first-class artifact in Azure Monitor, so you can debug them in the context of your broader distributed application alongside non-agentic components. Microsoft Foundry remains the surface for building and evaluating agents within the context of your project, while Azure Monitor provides the full-stack observability platform and underlying data foundation that powers those experiences.

Today, we’re announcing new capabilities in Azure Monitor across ingestion, performance, evaluation workflows, agent debugging, and instrumentation updates to help teams get telemetry faster, inspect agent behavior more deeply, and standardize observability across hosting environments and frameworks.

What’s new

Reducing pipeline latency from more than 60 seconds to 7.5 seconds at P90. This makes telemetry available faster for teams troubleshooting agents at scale.
Emitting events up to 1MB and up to 256kB per attribute. Prompts and responses can get large, and this helps avoid data truncation.
Introducing a new view that shows a list of all agents being monitored. Whether you use Microsoft Agent Framework, LangChain, Microsoft Copilot Studio, Foundry Hosting, AKS Hosting, or something else, they all show up here.

Image 1. Fleet View

Improving drill-in from Evaluations to underlying prompts/responses. Evaluations in Azure Monitor are powered by Foundry, and we continue to improve visuals.

Image 2. Evaluation Summary

Showing conversation context in end-to-end transaction view. In chat agents, conversations have become critical glue that connects traces and eases debugging.
Searching by text and showing prompt previews in end-to-end transaction view. Prompts and responses are essential to understanding agent logic, and now you can search based on keyword text in Search and End-to-end transaction details views.
Show evaluation scores in end-to-end transaction details and sort by evaluation score in Search. Evaluation is emerging as a “4^th pillar” of telemetry, and you’ll see it surface more prominently across Azure Monitor Application Insights.
Access the entire JSON blob of prompt/response text. This makes it easier to get to your underlying data and copy out of Azure Monitor for custom analysis/evaluation.

Image 3. Reference image for items No 5-8.Image 4. Correlates with item No 8. Raw JSON.

Adding a “trace tree” to enhance traversing the agent’s reasoning logic. This new addition to end-to-end transaction view makes traversing long-traces much easier.

Image 5. Trace Tree Graph View.

Enabling builders to annotate (i.e., manual evaluations) from transaction details. Get rid of spreadsheets on the side and annotate from within Azure Monitor.

Image 6. Human-in-the-loop Manual Evaluation

Enabling capture of end-user feedback (i.e., thumbs up/down). Brings end-user feedback alongside other telemetry for more powerful troubleshooting.
Extending AI-powered troubleshooting to agents. Observability agent offers full-stack, AI-powered troubleshooting and surfaces up findings in an issue. Learn More.

Image 7. Azure Copilot Observability Agent (in Azure Monitor)

Observability of Coding Agents. Get end-to-end visibility into agent and model usage, performance, and cost with Azure Monitor Application Insights, and built-in Grafana dashboards. Learn More.
A unified “Microsoft OpenTelemetry Distro” to observe agents hosted anywhere. A unified Microsoft OpenTelemetry Distro for observing agents hosted anywhere gives teams a single starting point across Foundry, Azure Monitor, and A365, reducing fragmentation and simplifying onboarding (GH Repos: Python, .NET, JavaScript).
Skills-based enablement. Getting started is easier. Just point your agent to a skill for AI-assisted instrumentation. We also plan to upgrade tools for instrumentation in Azure MCP.

What’s next

We’re continuing to invest in this area, with upcoming work focused on stronger security controls for prompts and responses, better cost transparency for agents, and clearer ways to measure ROI across your agent fleet.

These updates make it possible to observe agents without adopting a separate toolchain. Explore the new capabilities, and if you see gaps, let us know so we can continue shaping the roadmap based on your feedback. Learn More.