Making Azure the Best Place to Observe Your Apps with OpenTelemetry
Our goal is to make Azure the most observable cloud. To that end, we are refactoring Azure’s native observability platform to be based on OpenTelemetry, an industry standard for instrumenting applications and transmitting telemetry.
Query Azure Resource Graph from Azure Monitor
Azure Monitor now lets you query data in Azure Resource Graph from your Log Analytics workspace. You can utilize this feature to make your Azure Log Analytics queries aware of your resource’s properties such as tags, resource attributes and much more. You can run cross-service queries by using any client tools that support Kusto Query Language (KQL) queries, including the Log Analytics web UI, workbooks, PowerShell, and the REST API. Read more here
Log Analytics results upgrade
Log Analytics team has shipped one of it's most significant updates to date - a new and upgraded results set grid. The new results set boasts a brand new experience and designed aimed at empowering Log Analytics users to do more with Logs. The new result set introduces numerous new features and improvements and is designed to be more accessible and easy to use. Users can now use on grid filters, drag and drop actions and even Pivot mode to easily analyze their query results with no KQL knowledge needed.Accelerate your observability journey with Azure Monitor pipeline (preview)
In the ever-evolving landscape of digital infrastructure, transparency in resource and application performance is imperative. Success hinges on visibility, and that’s true whether you’re operating on Azure, on-premise, or at the edge. As organizations scale their infrastructures and applications, the volume of observability data naturally increases. This surge can complicate the management of networking, data storage and ingestion, often forcing a trade-off between cost management and observability. The complexity doesn’t end there. The very tools designed to ingest, process, and route this data can be both costly and complex, adding layers of operational challenges. Moreover, edge infrastructure is deployed near IoT devices for optimal data processing, high availability, and reduced latency. This adds its own set of challenges when it comes to collecting telemetry from such constrained environments. Recognizing these challenges, our team has been focused on providing a robust, highly scalable, and secure data ingestion solution through Azure Monitor. We are thrilled to announce the preview of the Azure Monitor pipeline at edge. What is Azure Monitor pipeline? Azure Monitor pipeline, similar to ETL (Extract, Transform, Load) process, enhances traditional data collection methods. It streamlines data collection from various sources through a unified ingestion pipeline and utilizes a standardized configuration approach that is more efficient and scalable. This is particularly beneficial for cloud-based monitoring in Azure. We are now extending our Azure Monitor pipeline capabilities from the cloud to the edge, enabling high-scale data ingestion with centralized configuration management. What is Azure Monitor pipeline at edge? Azure Monitor pipeline at edge is a powerful solution designed to facilitate high-scale data ingestion and routing from edge environments to Azure Monitor for observability. It leverages the robust capabilities of the vendor-agnostic tool - OpenTelemetry Collector, which is used by enterprises worldwide to manage high volumes of telemetry each month. With the Azure Monitor pipeline at edge, organizations can tap into the same highly scalable platform with a standardized configuration and reliability. Whether dealing with petabytes of data or seeking consistent observability experience across Azure, edge, and multi-cloud, this solution empowers organizations to reliably collect telemetry and drive operational excellence. The Azure Monitor pipeline at edge is equipped with out-of-the-box capabilities to receive telemetry from a diverse range of resources and route it to Azure Monitor. Here are some key features: High scale data ingestion: Customers have various devices and resources at edge, emitting high volume of data. With Azure Monitor pipeline at edge, you can seamlessly scale to support ingestion of high volume of data in the cloud. Azure Monitor pipeline can be deployed on your on-premises Kubernetes cluster as an Arc Kubernetes cluster extension. This allows it to adapt to your data scaling needs by running multiple replica sets and provides you with full control to define workflows and route high-volume data to Azure Monitor. Observing resources in isolated environments: In the manufacturing sector, resources are often located in isolated network zones without direct cloud connectivity, posing challenges for telemetry collection. With the Azure Monitor pipeline at edge, combined with Azure IoT Layered Network Management, you can facilitate a connection between Azure and Kubernetes clusters in isolated networks, deploy the Azure Monitor pipeline at edge, collect data from resources in segmented networks, and route it to Azure Monitor for comprehensive observability. Reliable data ingestion and prevent data loss: Edge environments frequently encounter intermittent connectivity, leading to potential data loss and disrupting data continuity. The Azure Monitor pipeline at edge allows you to cache logs during periods of intermittent connectivity. When connectivity is re-established, your data is synchronized with Azure Monitor, preventing data loss. Getting started It’s super easy to get started! You need to deploy the Azure Monitor pipeline on a single Arc-enabled Kubernetes cluster in your environment. Once that is done, you can configure your resources to emit the telemetry to Azure Monitor pipeline at edge and ingest into Azure Monitor for observability. Once you Arc-enable your on-prem Kubernetes cluster and the prerequisites are met, go the Extension section, select Azure Monitor pipeline extension (preview) and create the instance. Alternatively, from the search bar in the Azure portal, select Azure Monitor pipeline and then click Create. Enter the information related to the pipeline instance. The Dataflow tab allows you to create and edit dataflows for the pipeline instance. Configure your resources to emit the telemetry to the Azure Monitor pipeline. Learn more in our documentation. Pricing There is no additional cost to use Azure Monitor pipeline to send data to Azure Monitor. You will be only charged for data ingestion as per the current pricing. FAQ What telemetry can be collected using Azure Monitor pipeline? Currently, in public preview, you can collect syslogs and OTLP logs using Azure Monitor pipeline at edge. We will keep expanding the data collection capabilities based on your feedback and requirements. How can I perform transformations on the telemetry that is collected? You can certainly transform your telemetry! Since this is an extension of Azure Monitor pipeline, you can perform the data collection transformations in the Azure Monitor pipeline at cloud. Is this another agent for data collection? Azure Monitor pipeline at edge is engineered to function in environments where installing agents on resources is not feasible, whether due to technical limitations or warranty concerns. It enables you to get the telemetry from these resources and acts as a central forwarding component to ingest high volume data. I have 100 Linux servers in my on-prem environment. Do I need to deploy Azure Monitor pipeline at edge on all of them? You need to deploy the Azure Monitor pipeline at edge on a single Arc-enabled Kubernetes cluster and configure it to ingest data into Azure Monitor. Once that is completed, you can configure your Linux servers to emit telemetry to the Azure Monitor pipeline at edge instance.Join Azure Monitor 'Monthly Connections' on June 30 for live Q&A
Join the Azure Monitor engineering team for our first 'Monthly Connections' event on June 30, 2020. This will be an "Ask-Me-Anything" event where you can have all of your monitoring questions answered live by our panel of experts. We will also highlight all the new scenarios & capabilities launched in the last month, that you can start using today!
Azure Copilot Observability Agent is generally available, with autonomous operations in preview
Complex cloud environments have outpaced manual operations. Agentic cloud operations connect people, tools, and data to streamline investigation workflows and move teams from scattered signals to evidence-backed next steps. With unified observability, teams can investigate Azure-monitored applications, Azure Kubernetes Service (AKS) environments, VMs, Foundry telemetry, infrastructure, and platform signals with greater context and control. Powered by Azure Monitor, the Azure Copilot Observability Agent is now generally available. It helps engineering, SRE, DevOps, and operations teams move from telemetry and alert noise to investigated issues, explainable reasoning, and recommended next steps that can reduce Time-To-Mitigate (TTM). Autonomous operations are also available in public preview. They help prepare context and reduce triage work while people remain responsible for mitigation decisions and any changes to the environment. From alert noise to investigated issues The Observability Agent helps teams reduce the effort required to understand operational problems. Instead of starting every investigation from a dashboard, query editor, or alert payload, teams can work with an AI companion that reasons across telemetry, Azure resource context, discovered topology, and custom instructions to identify what changed, what is correlated, and what evidence supports the conclusion. Teams can start with natural-language exploration and continue into deeper investigations when an issue requires more evidence. That light-to-deep workflow helps responders move from broad questions to a structured investigation without losing the reasoning trail. Here's what this looks like in practice: after a deployment, several alerts might fire across an app, database dependency, and compute resource. The Observability Agent can group those signals around the affected service, identify when the regression started, compare related dependencies and infrastructure metrics, and capture the findings in an Azure Monitor issue. The responder can then validate the evidence, add team context, route work to the right owner, and decide whether a rollback, configuration change, or code fix is appropriate. Explainable investigations across Azure-monitored signals Operations teams need more than a chatbot that answers questions. The Observability Agent follows an investigation workflow: it frames hypotheses, gathers evidence, compares signals by time, scope, and type, rules out weak explanations, and shows the reasoning path behind its findings. The Observability Agent can help teams: Investigate incidents and alerts across Azure-monitored applications, Azure Kubernetes Service (AKS) environments, VMs, Foundry telemetry, infrastructure, and platform signals Correlate related signals to reduce noise and surface higher-signal issues with context Explore telemetry using natural language while preserving transparency into the supporting data Compare signals by time, scope, and type to separate likely causes from coincidental changes Provide a reasoning trail that shows what the agent found, what it ruled out, and why Recommend next steps that engineers can review before deciding how to act This same investigation model applies to specialized skills and issue types, including customer's application, Azure Kubernetes Service (AKS), Foundry, VMs, and GenAI issues. When the relevant telemetry is available, the Observability Agent can correlate logs, metrics, traces, alerts, dependencies, resource graph, resource health, activity logs, Foundry telemetry, and changes. This helps teams investigate customer-visible issues with evidence, including latency, token spikes, tool-call failures, agent errors, hallucinations, deployments, API failures, performance regressions, infrastructure dependencies, and platform incidents. This explainability is central to the product. In production operations, trust is earned through evidence. The Observability agent is built to support human judgment, not bypass it. . Azure expertise, with context from your environment Context matters in every investigation. The same symptom can mean different things depending on application architecture, recent deployments, dependencies, historical incidents, and team practices. The Observability Agent brings Microsoft and Azure operational knowledge into the investigation experience. It can use discovered topology, Azure resource context, logs, metrics, traces, and custom instructions to ground investigations in signals that are more relevant to your environment. Native to Azure Monitor, with humans in control Because the Observability Agent is built into Azure Monitor, teams can use it close to the telemetry, alerts, and workflows they already rely on. Investigations can also be captured as Azure Monitor issues, creating a shared case file for humans and agents to collaborate on evidence, reasoning, and next steps. The Observability Agent is designed for governed AI operations inside Azure Monitor. Interactive chat and investigations use the signed-in user's identity and Azure role-based access control (RBAC). Prompts and responses are not used to train foundation models, and the agent doesn't restart resources, change configuration, or resolve issues on its own. Autonomous operations in public preview Alongside general availability, autonomous operations for the Observability Agent are available in public preview. When enabled, the agent can analyze alerts in the background, correlate related alerts when they likely represent the same incident, create Azure Monitor issues automatically, and run deep investigations on agent-created issues. This automatic triage helps reduce alert noise by turning streams of individual alerts into higher-signal issues with context, findings, and recommended next steps. Teams can review the issue, continue the investigation, and decide what action to take. Autonomous operations are designed to prepare context and reduce triage work, not to remove human control. Engineers remain responsible for decisions, approvals, and any changes to the environment. Next steps Check out our latest announcements and related blogs: Azure Blog and OMB Blog. Learn how to use the Observability Agent in Azure Copilot Observability Agent. Explore how investigations work in Deep investigations in the Azure Copilot Observability Agent. Learn more on how to Chat with your observability data Learn how teams preserve context in Azure Monitor issues. Review preview details in Autonomous operations in the Azure Copilot Observability Agent. Stay connected Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next. Live webinar - a walkthrough of real Observability Agent scenarios, best practices, and what's available today - along with a look at what's coming next, and live Q&A with the product team. Register for the Observability Agent webinar. We'd love your feedback The Observability agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience, or reach us at enauerman@microsoft.com.Making it easier to manage your workspace with a clearer Log Analytics Workspace menu
Customers have been using Azure Monitor Log Analytics workspaces to store and analyze logs for almost a decade. Over these years, the service has evolved, with new options being added and old technologies phased out. We’ve tried to keep the workspace administration UI stable without making too many changes, but it’s time for a refresh. Today we start rolling out a revamped workspace menu that will help you get to the things you need quicker. Key items are now right at the top, where you expect them to be: The new menu is gradually being exposed, so you might not see it immediately. As part of this change, we're also clearly indicating menu options that are in the process of replacement or deprecation. Here are the relevant options and their replacement: Menu option Replacement technology The MMA and OMS agents are being replaced by the AMA agent. For more details and assistance in the process, see https://aka.ms/migrateToAMA. All table management operations are now done on a single table management UI, which covers all aspects of workspace tables, including custom logs. For details, see Manage tables in a Log Analytics workspace - Azure Monitor | Microsoft Learn. You no longer need connectors to export Azure Activity Logs to Log Analytics workspaces. For instructions, see Azure activity log - Azure Monitor | Microsoft Learn. Some agents in specific scenarios ingest logs to storage accounts. This option exists to collect logs from such accounts. See more details how to collect logs directly to Azure Monitor in https://aka.ms/migrateToAMA. Workspace summary is the old OMS portal for solutions built using View Designer. This technology was replaced with Workbooks and Insights several years ago. For more details, see View Designer in Azure Monitor is retiring on 31 August 2023 | Azure updates | Microsoft Azure. Azure Monitor VM Insights has evolved to use the new AMA agent, which also includes the Service Map functionality. For more details, see Migrate to Azure Monitor VM insights by 30 September 2025 when Service Map will be retired | Azure updates | Microsoft Azure. Azure Monitor had several technologies to help deploy and manage the old agent's data collection and logic. These technologies became obsolete with the move to the new AMA agent, which offers rich management and policy mechanisms. For more details, see https://aka.ms/migrateToAMA.Azure Monitor- Announcing Public Preview of Azure Metrics Data plane API
The Azure Monitor REST API , hosted behind Azure Resource Manager, is a widely used API service in Azure that allows customers to gain insights into Azure resources. However, customers who leverage this API to query metrics in bulk at high frequency and push metrics to other datastores for the purpose of deep analytics may experience throttling issues with the current API. To address this, and better empower these use cases, Microsoft Azure has introduced the Azure Metrics Data plane API, a new batch API that enables efficient and large-scale queries. We are excited today to announce the Public Preview of the Azure Metrics Data plane API. This powerful addition allows customers to query metrics for their Azure resources at a large scale. So, what is Azure Monitor Metrics Data plane API? Azure Metrics Data plane API is a new paradigm in Azure Monitoring which makes resource insight gathering more efficient with a higher capacity querying experience. It provides the capability to retrieve the metric data for up to 50 resource Ids in the same subscription and region in a single batch API call thus improving query throughput and reducing the risk of throttling. Here are some of the key features and benefits of Azure Metrics Data plane API - Higher Querying Limits: Azure Metrics Data plane API is designed for querying metric data for your resources with higher query limits than existing Azure Resource Manager APIs. This is particularly beneficial for customers who have large subscriptions with multiple resources, as they can query data without worrying about being throttled. Compared to the REST API, which only allows 12,000 API calls per hour, this new API enables you to perform up to 360,000 API calls per hour. This improvement in query throughput provides a more efficient and streamlined experience for customers. Efficiency: Collecting metrics for multiple resources in a single API call is more efficient than making multiple API calls for each resource. The Azure Metrics Data plane API offers a single batch API call which can accommodate up to 50 resource IDs, resulting in higher throughput and more efficient querying. Improved Performance: The Azure Metrics Data plane API can enhance the performance of client-side services by reducing the number of calls needed to extract the same amount of metric data for resources, resulting in improved performance. Overall, this API provides a powerful and efficient mechanism for organizations looking to monitor and manage their resources more effectively at a scale. By using this API, organizations can streamline their monitoring processes and improve their infrastructure's overall performance. We look forward to seeing the innovative solutions and exciting use cases that developers and organizations will create using the Azure Metrics Data plane API. Our API is being used by several customers in Private Preview and here is what some of them have to say about their experience with the Azure Metrics Data plane API: Customer Quote “Datadog uses Azure Monitor APIs to ingest metrics on behalf of thousands of organizations, including some of the largest Azure customers in the world. Through our partnership with the Azure Monitor team, we’ve been able to leverage the Data plane API to ingest data with minimal latency, enabling faster alerting and near real-team evaluation of health and performance even for enterprise-scale Azure environments,” says Jimmy Caputo, Director of Product Management at Datadog.” The new metrics Data plane API will be charged at a rate of $0.01/1000 API calls (Refer to pricing details here- Pricing - Azure Monitor | Microsoft Azure). The service is currently free to use, with billing set to begin on 10/1/2023. Please note current users of the Metrics REST API hosted behind the ARM APIs will see no change in pricing. Learn more about Azure Metrics Data plane API - Metrics in Azure Monitor - Azure Monitor | Microsoft Learn.
