Tag:"opentelemetry" | Microsoft Community Hub

Azure Monitor Application Insights Auto-Instrumentation for Java and Node Microservices on AKS
Key Takeaways (TLDR) Monitor Java and Node applications with zero code changes Fast onboarding: just 2 steps Supports distributed tracing, logs, and metrics Correlates application-level telemetry in Application Insights with infrastructure-level telemetry in Container Insights Available today in public preview Introduction Monitoring your applications is now easier than ever with the public preview release of Auto-Instrumentation for Azure Kubernetes Service (AKS). You can now easily monitor your Java and Node deployments without changing your code by leveraging auto-instrumentation that is integrated into the AKS cluster. This feature is ideal for developers or operators who are... Looking to add monitoring in the easiest way possible, without modifying code and avoiding ongoing SDK update maintenance. Starting out on their monitoring journey and looking to benefit from carefully chosen default configurations with the ability to tweak them over time. Working with someone else’s code and looking to instrument at scale. Or considering monitoring for the first time at the time of deployment. Before the introduction of this feature, users needed to manually instrument code, install language-specific SDKs, and manage updates on their own—a process that involved significant effort and numerous opportunities for errors. Now, all you need to do is follow a simple two-step process to instrument your applications and automatically send correlated OpenTelemetry-based application-level logs, metrics, and distributed tracing to your Application Insights resource. With AKS Auto-Instrumentation, you will be able to assess the performance of your application and identify the cause of any incidents more efficiently using the robust application performance monitoring capabilities of Azure Monitor Application Insights. This streamlined approach not only saves time but also ensures that your monitoring setup is both reliable and scalable. Feature Enablement and Onboarding To onboard to this feature, you will need to follow a two-step process: Prepare your cluster by installing the application monitoring webhook. Choose between namespace-wide onboarding or per-deployment onboarding by creating K8’s custom resources. Namespace-wide onboarding is the easiest method. It allows you to instrument all Java or Node deployments in your namespace and direct telemetry to a single Application Insights resource. Per-deployment onboarding allows more control by targeting specific deployments and directing telemetry to different Application Insights resources. Once the custom resource is created, you will need to deploy or redeploy your application, and telemetry will start flowing to Application Insights. For step-by-step instructions and to learn more about onboarding visit our official documentation on MS Learn. The Application Insights experience Once telemetry begins flowing, you can take advantage of Application Insights features such as Application Map, Failures/Performance Views, Availability, and more to help you efficiently diagnose and troubleshoot application issues. Let’s look at an example: I have an auto-instrumented distributed application running in the demoapp namespace of my AKS cluster. It consists of: One Java microservice Two Node.js microservices MongoDB and Redis as its data layer Scenario: End users have been complaining about some latency in the application. As the DRI, I can start my troubleshooting journey by going to the Application Map to get a topological view of my distributed application. I open Application Map and notice MicroserviceA has a red border - 50% of calls are erroring. The Container Insights card shows healthy pods - no failed pods or high CPU/memory usage. I can eliminate infrastructure issues as the cause of the slowness. In the Performance card, I spot that the rescuepet operation has an average duration of 10 seconds. That's pretty long. I drill in to get a distributed trace of the operation and find the root cause: an OutOfMemoryError. In this scenario, the issue has been identified as an out-of-memory error at the application layer. However, when the root cause is not in the code but in the infrastructure I get a full set of resource properties with every distributed trace so I can easily identify the infra resources running each span of my trace. I can click the investigate pods button to transition to Azure Monitor Container Insights and investigate my pods further. This correlation between application-level and infrastructure-level telemetry makes it much easier to determine whether the issue is caused by the application or the infrastructure. Pricing There is no additional cost to use AKS auto-instrumentation to send data to Azure Monitor. You will be only charged as per the current pricing. What’s Next Language Support This integration supports Java and Node workloads by leveraging the Azure Monitor OpenTelemetry distro. We have distros for .NET and Python as well and we are working to integrate these distros into this solution. At that point, this integration will support .NET, Python, Java and Node.js. For customers that want to instrument workloads in other languages such as Go, Ruby, PHP, etc. we plan to leverage open-source instrumentations available in the Open Telemetry community. In this scenario, customers will instrument their code using open source OpenTelemetry instrumentations, and we will provide mechanisms that will make it easy to channel the telemetry to Application Insights. Application Insights will expose an endpoint that accepts OpenTelemetry Language Protocol (OTLP) signals and configure the instrumented workload to channel the telemetry to this endpoint. Operating Systems and K8’s Controllers Right now, you can only instrument kubernetes deployments running on Linux node pools, but we plan to expand support to introduce support for Linux ARM64 node pools as well as support for StatefulSet, Job, Cronjob, and Replicaset controller types. Portal Experiences We are also working on Azure portal experiences to make onboarding easier. When our portal experiences for onboarding are released, users will be able to install the Application Insights extension for AKS using the portal and use a portal user interface to instrument their workloads instead of having to create custom resources. Beyond onboarding, we are working to build Application Insights consumption experiences within the AKS namespace and workloads blade. You will be able to see application-level telemetry right there in the AKS portal without having to navigate away from your cluster to Application Insights. FAQs: What are the advantages of AKS Auto-Instrumentation? No code changes required No access to source code required No configuration changes required Eliminates instrumentation maintenance What languages are supported by AKS Auto-Instrumentation? Currently, AKS Auto-Instrumentation supports Java and Node.js applications. Python and .NET support is coming soon. Moreover, we will be adding support for all OTel supported languages like Go soon via native OTLP ingestion. Does AKS Auto-Instrumentation support custom metrics? For Node.js applications, custom metrics require manual instrumentation with the Azure Monitor OpenTelemetry Distro. Java applications allow custom metrics with auto-instrumentation. Click here for more FAQs. This article was co-authored by Rishab Jolly and Abinet Abate
abinetabate
Apr 16, 2025 Place Azure Observability Blog
302Views
0likes
0Comments
Azure Monitor Private Link Scope (AMPLS) Scale Limits Increased by 10x!
What is Azure Monitor Private Link Scope (AMPLS)? Azure Monitor Private Link Scope (AMPLS) is a feature that allows you to securely connect Azure Monitor resources to your virtual network using private endpoints. This ensures that your monitoring data is accessed only through authorized private networks, preventing data exfiltration and keeping all traffic inside the Azure backbone network. AMPLS – Scale Limits Increased by 10x in Public Cloud - Public Preview In a groundbreaking development, we are excited to share that the scale limits for Azure Monitor Private Link Scope (AMPLS) have been significantly increased by tenfold (10x) in Public Cloud regions as part of the Public Preview! This substantial enhancement empowers our customers to manage their resources more efficiently and securely with private links using AMPLS, ensuring that workload logs are routed via the Microsoft backbone network. Addressing Customer Challenges Top Azure Strategic 500 customers, including leading Telecom service providers, Banking & Financial services customers, have reported that the previous limits of AMPLS were insufficient to meet their growing demands. The need for private links has surged 3-5 times beyond capacity, impacting network isolation and integration of critical workloads. Real-World Impact Our solution now enables customers to scale their Azure Monitor resources significantly, ensuring seamless network configurations and enhanced performance. Scenario 1: A Leading Telecom Service Provider known for its micro-segmentation architecture, have faced challenges with large-scale monitoring and reporting due to limitations on AMPLS. With the new solution, the customer can now scale up to 3,000 Log Analytics and 10,000 Application Insights workspaces with a single AMPLS resource, allowing them to configure over 13,000 Azure Monitor resources effortlessly. Scenario 2: A Leading Banking & Financial Services Customer have faced the scale challenges in delivering personalized insights due to complex workflows. By utilizing Azure Monitor with network isolation configurations, the customer can now scale their Azure Monitor resources to ensure secure telemetry flow and compliance. They have enabled thousands of Azure Monitor resources configured with AMPLS. Key Benefits to the Customer We believe that the solution our team has developed will significantly improve our customers' experience, allowing them to manage their resources more efficiently and effectively with private links using AMPLS. An AMPLS object can now connect up to 3,000 Log Analytics workspaces and 10,000 Application Insights components. (10x Increase) The Log Analytics workspace limit has been increased from 300 to 3,000 (10x increase). The Application Insights limit has increased from 1,000 to 10,000 (10x increase). An Azure Monitor resources can now connect up to 100 AMPLSs (20x increase). Data Collection Endpoint (DCE) Log Analytics Workspace (LA WS) Application Insights components (AI) An AMPLS object can connect to 10 private endpoints at most. Redesign of AMPLS – User experience to load 13K+ resources with Pagination Call to Action Explore the new capabilities of Azure Monitor Private Link Scope (AMPLS) and see how it can transform your network isolation and resource management. Visit our Azure Monitor Private Link Scope (AMPLS) documentation page for more details and start leveraging these enhancements today! For detailed information on configuring Azure Monitor private link scope and azure monitor resources, please refer to the following link: Configure Azure Monitor Private Link Scope (AMPLS) Configure Private Link for Azure Monitor
Mahesh_Sundaram
Mar 21, 2025 Place Azure Observability Blog
237Views
0likes
0Comments
Accelerate your observability journey with Azure Monitor pipeline (preview)
In the ever-evolving landscape of digital infrastructure, transparency in resource and application performance is imperative. Success hinges on visibility, and that’s true whether you’re operating on Azure, on-premise, or at the edge. As organizations scale their infrastructures and applications, the volume of observability data naturally increases. This surge can complicate the management of networking, data storage and ingestion, often forcing a trade-off between cost management and observability. The complexity doesn’t end there. The very tools designed to ingest, process, and route this data can be both costly and complex, adding layers of operational challenges. Moreover, edge infrastructure is deployed near IoT devices for optimal data processing, high availability, and reduced latency. This adds its own set of challenges when it comes to collecting telemetry from such constrained environments. Recognizing these challenges, our team has been focused on providing a robust, highly scalable, and secure data ingestion solution through Azure Monitor. We are thrilled to announce the preview of the Azure Monitor pipeline at edge. What is Azure Monitor pipeline? Azure Monitor pipeline, similar to ETL (Extract, Transform, Load) process, enhances traditional data collection methods. It streamlines data collection from various sources through a unified ingestion pipeline and utilizes a standardized configuration approach that is more efficient and scalable. This is particularly beneficial for cloud-based monitoring in Azure. We are now extending our Azure Monitor pipeline capabilities from the cloud to the edge, enabling high-scale data ingestion with centralized configuration management. What is Azure Monitor pipeline at edge? Azure Monitor pipeline at edge is a powerful solution designed to facilitate high-scale data ingestion and routing from edge environments to Azure Monitor for observability. It leverages the robust capabilities of the vendor-agnostic tool - OpenTelemetry Collector, which is used by enterprises worldwide to manage high volumes of telemetry each month. With the Azure Monitor pipeline at edge, organizations can tap into the same highly scalable platform with a standardized configuration and reliability. Whether dealing with petabytes of data or seeking consistent observability experience across Azure, edge, and multi-cloud, this solution empowers organizations to reliably collect telemetry and drive operational excellence. The Azure Monitor pipeline at edge is equipped with out-of-the-box capabilities to receive telemetry from a diverse range of resources and route it to Azure Monitor. Here are some key features: High scale data ingestion: Customers have various devices and resources at edge, emitting high volume of data. With Azure Monitor pipeline at edge, you can seamlessly scale to support ingestion of high volume of data in the cloud. Azure Monitor pipeline can be deployed on your on-premises Kubernetes cluster as an Arc Kubernetes cluster extension. This allows it to adapt to your data scaling needs by running multiple replica sets and provides you with full control to define workflows and route high-volume data to Azure Monitor. Observing resources in isolated environments: In the manufacturing sector, resources are often located in isolated network zones without direct cloud connectivity, posing challenges for telemetry collection. With the Azure Monitor pipeline at edge, combined with Azure IoT Layered Network Management, you can facilitate a connection between Azure and Kubernetes clusters in isolated networks, deploy the Azure Monitor pipeline at edge, collect data from resources in segmented networks, and route it to Azure Monitor for comprehensive observability. Reliable data ingestion and prevent data loss: Edge environments frequently encounter intermittent connectivity, leading to potential data loss and disrupting data continuity. The Azure Monitor pipeline at edge allows you to cache logs during periods of intermittent connectivity. When connectivity is re-established, your data is synchronized with Azure Monitor, preventing data loss. Getting started It’s super easy to get started! You need to deploy the Azure Monitor pipeline on a single Arc-enabled Kubernetes cluster in your environment. Once that is done, you can configure your resources to emit the telemetry to Azure Monitor pipeline at edge and ingest into Azure Monitor for observability. Once you Arc-enable your on-prem Kubernetes cluster and the prerequisites are met, go the Extension section, select Azure Monitor pipeline extension (preview) and create the instance. Alternatively, from the search bar in the Azure portal, select Azure Monitor pipeline and then click Create. Enter the information related to the pipeline instance. The Dataflow tab allows you to create and edit dataflows for the pipeline instance. Configure your resources to emit the telemetry to the Azure Monitor pipeline. Learn more in our documentation. Pricing There is no additional cost to use Azure Monitor pipeline to send data to Azure Monitor. You will be only charged for data ingestion as per the current pricing. FAQ What telemetry can be collected using Azure Monitor pipeline? Currently, in public preview, you can collect syslogs and OTLP logs using Azure Monitor pipeline at edge. We will keep expanding the data collection capabilities based on your feedback and requirements. How can I perform transformations on the telemetry that is collected? You can certainly transform your telemetry! Since this is an extension of Azure Monitor pipeline, you can perform the data collection transformations in the Azure Monitor pipeline at cloud. Is this another agent for data collection? Azure Monitor pipeline at edge is engineered to function in environments where installing agents on resources is not feasible, whether due to technical limitations or warranty concerns. It enables you to get the telemetry from these resources and acts as a central forwarding component to ingest high volume data. I have 100 Linux servers in my on-prem environment. Do I need to deploy Azure Monitor pipeline at edge on all of them? You need to deploy the Azure Monitor pipeline at edge on a single Arc-enabled Kubernetes cluster and configure it to ingest data into Azure Monitor. Once that is completed, you can configure your Linux servers to emit telemetry to the Azure Monitor pipeline at edge instance.
XemaPathak
Jan 22, 2025 Place Azure Observability Blog
10KViews
7likes
3Comments
What’s new in Observability and Resiliency @ Ignite 2024
At Ignite 2024, we are excited to announce new capabilities that will help enhance the performance of your business-critical applications and workloads across cloud-to-edge environments.
Julie_Wang
Nov 22, 2024 Place Azure Observability Blog
1.7KViews
3likes
1Comment
What's new in observability @ Build 2024
Check out our new announcements at Build 2024 and where you can find us at the event!
Julie_Wang
May 21, 2024 Place Azure Observability Blog
4.6KViews
4likes
0Comments
Optimizing Cost using the Azure Monitor OpenTelemetry Distro
Immediately after Willow deployed the Azure Monitor OpenTelemetry Distro, their Azure Costs spiked, and they approached Azure Monitor Team for strategies to optimize cost. By following the strategies in this blog, they were able to reduce costs by over 90%!
MattMc
Apr 17, 2024 Place Azure Observability Blog
4KViews
2likes
0Comments
Making Azure the Best Place to Observe Your Apps with OpenTelemetry
Our goal is to make Azure the most observable cloud. To that end, we are refactoring Azure’s native observability platform to be based on OpenTelemetry, an industry standard for instrumenting applications and transmitting telemetry.
MattMc
Mar 29, 2024 Place Azure Observability Blog
18KViews
12likes
2Comments
OpenTelemetry + Azure Monitor
Learn about Azure Monitor's investments in OpenTelemetry and how they may impact you now and in the future.
MattMc
Dec 06, 2023 Place Azure Observability Blog
22KViews
6likes
2Comments
General Availability: Azure Monitor managed service for Prometheus
Today at Microsoft Build 2023 we are excited to announce that Azure Monitor managed service for Prometheus is now generally available!
Scott_Kinghorn
Oct 04, 2023 Place Azure Observability Blog
16KViews
4likes
2Comments
What’s new in Azure Monitor @ Build 2023
Check out our announcements at Build 2023!
artigulwadi
May 25, 2023 Place Azure Observability Blog
8.6KViews
5likes
0Comments