NOTE: A more recent OpenTelemetry Update has been published here.
Microsoft is excited to embrace OpenTelemetry as the future of telemetry instrumentation. You, our customers, have asked for vendor-neutral instrumentation, and we're delighted to partner with the OpenTelemetry community to create consistent APIs/SDKs across languages.
Microsoft worked together with project stakeholders from two previously popular open-source telemetry projects, OpenCensus and OpenTracing, to help create a single project -- OpenTelemetry. OpenTelemetry includes contributions from all major cloud and Application Performance Management (APM) vendors and is housed by the Cloud Native Computing Foundation (CNCF) of which Microsoft is a Platinum Member.
This blog focuses on OpenTelemetry as it relates to Azure Monitor Application Insights. In the future, we hope to provide more context on how OpenTelemetry is impacting other parts of Azure!
Concepts and Progress
Telemetry, the data collected to observe your application, can be broken into three types or "pillars":
- Distributed Traces
- Metrics
- Logs
Initially the OpenTelemetry community took on Distributed Traces, followed by Metrics and Logs. A complete observability story includes all three pillars. Azure Monitor is working on an OpenTelemetry-based supported offering that includes Distributed Traces and metrics with plans to add the logging pillar next.
In the following sections, we'll cover some telemetry collection basics.
Instrumenting your application
At a basic level, "instrumenting” is simply enabling an application to capture telemetry.
There are two methods to instrument your application:
- Manual Instrumentation
- Automatic Instrumentation (Auto-Instrumentation)
Manual instrumentation is coding against the OpenTelemetry API. In the context of an end-user, it typically refers to installing a language-specific SDK in an application. Azure Monitor’s OpenTelemetry-based manual instrumentation offerings on our roadmap consist of Azure Monitor Exporters in C#, JavaScript (Node.js), and Python.
“Manual” does not mean you’ll be required to write complex code to define spans for distributed traces (though it remains an option). A rich and growing set of instrumentation libraries maintained by OpenTelemetry contributors will enable you to effortlessly capture telemetry signals across common frameworks and libraries. A subset of OpenTelemetry Instrumentation Libraries will be supported by Azure Monitor, informed by customer feedback. Additionally, we are working to instrument the most popular Azure Service SDKs to emit OpenTelemetry Protocol (OTLP) and optimizing them for Azure Monitor.
Auto-instrumentation is enabling telemetry collection through configuration without touching the application's code. While more convenient, it tends to be less configurable and it’s not available in all languages. Azure Monitor’s OpenTelemetry-based auto-instrumentation offering consists of the Java 3.X GA offering, and we continue to invest in it informed by customer feedback. The OpenTelemetry community is also experimenting with C# and Python auto-instrumentation, but Azure Monitor is focused on creating a simple and effective manual instrumentation story in the near-term.
OpenTelemetry + Azure Monitor Roadmap
*We are evaluating the suitability of OpenTelemetry JavaScript for browser scenarios. We continue to recommend our Application Insights Client-side JavaScript SDK for the near future.
Sending your telemetry
There are two ways to send your data to Azure Monitor (or any vendor):
- Direct Exporter
- Via an Agent
A direct exporter sends telemetry in-process (from the application’s code) directly to Azure Monitor’s ingestion endpoint. The main advantage of this approach is simplicity.
Azure Monitor’s immediate focus is on direct exporters for our supported languages. This is due to our roots in the developer productivity space and due to the complexities integrating OTLP into our existing Azure Monitor Agent.
We plan to support an agent-based approach in the future, and work is underway to add OTLP support to Azure Monitor Agent. Our objective is to provide a path for any OpenTelemetry supported langauge to send to Azure Monitor via OTLP. This will enable customers to observe applications written in languages beyond our supported languages.
*Some customers have begun to use the OpenTelemetry-Collector as an agent alternative even though Microsoft doesn’t officially support “Via an Agent” approach for application monitoring yet. In the meantime, the open-source community has contributed an OpenTelemetry-Collector Azure Monitor Exporter that some customers are using to send data to Azure Monitor Application Insights.
How does this affect me?
If you are an existing Azure Monitor Application Insights customer, we don’t recommend that you migrate to OpenTelemetry yet (except for Java applications where we recommend our OpenTelemetry-based Java 3.X GA offering for everyone which GA’d in November 2020).
Here’s why. The existing Application Insights SDKs are feature-rich, and it’ll take some time before OpenTelemetry matures to feature parity. As the OpenTelemetry-based offering gets closer to feature parity, we’ll provide clear guidance to existing customers using Application Insights SDKs.
If you are developing a new application and want to onboard to Azure Monitor Application Insights with the most feature-rich experience, we recommend using the existing Application Insights SDKs for C# (ASP.NET or ASP.NET Core) and Javascript (Node.js) or Python-OpenCensus. The features only in existing Application Insights SDKs as of the end of 2022, include:
- Ability to create custom events (outside the context of a span)
- Enhanced data transport security using AAD Authentication
- Additional diagnostic tools (.NET only): Profiler and Snapshot Debugger
If you are developing a new application and are willing to sacrifice the features noted above to use OpenTelemetry, try it out and let us know what you think. We will continue to add new features regularly. You can see our progress for C#, JavaScript (Node.js), and Python in open-source repos.
Scenarios that may sway you toward OpenTelemetry sooner-than-later include sending telemetry to Azure Monitor + another vendor simultaneously, collecting and converging existing instrumentation protocols, or leveraging features available in the OpenTelemetry-Collector. For example, some developers may want to use a local tool such as Jaeger or Zipkin to test on a dev box and configure their instrumentation to use Azure Monitor when running in a production environment.
Looking Forward
We are committed to OpenTelemetry as the future of telemetry instrumentation, and over time we expect Azure services and platform to adopt it more broadly. This journey will take time, and we are committed to bringing you, our customers, along on the journey with us.
Learn More
- The Azure Monitor OpenTelemetry FAQ provides additional context and links out to more supporting content.
- Sources that explain the three pillars of observability in detail include the OpenTelemetry community website and Distributed Systems Observability by Cindy Sridharan.