Microsoft is excited to embrace OpenTelemetry as the future of telemetry instrumentation. You, our customers, have asked for vendor-neutral instrumentation, and we're delighted to partner with the OpenTelemetry community to create consistent APIs/SDKs across languages.
Microsoft worked together with project stakeholders from two previously popular open-source telemetry projects, OpenCensus and OpenTracing, to help create a single project -- OpenTelemetry. OpenTelemetry includes contributions from all major cloud and Application Performance Management (APM) vendors and is housed by the Cloud Native Computing Foundation (CNCF) of which Microsoft is a Platinum Member.
This blog focuses on OpenTelemetry as it relates to Azure Monitor Application Insights. In the future, we hope to provide more context on how OpenTelemetry is impacting other parts of Azure!
Concepts and Progress
Telemetry, the data collected to observe your application, can be broken into three types or "pillars":
Initially the OpenTelemetry community took on Distributed Traces. Metrics and Logs are still in progress. A complete observability story includes all three pillars. Azure Monitor is working on an OpenTelemetry-based supported offering that includes Distributed Traces only with plans to add the other pillars as they mature in the OpenTelemetry community.
In the following sections, we'll cover some telemetry collection basics.
Instrumenting your application
At a basic level, "instrumenting” is simply enabling an application to capture telemetry.
There are two methods to instrument your application:
Automatic Instrumentation (Auto-Instrumentation)
“Manual” does not mean you’ll be required to write complex code to define spans for distributed traces (though it remains an option). A rich and growing set of instrumentation libraries maintained by OpenTelemetry contributors will enable you to effortlessly capture telemetry signals across common frameworks and libraries. A subset of OpenTelemetry Instrumentation Libraries will be supported by Azure Monitor, informed by customer feedback. Additionally, we are working to instrument the most popular Azure Service SDKs to emit OpenTelemetry Protocol (OTLP) and optimizing them for Azure Monitor.
Auto-instrumentation is enabling telemetry collection through configuration without touching the application's code. While more convenient, it tends to be less configurable and it’s not available in all languages. Azure Monitor’s OpenTelemetry-based auto-instrumentation offering consists of the Java 3.X GA offering, and we continue to invest in it informed by customer feedback. The OpenTelemetry community is also experimenting with C# and Python auto-instrumentation, but Azure Monitor is focused on creating a simple and effective manual instrumentation story in the near-term.
Sending your telemetry
There are two ways to send your data to Azure Monitor (or any vendor):
Via an Agent
A direct exporter sends telemetry in-process (from the application’s code) directly to Azure Monitor’s ingestion endpoint. The main advantage of this approach is simplicity.
Azure Monitor’s immediate focus is on direct exportersfor our supported languages. This is due to our roots in the developer productivity space and due to the complexities integrating OTLP into our existing Azure Monitor Agent.
We plan to support an agent-based approach in the future, though the details and timeline aren't available yet. Our objective is to provide a path for any OpenTelemetry supported langauge to send to Azure Monitor via OTLP. This will enable customers to observe applications written in languages beyond our supported languages.
*Some customers have begun to use the OpenTelemetry-Collector as an agent alternative even though Microsoft doesn’t officially support “Via an Agent” approach for application monitoring yet. In the meantime, the open source community has contributed an OpenTelemetry-Collector Azure Monitor Exporter that some customers are using to send data to Azure Monitor Application Insights.
How does this affect me?
If you are an existing Azure Monitor Application Insights customer, we don’t recommend that you migrate to OpenTelemetry yet (except for Java applications where we recommend our OpenTelemetry-based Java 3.X GA offering for everyone which GA’d in November 2020).
Here’s why. The existing Application Insights SDKs are feature-rich, and it’ll be quite a while until OpenTelemetry matures to feature parity. As the OpenTelemetry-based offering gets closer to feature parity, we’ll provide clear guidance to existing customers using Application Insights SDKs.
Ability to create custom metrics and events
Ability to pass metrics unaffected by sampling
Enhanced data transport security using AAD Authentication
Enhanced data transport reliability using offline disk storage with retry logic
Additional diagnostic tools (.NET only): Profiler and Snapshot Debugger
Scenarios that may sway you toward OpenTelemetry sooner-than-later include sending telemetry to Azure Monitor + another vendor simultaneously, collecting and converging existing instrumentation protocols, or leveraging features available in the OpenTelemetry-Collector. For example, some developers may want to use a local tool such as Jaeger or Zipkin to test on a dev box and configure their instrumentation to use Azure Monitor when running in a production environment.
We are committed to OpenTelemetry as the future of telemetry instrumentation, and over time we expect Azure services and platform to adopt it more broadly. This journey will take time, and we are committed to bringing you, our customers, along on the journey with us.