Operating AI Agents on Azure: Observability with Azure AI Foundry

skundapura

Microsoft

Apr 29, 2026

1. This blog helps Azure infrastructure and platform teams understand how to operate AI agents reliably in production, moving beyond experimentation and demos. 2. It explains why agent observability is fundamentally different from traditional application monitoring and why logs alone are not enough. 3. It compares no‑code, low‑code, and pro‑code tracing approaches supported by Azure AI Foundry so teams can choose the right operational model. 4. It enables SREs and cloud architects to make informed decisions about governance, reliability, and troubleshooting for agent‑based workloads running on Azure.

Azure AI Foundry is Microsoft’s enterprise platform for building, deploying, and operating AI applications and intelligent agents as first‑class Azure workloads. From an infrastructure perspective, Foundry acts as a control plane that brings together model hosting, agent execution, tooling, evaluation, and observability under a single, governed environment. It integrates natively with Azure services such as Azure Monitor, Application Insights, managed identities, and role‑based access control, allowing AI workloads to be managed with the same rigor as traditional applications. Azure AI Foundry supports both portal‑driven and code‑driven workflows, enabling platform teams to standardize lifecycle management while supporting diverse development styles. This makes Foundry particularly suited for production environments where governance, auditability, and operational consistency are non‑negotiable.

Microsoft Agent Framework (MAF)

Microsoft Agent Framework is an Azure‑native framework designed to build and operate intelligent agents directly within Azure AI Foundry. It treats agents as managed workloads, enabling platform teams to deploy, secure, and monitor them through the Foundry control plane. Tracing is enabled automatically without code changes, capturing agent reasoning, tool calls, and multi‑agent interactions in Application Insights.

From an operational standpoint, MAF offers the strongest alignment with enterprise governance and no‑code observability. SRE teams gain immediate visibility without custom instrumentation, making it the preferred choice where standardization, compliance, and operational simplicity are critical.

Semantic Kernel (SK)

Semantic Kernel provides a structured way to orchestrate LLMs with plugins, planners, and functions in .NET and Python environments. When connected to Azure AI Foundry, it automatically emits telemetry for prompts, responses, function execution, and token usage using Azure inference connectors.

Operationally, Semantic Kernel sits in the low‑code category. It offers strong observability with minimal configuration while allowing teams to retain code‑level control. This makes it suitable for teams that need transparency and structure without fully managed agent hosting.

LangChain

LangChain is a widely adopted open‑source framework for building agent workflows and retrieval‑augmented applications. Azure AI Foundry integrates with LangChain using OpenTelemetry‑based tracers, allowing chain execution, tool calls, and model interactions to be captured in Application Insights.

From a platform perspective, LangChain tracing requires explicit configuration but delivers consistent observability once enabled. It fits organizations that standardize on OSS tooling while still needing enterprise‑grade monitoring and debugging through Azure Monitor.

LangGraph

LangGraph extends LangChain by enabling graph‑based, stateful agent orchestration with conditional routing. Azure AI Foundry traces LangGraph workflows using the same OpenTelemetry integration, capturing node transitions, tool usage, and execution paths.

This model is operationally valuable for complex workflows but introduces higher configuration overhead. It suits advanced teams that need deep insight into non‑linear agent execution while maintaining centralized observability.

OpenAI Agent SDK

The OpenAI Agent SDK provides low‑level control for building custom agent runtimes. Unlike other frameworks, it does not emit telemetry by default. To integrate with Azure AI Foundry, teams must manually instrument OpenTelemetry spans and export them to Application Insights.

This approach offers maximum flexibility but shifts observability responsibility to the engineering team. It is best suited for specialized scenarios where bespoke agent execution outweighs operational simplicity.

Log Tracing in Azure AI Foundry

Log tracing in Azure AI Foundry addresses a core operational gap in agent‑based systems that traditional logging cannot solve. AI agents are non‑linear by nature, often invoking multiple tools, branching based on intermediate reasoning, and coordinating with other agents. Azure AI Foundry uses OpenTelemetry and Azure Monitor Application Insights to capture detailed execution traces that show how requests flow through agents, models, and tools. These traces include timing, errors, token usage, and reasoning paths, enabling teams to debug failures and performance issues with precision. By exposing this telemetry through the Foundry portal and Azure Monitor, platform teams gain a unified view of agent behavior that supports troubleshooting, governance reviews, and production reliability at scale.

Depth of Agent Visibility

Capability	Microsoft Agent Framework (MAF)	Semantic Kernel (SK)	LangChain	LangGraph	OpenAI Agent SDK
Agent decision flow visibility	You can see the full step‑by‑step path of what the agent decided to do, including why it chose a particular action, which step came first, and how it reached the final answer. This is visible automatically in Azure AI Foundry without adding code.	You can see the main execution path of the agent, including which skill or planner was used, but fine‑grained reasoning steps may be abstracted unless you add extra logging.	You usually see the final chain execution and intermediate steps, but the “why” behind decisions is not always clear unless you manually add tracing.	You can see each node in the graph and which path was taken at runtime, making it easier to understand branching decisions.	You mostly see inputs and outputs. Internal reasoning and decision paths are not visible unless you explicitly create and manage custom spans.
Tool invocation tracing	Every tool call is captured automatically, showing which tool was called, with what input, how long it took, and what result it returned, all correlated to the agent run.	Tool and function calls are tracked, but details depend on how plugins are implemented and may need configuration to capture inputs and outputs clearly.	Tool calls are visible when tracing is enabled, but they may appear as generic steps unless you customize the tracer.	Tool usage is clearly tied to graph nodes, making it easier to identify which step called which tool.	Tool calls are only visible if you explicitly instrument them using OpenTelemetry or custom logging.
Multi‑agent support visibility	When multiple agents work together, you can follow a single request as it moves from one agent to another, clearly seeing which agent handled which part of the task.	Multi‑agent flows are possible, but visibility across agents is limited and usually requires manual correlation.	Multi‑agent patterns are supported conceptually, but tracing across agents is not automatic and can be difficult to follow.	Designed for complex workflows, LangGraph allows visibility across multiple agents or components, but still requires configuration.	No built‑in way to trace requests across multiple agents unless you design and implement it yourself.
Automatic span hierarchy	Traces are automatically structured in a parent‑child hierarchy, such as request → agent → decision → tool → model call, without manual setup.	Most spans are structured correctly by default, but complex scenarios may require tuning.	Span hierarchy is available when using the provided Azure tracer, but understanding relationships may require experience.	Hierarchy is preserved across graph nodes, making execution order clearer.	No automatic hierarchy; you must manually create and link spans to understand execution flow.

Tracing Integrations Comparison

Aspect	Microsoft Agent Framework (MAF)	Semantic Kernel (SK)	LangChain	LangGraph	OpenAI Agent SDK
Integration type with Azure AI Foundry	Tracing is built directly into Azure AI Foundry when agents are created or managed in the Foundry portal. The platform automatically enables tracing at the server side.	Tracing works through Azure AI inference connectors that integrate Semantic Kernel with Foundry. Telemetry is emitted automatically once configured.	Tracing is enabled using an OpenTelemetry‑based tracer provided by the langchain-azure-ai package and connected to Foundry.	Uses the same OpenTelemetry tracer as LangChain, extended to support graph‑based execution paths.	No built‑in integration. You must manually add OpenTelemetry instrumentation and export traces to Application Insights.
Code changes required	No code changes are required if you use Foundry‑managed agents. Tracing works out of the box.	Minimal code or configuration is required to attach the Azure AI inference connector and enable telemetry.	Low‑code setup. You must add the tracer dependency and configure environment variables.	Low‑code setup similar to LangChain, plus configuration to trace graph nodes.	Pro‑code. You must explicitly write tracing logic, define spans, and manage exporters yourself.
What tracing actually captures	You can see agent executions, internal decision steps, tool and MCP calls, model requests, token usage, latency, errors, and multi‑agent interactions in one correlated trace.	You can see prompts, responses, function or plugin calls, token usage, latency, and planner execution, depending on configuration.	You see chain execution steps, tool calls, and LLM interactions, but understanding deeper reasoning may require reading the chain logic.	You see each graph node, execution order, conditional branching, and related tool calls, which helps debug complex workflows.	By default you only see inputs and outputs. Detailed decision steps or tool usage appear only if you manually instrument them.
Multi‑agent tracing support	Fully supported. You can follow a single request as it moves between multiple agents inside one trace.	Partially supported. Multi‑agent flows are possible but cross‑agent visibility may need manual correlation.	Limited. Multi‑agent patterns exist, but traces across agents are not clearly linked by default.	Better support than LangChain due to graph structure, but still requires deliberate configuration.	Not supported by default. You must design and implement cross‑agent tracing manually.
Who should use this approach	Platform and infrastructure teams that want enterprise‑grade observability with zero instrumentation effort.	Teams building structured AI workflows in .NET or Python that want good visibility with minimal effort.	Open‑source Python teams that want Azure‑grade observability without leaving LangChain.	Teams building complex, stateful, or branching agent workflows that need visibility into execution paths.	Advanced teams that need full control and are comfortable managing observability pipelines themselves.