Building AI Agents from Zero to Production

Most agent demos stop at "it answered my question." Production doesn't. The gap between a notebook that calls an LLM and a governed, observable, multi-agent system your organisation can actually depend on is where real engineering happens, evaluation, deployment, data sovereignty, tool governance, and cross-team interoperability.

Microsoft's open-source course Building AI Agents from Zero to Production walks that entire arc in seven lessons, using one realistic use case and the Microsoft Agent Framework (MAF) plus Microsoft Foundry. This post is a developer-focused tour of what it teaches, the architecture decisions behind each stage, and the code patterns that matter when you move from prototype to production.

Who this is for

• AI engineers building their first or first production, agent system.
• Backend and full-stack developers integrating agents into real applications and CI/CD.
• Cloud architects who need data sovereignty, private networking, and governance around agent workloads.
• Technical leads deciding how to standardise tools and orchestration across multiple teams.

The samples are Python 3.12+, served through Microsoft Foundry using GPT-5 series models (for example gpt-5.1). Lesson 4 adds a TypeScript/React frontend. You will want an Azure subscription and the Azure CLI.

The AI Agent Development Lifecycle

The course is organised around a lifecycle rather than a feature list. Each lesson is a stage, and each stage assumes the previous one is solved:

The thread running through all seven is a single scenario: a Developer Onboarding agent system that helps a new hire find the right teammates, get a sensible first task, and pull learning resources and code snippets. It is deliberately mundane, which is exactly why it exposes the production concerns that flashy demos hide.

Lesson 1 — Agent Design: three components, one graph

The course defines an agent by three parts: an LLM for reasoning, tools to act, and memory to retain context. The design work is context engineering — making sure the right information reaches the model at the right moment, no more and no less.

Rather than one monolithic assistant, the onboarding system is split into specialists coordinated by a triage agent using handoff orchestration:

Architecturally this is a directed graph: User → Triage → [Employee, Learning, Coding]. Splitting responsibilities early pays off later, each agent gets a tightly scoped prompt (less hallucination), can be evaluated independently, and can be upgraded without touching its peers.

Lesson 2 — Development: standalone agents with MAF

Here the design becomes code. Each specialist is a small, independently runnable service built with the Microsoft Agent Framework, authenticated to Foundry with your Azure CLI login. Setup is deliberately boring:

az login
az account set --subscription "<your-subscription-id>"
cp .env.example .env
# Fill FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL (e.g. gpt-5.1)

# Create the employee-directory vector store once; note the printed VECTOR_STORE_ID
python lesson-2-agent-development/setup_vector_store.py

# Start an agent — serves on http://localhost:8090
python lesson-2-agent-development/employee-search-agent.py

The FoundryChatClient auto-reads any FOUNDRY_-prefixed environment variables and uses AzureCliCredential, so there are no keys in code. The lesson ships six samples, each on its own port, so you can chat with them individually in the local DevUI before wiring them together:

Why this matters: keeping each agent as its own process with its own port is a testability decision, not an accident. You can smoke-test one specialist in isolation, then compose them in agent-orchestration.py.

Lesson 3 — Evaluation: you can't unit-test a probability distribution

This is the lesson that separates a demo from a product. Agents are non-deterministic, so traditional assertions don't fit. The course uses three complementary layers:

• Observability / tracing — always on, via OpenTelemetry to Application Insights.
• Smoke tests — fast, run on every deploy.
• Evaluations — deeper, model-based scoring run on-demand or nightly.

Turning on tracing is a single call:

from agent_framework.foundry import FoundryChatClient

client = FoundryChatClient()
client.configure_azure_monitor()   # export traces + metrics to Application Insights

For quality it uses Foundry's built-in "LLM-as-a-judge" evaluators against real persisted responses (identified by response_id), not freshly regenerated ones:

The judge model is set independently via AZURE_AI_MODEL_DEPLOYMENT_NAME, so you can evaluate a cheap production model with a stronger one. The run prints a report_url that deep-links into the Foundry portal.

Lesson 4 — Deployment: a hosted agent, a UI, and a CI gate

Now the agent becomes a managed service. It is deployed as a Foundry Hosted Agent a Microsoft-managed execution environment and fronted by an OpenAI ChatKit React UI talking to a FastAPI backend:

ChatKit React (3000) → FastAPI backend (8001) → Foundry Hosted Agent → tools

Building the agent is declarative attach tools, name it, serve it:

agent = client.as_agent(
    name="DevOnboardingAgent",
    instructions="...",
    tools=[file_search_tool, learn_mcp_tool],
)
# served with: from_agent_framework(agent).run()

The recommended deploy path is the Azure Developer CLI:

cd hosted-agent
azd auth login
azd agent deploy

The genuinely production-minded part is the smoke test as a post-deploy CI gate. Six cases cover reachability, each scenario, off-topic prompt adherence, and multi-turn threading (verifying state via previous_response_id). The GitHub Action runs them against the freshly deployed agent:

export FOUNDRY_TOKEN=$(az account get-access-token \
  --resource https://ai.azure.com/ --query accessToken -o tsv)

python runner.py \
  --project-endpoint "https://<account>.services.ai.azure.com/api/projects/<project>" \
  --agent-name dev-onboarding \
  --tests-file tests/smoke-tests.json

Pitfall to remember: the token audience must be https://ai.azure.com/. A cognitiveservices.azure.com token is rejected by the Responses API — a mistake that costs many engineers an afternoon.

Lesson 5 — Production: separating where an agent runs from where its data lives

The pivotal concept for enterprise readiness is the distinction between a Hosted Agent (compute, scaling, identity) and a Capability Host (where conversation history, files, and embeddings actually reside):

"Basic" setup uses Microsoft-managed storage and is perfect for getting started. "Standard" setup redirects each data plane to your own Azure resources through a project-level capability host, this is how you keep customer data in your tenant, inside your network boundary:

PUT .../accounts/{account}/projects/{project}/capabilityHosts/{name}?api-version=2025-06-01
{
  "properties": {
    "capabilityHostKind": "Agents",
    "threadStorageConnections": ["my-cosmosdb-connection"],
    "vectorStoreConnections":  ["my-ai-search-connection"],
    "storageConnections":      ["my-storage-connection"]
  }
}

Operational constraints worth internalising before you provision: there is one capability host per scope (a second attempt returns 409 Conflict), configuration is immutable (delete and recreate to change it), deletion is destructive, and the account-level host must exist before the project-level one.

Lesson 6 — Toolbox: govern tools once, reuse everywhere

Left unchecked, every team re-implements the same tools, scatters credentials, and loses governance visibility. The Microsoft Foundry Toolbox solves this by exposing a curated, versioned set of tools behind a single MCP-compatible endpoint, with credentials held in Foundry connections rather than agent code.

You build a toolbox version once:

from azure.ai.projects.models import MCPTool, ToolboxSearchPreviewTool, WebSearchTool

toolbox_version = project.toolboxes.create_toolbox_version(
    name="agent-tools",
    description="Web search + an MCP server + tool search",
    tools=[
        WebSearchTool(),
        MCPTool(
            server_label="myserver",
            server_url="https://your-mcp-server.example.com",
            require_approval="never",
            project_connection_id="my-key-auth-connection",  # credentials live in Foundry
        ),
        ToolboxSearchPreviewTool(),
    ],
)

And every agent consumes it through one endpoint, no per-team tool code:

from agent_framework import MCPStreamableHTTPTool

mcp_tool = MCPStreamableHTTPTool(
    name="toolbox",
    url=TOOLBOX_ENDPOINT,   # {project_endpoint}/toolboxes/{name}/mcp?api-version=v1
    http_client=http_client,
    load_prompts=False,
)
agent = chat_client.as_agent(name="my-toolbox-agent", instructions="...", tools=[mcp_tool])

Versioning is blue/green: create a new version, test it on its version-specific endpoint, then promote it to default and every consumer picks it up with zero code changes. A Guardrail (RAI) policy can be applied at the toolbox layer, independent of model-level content filters. Note the toolbox management APIs are currently preview; the portal or VS Code Foundry Toolkit are practical alternatives for creation today.

Lesson 7 — Multi-Agent & A2A: agents as networked peers

The final lesson contrasts two ways agents collaborate:

• Handoff / Workflow — in-process, same codebase, fastest, tightest coupling.
• Agent-to-Agent (A2A) — cross-process over an open protocol, so agents from different teams, orgs, or frameworks interoperate.

A2A gives each agent a discoverable Agent Card at /.well-known/agent-card.json and a task lifecycle (submitted → working → completed/failed). The elegant part: A2AExecutor wraps an existing MAF agent with no changes to that agent's code.

from agent_framework.a2a import A2AExecutor
from a2a.server.apps import A2AStarletteApplication
from a2a.server.tasks import InMemoryTaskStore

agent_card = AgentCard(
    name="Coding Assistant",
    url="http://localhost:9000/",
    version="1.0.0",
    capabilities=AgentCapabilities(streaming=True),
    skills=[AgentSkill(id="generate-code", name="Generate code", tags=["code"])],
)
request_handler = DefaultRequestHandler(
    agent_executor=A2AExecutor(agent),      # wraps your existing MAF agent unchanged
    task_store=InMemoryTaskStore(),
)
app = A2AStarletteApplication(agent_card=agent_card, http_handler=request_handler).build()

Consuming a remote agent then looks exactly like calling a local one:

from agent_framework.a2a import A2AAgent

remote_agent = A2AAgent(name="remote-coding-assistant", url="http://localhost:9000")
result = await remote_agent.run("Write a Python function that reverses a string.")

Because an A2AAgent can be a participant inside a HandoffBuilder workflow, you can mix in-process routing with remote services in the same orchestration. For enterprise use, A2AAgent accepts an auth_interceptor for bearer tokens, and the Agent Card carries security_schemes.

Responsible and secure by design

Production readiness in this course is not just uptime, it is governance:

• Identity over keys — AzureCliCredential and managed identity throughout; no secrets in code.
• Least privilege — CI runners get a scoped Azure AI User role assignment on the specific project.
• Data sovereignty — capability hosts keep conversation history, files, and embeddings in your own Cosmos DB, Storage, and AI Search.
• Tool approval and guardrails — MCP approval_mode and toolbox-level RAI policy gate what agents can do.
• Grounded evaluation — groundedness and tool-utilization scoring catch hallucination and unused-tool behaviour before users do.
• Cost hygiene — the lessons create real Azure resources; delete the resource group when done: az group delete --name <rg> --yes --no-wait.

Key takeaways

• Design as a graph of specialists. Handoff orchestration with tightly scoped agents beats one monolith on reliability and testability.
• One .run() contract, many backends. The Agent Framework keeps orchestration code stable from local dev to hosted production.
• Evaluate continuously. Tracing + smoke tests + model-based evaluators are three layers, not alternatives.
• Separate compute from data. Hosted Agents run the agent; Capability Hosts give you sovereignty — you need both for enterprise.
• Govern tools centrally. A versioned toolbox behind one MCP endpoint kills tool sprawl and credential duplication.
• Open protocols for interop. A2A lets agents cross team, org, and framework boundaries without rewrites.

Get started

Clone the repo (skip the 50+ translations for a faster download) and work through the lessons in order:

git clone --filter=blob:none --sparse https://github.com/microsoft/Building-AI-Agents-From-Zero-To-Production.git
cd Building-AI-Agents-From-Zero-To-Production
git sparse-checkout set --no-cone '/*' '!translations' '!translated_images'

References

• Building AI Agents from Zero to Production — course repo
• Microsoft Agent Framework
• Microsoft Foundry documentation
• Agent-to-Agent (A2A) protocol specification
• a2a-python SDK
• AI Agents for Beginners
• MCP for Beginners
• Microsoft Foundry Discord