A companion piece to Infrastructure as Code for AI: Building and Deploying Microsoft Hosted Agents with Terraform.
Just announced — source-code deploy (preview). Foundry has just added a second Hosted Agent deploy path alongside the container path this post covers. Instead of a container image, you upload a
.zipof your source plus arequirements.txt(Python 3.13 / 3.14) or a.csproj(.NET 10), and the Agent Service either builds dependencies for you (remote_build) or runs a prebuilt bundle (bundled). The version definition usescode_configurationinstead ofcontainer_configuration— the two are mutually exclusive on a given version. Versioning is content-addressable on the zip's SHA-256, so the dedup behaviour described below still applies. Required roles shift slightly: deploying the agent needs Foundry Project Manager at project scope, and the platform-assigned agent identity gets Foundry User (both handled automatically byazdand the Foundry VS Code Toolkit). The DevOps loop in this post — immutable versions, eval gating, manifest-driven promotion, traffic-split canary, per-version observability — transfers directly; only the build-and-push stage changes (no Dockerfile, no ACR forremote_build). The container path covered here remains fully supported and is still the right choice if you need custom base images, system packages, or non-Python/.NET runtimes. Full details: Deploy a hosted agent from source code (preview).
What this post assumes. It describes recommended enterprise DevOps patterns on top of Microsoft Foundry Hosted Agents. Some patterns — evaluation gating, traffic-based rollout, manifest-driven promotion — are best practices and may not be enforced by the platform itself. Hosted Agents and several related capabilities (A2A, certain deployment and routing controls) are in preview and may evolve.
TL;DR
- Terraform provisions the platform: Foundry account, project, model deployment, ACR, App Insights, RBAC.
- DevOps pipelines ship agent versions, not source branches — the deploy artifact is a container image digest plus an immutable version spec.
- Evaluation should be treated as a release gate, not a dashboard. Quality regressions should fail the build the same way unit-test failures do.
- Traffic split between versions is the rollout and rollback primitive. Rollback typically avoids rebuilding or redeploying artifacts.
- Observability is sliced per version — during canary, two versions serve simultaneously and aggregate metrics lie.
The Delivery Pipeline at a Glance
Terraform ───► Foundry project (AIServices) + model deployment + ACR + App Insights
│
PR opened ▼
└─► docker build ───► push to ACR ───► capture image digest
│
▼
Foundry SDK: create agent version
(image digest + cpu/mem + env + protocols)
│
▼
Evaluation gate ────► fail → stop
│
▼ pass
Promote via manifest → staging → prod
│
▼
Traffic-split canary (0% → 10% → 100%)
│
▼
App Insights: per-version latency, cost,
sampled quality, sandbox sizing
Infrastructure as Code gets the platform stood up. It does not, on its own, ship an agent. The gap between terraform apply succeeding and a customer-facing agent reliably serving requests in production is where DevOps lives — and for Microsoft Hosted Agents on Microsoft Foundry, that gap has its own shape.
A Hosted Agent is not a prompt and a tool list. It is your own code, packaged as a container image, pushed to Azure Container Registry, and deployed to a Foundry project. The Foundry Agent Service pulls the image, provisions an isolated execution environment per agent session, assigns the agent its own dedicated Microsoft Entra ID (agent identity), and exposes a dedicated endpoint. An agent supports up to four protocols, any of which can be combined in a single deployment:
- Responses (
.../protocols/openai/responses) — OpenAI-compatible chat-style API. Implemented in the container. - Invocations (
.../protocols/invocations) — arbitrary JSON in / arbitrary JSON out for webhook receivers and non-conversational workloads. Implemented in the container. - A2A (
.../protocols/a2a, preview) — the open Agent2Agent protocol for agent-to-agent delegation across frameworks and vendors. Surfaced on its own endpoint path by the platform. - Activity — the Teams / M365 channel protocol. The platform bridges Responses to Activity automatically when an agent is published to a Microsoft 365 channel.
Microsoft manages the runtime, scaling, session state, and lifecycle. You ship the image and the version definition.
Important — Foundry version compatibility. Hosted Agents are supported on the new Microsoft Foundry project resource model (
azurerm_cognitive_account_projectunder a Cognitive Services account ofkind = "AIServices"). The older Azure AI Foundry Hub model (azurerm_ai_foundry/azurerm_ai_foundry_project,kind = "Hub") — the Azure ML–derived workspace surface — does not expose Hosted Agent capabilities. They are two distinct Azure resource types with different APIs. Everything in this post assumes the new Foundry project.
That shape drives three things every DevOps loop for Hosted Agents has to handle:
- The deploy artifact is a container image plus an immutable agent version. A version snapshots the image digest, CPU/memory, environment variables, and protocol configuration. To change anything, you create a new version. The platform supports weighted traffic between versions, which is your blue/green and canary primitive.
- The agent identity is created for you, per agent. You don't pick one or wire managed-identity references manually. Each agent is assigned a dedicated Microsoft Entra ID (agent identity) at deploy time; RBAC to downstream resources is granted to that identity.
- Quality is non-deterministic. Two
terraform applyruns against the same configuration produce identical resources. Two agent runs against the same input can produce different outputs. Your pipeline has to gate on evaluation, not only on tests passing and HTTP 200s.
This post lays out an end-to-end DevOps loop on top of that shape: how to structure the repository, what runs in CI versus CD, how to gate releases on evaluation, how to promote across environments, how to use version traffic split for safe rollouts and instant rollback, and what observability is worth wiring beyond the defaults.
A Quick Tour of Microsoft Foundry
If you've spent more time in Azure OpenAI or AI Studio than in Foundry, a short orientation helps before the DevOps patterns make sense.
Microsoft Foundry is Microsoft's unified platform for building, evaluating, deploying, and operating AI applications and agents. It consolidates what used to be spread across Azure OpenAI, Azure AI Studio, and the AI Hub model into a single resource and a single portal at ai.azure.com. Three pieces are worth knowing up front.
The resource model
Foundry is built on two Azure resources:
- Foundry account — an
azurerm_cognitive_accountwithkind = "AIServices",project_management_enabled = true, acustom_subdomain_name, and a managed identity. This is the top-level container: it holds your model deployments (Azure OpenAI and the broader Foundry model catalog), connections to backing services, and the Foundry-managed Toolbox MCP endpoint. - Foundry project — an
azurerm_cognitive_account_projectunder that account. A project is the scope for agents, evaluations, conversation history, indexes, and per-app connections. One project per app or per environment is the usual shape.
This is the new Foundry model — and it is the only model that supports Hosted Agents. The older Azure AI Foundry Hub (azurerm_ai_foundry + azurerm_ai_foundry_project, kind = "Hub") is a separate Azure ML–derived workspace and cannot host Hosted Agents. The two surfaces look superficially similar in the portal but are distinct Azure resource types with different APIs and feature sets. If a tutorial, sample, or piece of Terraform you find online creates an azurerm_ai_foundry Hub, it is targeting the classic surface and the Hosted Agents APIs (/agents, agent versions, traffic split, dedicated endpoints) will not be available against it. To use Hosted Agents you must provision a new Foundry account + project as described above. There is no in-place upgrade from a Hub.
What Foundry gives you
A Foundry project is more than a container. Out of the box it provides:
- A model catalog and deployment surface — Azure OpenAI models (GPT-4.1, GPT-4o, o-series, embeddings), plus open and partner models, all deployed and invoked through the same project endpoint with the same auth model.
- Two agent execution modes — prompt-based agents (defined entirely by instructions + tool configuration in the portal, suitable for conversational assistants) and Hosted Agents (your own containerized code, the subject of this post).
- A managed Toolbox — a project-level MCP endpoint that exposes Foundry-curated tools (Code Interpreter, Web Search, Azure AI Search, OpenAPI, custom MCP, A2A) with consolidated auth. Hosted Agent code connects to the Toolbox using standard MCP client libraries.
- First-class evaluation — datasets, graders (similarity, LLM-as-judge, safety, groundedness), and evaluation runs as a built-in concept, not a bolt-on.
- Built-in tracing — OpenTelemetry traces from agents land in a linked Application Insights resource automatically. No manual instrumentation needed to get the basics.
- Per-agent identity — when you deploy a Hosted Agent, the platform creates a dedicated Microsoft Entra ID (agent identity) for it and gives it a dedicated endpoint. RBAC to downstream resources is granted to that identity.
How the pieces line up for Hosted Agents
For the rest of this post, the mental model is:
Resource group
└── Foundry account (Cognitive Services, kind=AIServices)
├── Model deployments (e.g. gpt-4.1)
└── Foundry project
├── Hosted Agent: customer-support
│ ├── Version v1 (image digest A, 100% traffic)
│ └── Version v2 (image digest B, 0% traffic — canary)
├── Hosted Agent: webhook-handler
├── Evaluations
├── Connections (ACR, AI Search, Key Vault…)
└── Toolbox (MCP)
Terraform provisions the account, project, model deployments, ACR, App Insights, and RBAC. Hosted Agents — images, versions, traffic weights — are managed through azd or the Foundry SDK. That boundary is what the rest of this post automates.
The minimal Terraform shape
For Hosted Agents you need the new-model shape instead. The skeleton below is the minimum that lets you deploy a Hosted Agent on top of it — storage, Key Vault, monitoring, networking, and OIDC for CI live alongside for more details see Infrastructure as Code for AI: Building and Deploying Microsoft Hosted Agents with Terraform | Microsoft Community Hub.
# Foundry account (new model — required for Hosted Agents)
resource "azurerm_cognitive_account" "foundry" {
name = "ai-${local.name}"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
kind = "AIServices"
sku_name = "S0"
project_management_enabled = true
custom_subdomain_name = "ai-${local.name}" # required for AAD auth
identity {
type = "SystemAssigned"
}
}
# Model deployment the agent will call
resource "azurerm_cognitive_deployment" "gpt" {
name = "gpt-4.1" # stable name — agents pin to this
cognitive_account_id = azurerm_cognitive_account.foundry.id
model {
format = "OpenAI"
name = "gpt-4.1"
version = "2025-04-14"
}
sku {
name = "GlobalStandard"
capacity = 10
}
}
# Foundry project — the scope for Hosted Agents, evals, conversations
resource "azurerm_cognitive_account_project" "main" {
name = "proj-${local.name}"
cognitive_account_id = azurerm_cognitive_account.foundry.id
location = azurerm_resource_group.main.location
identity {
type = "SystemAssigned"
}
}
# Container registry the agent image is pushed to and pulled from
resource "azurerm_container_registry" "acr" {
name = "acr${replace(local.name, "-", "")}"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
sku = "Standard"
admin_enabled = false # use RBAC, not admin user
}
# The project's managed identity needs to pull the agent image
resource "azurerm_role_assignment" "project_acr_pull" {
scope = azurerm_container_registry.acr.id
role_definition_name = "AcrPull" # use Container Registry Repository Reader if the ACR has ABAC enabled
principal_id = azurerm_cognitive_account_project.main.identity[0].principal_id
}
A few things worth calling out:
kind = "AIServices"+project_management_enabled = true+custom_subdomain_nameare what make this a new-model Foundry account. Omitproject_management_enabledandazurerm_cognitive_account_projectwill not provision; omitcustom_subdomain_nameand you lose the Foundry endpoint shape that Entra-authenticated access depends on.azurerm_cognitive_account_projectis the new-Foundry project resource. Do not useazurerm_ai_foundry_project— that targets the Hub model and does not host agents.- Keep the model deployment
namestable. Agent code (and youragent.yaml) pins to the deployment name, not the model version. Changing the version is safe; changing the name forces a new agent version. - The project MI needs ACR pull, not push. CI pushes the image (via its own identity); the platform pulls it on the project's behalf when the agent runs.
- ABAC-enabled ACR is supported but requires
--source-acr-auth-id [caller]onaz acr buildin your CI script — a common gotcha. - A note on the provider. Everything above uses the hashicorp/azurerm provider. Foundry's surface evolves quickly, and you will occasionally hit a property or child resource that AzureRM hasn't caught up with yet — project connections, capability hosts, and some newer agent-related fields are common examples. When that happens, reach for azure/azapi: use azapi_update_resource to patch a missing property on an AzureRM-owned resource, and azapi_resource for resources AzureRM doesn't model at all. Keep AzureRM as the default and use AzAPI as a targeted gap-filler, so you don't fork ownership of mainstream resources.
The Hosted Agent Delivery Loop
A working delivery loop has five stages. Each maps to a specific artifact, a specific tool, and a specific failure mode.
| Stage | Artifact | Tool | Primary failure mode |
|---|---|---|---|
| Infra provisioning | Terraform state | terraform apply | Quota, RBAC propagation, ACR not reachable |
| Image build & push | OCI image in ACR (ACR must remain publicly reachable today) | docker build / az acr build | Image too large, base image CVEs |
| Agent version create | Immutable version (image digest + config) | azd or Foundry SDK | Bad env var, wrong protocol declared |
| Evaluation | Eval dataset + grader | Foundry evaluators | Quality / safety regression |
| Traffic shift & observe | Version weights, App Insights traces | Foundry SDK + Azure Monitor | Silent quality decay, sandbox over/under-sizing |
The first stage is where the prior post left off. The remaining four are this post.
Infra provisioning assumes the standard pattern: terraform plan runs on every PR as a review gate (posted as a PR comment) and terraform apply runs only on merge to the environment branch. Everything below assumes the platform is already applied.
Repository Shape
A repository that supports the loop end-to-end looks roughly like this:
agent-platform/
├── infra/ # Terraform from the prior post (AIServices + project)
│ ├── modules/foundry-project/
│ └── environments/
│ ├── dev.tfvars
│ ├── staging.tfvars
│ └── prod.tfvars
├── agents/
│ ├── customer-support/
│ │ ├── Dockerfile
│ │ ├── src/ # Agent code (Python or C#)
│ │ ├── agent.yaml # Version spec: image, cpu/memory, protocols, env
│ │ ├── evals/
│ │ │ ├── dataset.jsonl
│ │ │ └── graders.yaml
│ │ └── README.md
│ └── webhook-handler/
│ └── ...
├── scripts/
│ ├── deploy_agent_version.py # Build → push → create version → optional weight shift
│ ├── run_evals.py
│ └── promote_version.py # Shifts traffic between versions
└── .github/workflows/
├── infra.yml # Terraform plan/apply
├── agent-pr.yml # Build, push to ACR, deploy candidate version, run evals
└── agent-release.yml # Promote a tested version to staging / prod
Two deliberate choices. First, infrastructure and agents live in the same repo but in separate top-level directories with separate pipelines. They have different cadences and different reviewers. Second, each agent is its own folder with its own Dockerfile, code, version spec, and eval suite. A single PR touches one agent's directory cleanly; a code-review diff stays focused.
The Agent Version as the Deploy Unit
A Hosted Agent is deployed as a version. A version is immutable — once created it captures:
- the container image digest (not just the tag — the digest, so it cannot drift),
- CPU and memory allocation for the per-session sandbox (e.g. 1 vCPU / 2 GiB),
- the container protocols the image implements —
responses,invocations, or both, - environment variables passed to the container at runtime,
- any other version-scoped configuration (e.g. base model deployment name).
The container's
container_protocol_versionsonly declaresresponsesand/orinvocations— the two protocols the container itself implements. A2A (preview) is surfaced by the platform on its own endpoint path, and Activity is bridged from Responses automatically when the agent is published to a Microsoft 365 channel. Under the hood, agent versions run on Azure Container Apps with VM-isolated sandboxes, which is also why you may see the term revision in some Container Apps–surfaced APIs and limits — a Hosted Agent version corresponds to one such revision.
To change any of those, you create a new version. The platform keeps the old one and shifts traffic between them by weight. This is the primitive you use for canary rollouts and for rollback — both reduce to a traffic-weight change, not a redeploy.
An agent.yaml per agent makes the version reproducible from source:
# agents/customer-support/agent.yaml
name: customer-support
container:
image: ${ACR_LOGIN_SERVER}/customer-support # digest resolved at deploy time
cpu: 1
memory: 2Gi
protocols: # container_protocol_versions
- responses
# add `invocations` here if the container also handles webhook-style payloads
env:
# The platform automatically injects FOUNDRY_PROJECT_ENDPOINT,
# AZURE_AI_MODEL_DEPLOYMENT_NAME, and APPLICATIONINSIGHTS_CONNECTION_STRING
# — you only set what's specific to your agent.
LOG_LEVEL: info
metadata:
owner: support-team
source_commit: ${GITHUB_SHA}
scripts/deploy_agent_version.py is the executable form of this spec. Its job per agent is:
- Build the container image (
docker buildlocally, oraz acr buildserver-side for ABAC ACRs). - Push to ACR and capture the resulting image digest — not the
:latesttag. - Resolve environment variables from the target environment's config.
- Call the Foundry SDK to create a new agent version pinned to that digest.
- Emit a
deployment-manifest.jsoncontaining the agent name, version ID, image digest, source commit SHA, and the eval dataset hash used.
One gotcha: the platform deduplicates. A create version call with no change to the version parameters (same image digest, same env, same CPU/memory, same protocols) will not produce a new version object. Write the script to treat "no new version returned" as success and reuse the existing version ID in the manifest, not as a failure to retry.
That manifest is the cross-pipeline contract. PR pipelines produce one. Promotion pipelines consume one. Rollback consumes a previous one.
Evaluation as a Release Gate
Foundry ships evaluators (datasets, graders, evaluation runs) as a first-class platform feature. Whether to block a release on their results is a team decision, not a platform mandate — but it is the recommended pattern for any agent serving real users. A pipeline that promotes an agent because the image built, the container started, and the version was created with HTTP 200 will eventually ship a regression that an integration test cannot catch. Treat the eval suite the way you treat unit tests: failures stop the pipeline.
A minimal but honest evaluation setup has three pieces.
A reference dataset. Twenty to fifty representative scenarios is enough to start. Each row is an input plus either a reference answer, a set of must-include facts, or a rubric. Store as JSONL alongside the agent:
{"id":"refund-1","input":"How do I get a refund for order 12345?","must_include":["return window","14 days","original payment method"]}
{"id":"escalate-1","input":"This is the third time my package is late.","rubric":"Agent should acknowledge, apologize, offer escalation, not promise compensation."}
Graders. Foundry's evaluators library ships templates — exact match, similarity, LLM-as-judge for rubric scoring, and built-in safety and groundedness graders. Pick what matches your dataset shape. LLM-as-judge is the workhorse for open-ended responses; pin its model deployment explicitly so the grader itself does not drift between runs.
Thresholds. Decide what "passing" means before the first run. A common pattern:
- Hard floor on safety / groundedness — any regression fails the build.
- Relative threshold on quality — no more than X% drop versus the last known-good version.
- Absolute floor on must-include coverage — for example ≥ 90%.
Wire it into the PR pipeline:
# .github/workflows/agent-pr.yml (excerpt)
- name: Build, push, and create candidate version
run: |
python scripts/deploy_agent_version.py \
--agent customer-support \
--project $EVAL_PROJECT \
--version-suffix pr-${{ github.event.number }} \
--traffic 0 # create the version, do not route traffic yet
- name: Run evaluations against candidate endpoint
run: |
python scripts/run_evals.py \
--agent customer-support \
--version pr-${{ github.event.number }} \
--baseline last-known-good \
--fail-on-regression
The PR creates a candidate version with zero traffic weight against a long-lived "eval" Foundry project, runs evaluations against the candidate version's dedicated endpoint, and then deletes the candidate version on PR close. A standing eval project beats a per-PR Foundry project — provisioning a project per PR is slow and adds RBAC overhead that does not earn its keep.
Environment Promotion
Three environments is the floor: dev, staging, prod. Each is its own Foundry project, ideally its own Foundry account in its own resource group. What promotes between them is the image digest and the version spec — not source code, and not "redeploy from main."
A workable model:
- dev — every push to a feature branch builds an image and creates a dev version. Loose evaluation thresholds. Used for human poking and end-to-end debugging.
- staging — merges to
maincreate a staging version. Full eval suite, strict thresholds. Same sandbox sizing, same env vars, same protocols as prod. - prod — manually approved promotion from staging. Promotion script reads the staging manifest, finds the image digest that passed, and creates the prod version pointed at that exact digest. No rebuild.
The "same digest" rule is the recommended pattern for safe promotion. If staging passed evaluations on customer-support@sha256:abc… running gpt-4.1, prod should get that exact image. Re-building from main in the prod pipeline reintroduces the risk you spent staging trying to eliminate — a different base-image patch level, a different transitive dependency, a different build clock — even though nothing in your source changed.
GitHub Actions environments make the approval concrete:
jobs:
promote-prod:
needs: deploy-staging
environment: production # requires reviewer approval
runs-on: ubuntu-latest
steps:
- name: Create prod version from staging manifest
run: |
python scripts/deploy_agent_version.py \
--agent customer-support \
--project $PROD_PROJECT \
--from-manifest staging-manifest.json \
--traffic 10 # canary at 10%
The canary weight is the second half of safe promotion: create the prod version, give it a small fraction of traffic, watch the App Insights traces, then shift the rest with promote_version.py.
Traffic-Split Rollout and Instant Rollback
Weighted version traffic changes the rollback model entirely. Rollback typically avoids rebuilding or redeploying artifacts — the previous version is still there, ready to take traffic.
A typical canary flow:
- Create new version
v42at 0% traffic. Endpoint exists; no production calls reach it. - Shift to 10%. Observe for an hour or a day, depending on traffic volume.
- Shift to 50%, then 100%. Old version stays at 0% but is not deleted.
- After a stability window (commonly a week), delete the previous version to free quota.
Rollback is the reverse: shift weights back to the previous version. It is a control-plane call, not a deploy. The agent's endpoint URL does not change, sessions in flight continue on whichever version they started on, and new sessions land on whatever the weights say.
Two consequences worth internalizing:
- Keep at least the last two known-good versions live. Rollback is only as fast as your ability to flip weights to a version that already exists.
- Do not skip the canary step under deadline pressure. A 0%→100% cutover gives you the same blast radius as a non-canaried deploy. The platform supports incremental rollout; use it.
For a destructive change — a removed protocol, a renamed agent, an env var the previous version cannot tolerate — rollback may not be safe. Forward-fix is the answer. Identify those changes in PR review and require an explicit "rollback path: forward-fix" note in the PR.
Handling Model Version Changes
A model deployment bump is the highest-blast-radius runtime change you can make to a Hosted Agent: the agent's behaviour on every input can shift. Treat it like a dependency upgrade.
- Open a PR that changes only the
AZURE_AI_MODEL_DEPLOYMENT_NAME(or the model version on the deployment, via Terraform). - Build a new image if needed, create a new agent version, run the full eval suite at 0% traffic.
- Run a larger regression dataset if you have one.
- Require a human reviewer who is not the PR author.
- Promote through staging, then canary in prod for at least one business day before shifting full traffic.
If the new model is faster or cheaper, the temptation is to skip steps. Don't. A quality regression in prod almost always costs more than a careful upgrade.
The Terraform side is small: openai_model_version is a variable on the azurerm_cognitive_deployment. Terraform recreates the deployment if the version changes. The Hosted Agent picks up the new deployment the next time it calls the model — if you kept the deployment name stable, which is your contract with the agent code. If you change the deployment name as well, the agent needs a new version that knows the new name.
Observability That Actually Tells You Something
The platform injects an Application Insights connection string into every Hosted Agent container as an environment variable. Agents that use the protocol libraries emit OpenTelemetry traces by default. That gives you per-request latency, token counts, tool invocations, and conversation IDs out of the box. That is the floor. Add to it:
- Custom span attributes on every request. Agent name, agent version ID, image digest (short), model deployment name. Without these, post-incident analysis cannot tell you which version was live when a problem started — especially during a traffic-split rollout where two versions are serving simultaneously.
- Quality signal capture. Sample a percentage of production conversations into a queue for offline grading. Run the same graders you used in CI against that sample on a schedule. This is your drift detector for response quality.
- Sandbox right-sizing signals. Hosted Agents bill on the CPU/memory you allocate per session. Oversizing multiplies cost by your concurrency. Track CPU and available memory inside the sandbox and compare against the version's allocation — if peaks stay below ~50%, the next version should drop a tier; if they push above ~70%, raise it. Right-sizing is a per-version decision because versions are immutable.
- Per-version error and latency. Slice every standard metric by version ID. A canary that looks fine in aggregate can be quietly worse than the previous version on specific request shapes.
- Cost dimensions. Tag traces with
customer_idortenant_idif you have multi-tenancy. Aggregating session cost by tenant in App Insights is straightforward once the dimension is on the span. - Alerts on shape, not just rate. A doubling in average response length or a sudden drop in tool invocation frequency often precedes a quality regression that error-rate alerts will miss entirely.
A weekly "agent health" report in your team channel — pulling these App Insights queries together — beats a perfect dashboard nobody opens.
A Pragmatic Maturity Path
Most teams cannot build the whole loop on day one. A reasonable order:
- Infrastructure in Terraform. AIServices account, project, model deployment, ACR, App Insights, role assignment so the project MI can pull from ACR.
- First agent deployed manually with
azd. Just to prove the round trip end to end. agent.yamlplus a deploy script that builds, pushes by digest, and creates a version. One environment.- Three environments with manual promotion by manifest.
- A 20-row eval dataset with one grader, run on every PR. Advisory only at first.
- Eval as a blocking gate. Thresholds tuned from the advisory phase.
- Canary rollout via traffic split. Versions held live for a stability window before deletion.
- Production sampling into offline evaluation. Drift detection.
- Model version upgrade playbook. Documented, exercised once on a low-risk agent.
- Tested rollback via weight shift. The first time you discover a rollback bug should not be during an incident.
Each step is independently useful. Skipping ahead — particularly to step 6 without time in step 5 — produces thresholds that block legitimate changes and erode trust in the pipeline.
Where This Is Heading
The platform is moving. A few things to watch as you build:
- Declarative Hosted Agent versions in Terraform. AzureRM coverage of Hosted Agents and agent versions is expanding. Parts of the deploy script will collapse into Terraform as that lands. The script-driven approach in this post is the bridge, not the destination.
- Continuous evaluation as a first-class platform feature. Sampling production traffic into scheduled evals — what you wire by hand today — is moving into the Foundry control plane.
- Multi-agent composition over A2A. As the A2A endpoint moves from preview to general availability and more frameworks ship A2A clients, multi-agent workflows become a first-class deployment shape. The DevOps loop extends — version pinning between agents, eval at the workflow level, observability across the agent graph — but the manifest grows accordingly.
- Toolbox-managed tool surfaces. As more tool integrations move behind the project Toolbox MCP endpoint, the agent image gets smaller and the tool configuration becomes a project-level concern. That changes what belongs in
agent.yamlversus what belongs in Terraform.
The throughline: the more the platform absorbs, the more your job shifts from wiring plumbing to defining policy. What "good" means for your agent, what the quality floor is, who can approve a model upgrade, how fast you can roll back. Those decisions do not get automated away. The pipeline just makes them executable.
Conclusion
Terraform provisions the Foundry project, model deployment, ACR, and observability. The DevOps loop on top of it — container builds pinned by digest, immutable agent versions, evaluation as a release gate, manifest-driven promotion across environments, traffic-split canary and rollback, and observability sliced by version — gets Hosted Agents to production and keeps them there.
Build it incrementally. Treat the image digest and the version spec as the deploy artifact, not the source branch. Make evaluation a check the pipeline cares about. Use version weights as your rollout and rollback primitive. And design for the day the platform absorbs the next layer of plumbing, so that when it does, your work moves up the stack instead of getting thrown away.