Microsoft Developer Community Blog articles

From AI Suggestions to Autonomous CRM Actions in Dynamics 365

SachinDas — Tue, 09 Jun 2026 16:39:43 GMT

🔷 Executive Summary

Most AI implementations in Dynamics 365 start—and end—with case summarization.

While useful, summarization alone does not fundamentally transform service operations.

In this post, I’ll walk through a CRM Copilot Agent Accelerator built on Microsoft Power Platform that goes far beyond summarization. It introduces a modular, extensible AI architecture that evolves from:

AI-generated insights
to predictive intelligence
to autonomous execution

This approach enables organizations to reduce manual effort, improve decision quality, and scale support operations without additional Copilot licensing.

🔷 The Business Problem

In most enterprise service operations:

Agents spend 30–40% of their time on repetitive tasks
Case triage requires manual reading of history
Decisions vary significantly between agents
Knowledge base usage is inconsistent
Escalations are reactive rather than predictive

The result?

Slower resolutions
Increased SLA breaches
Poor customer experience
High onboarding time for new agents

🔷 Solution Overview: CRM Copilot Agent Accelerator

The solution introduces a layered AI-first architecture built on:

Dynamics 365 + Dataverse (data foundation)
Power Automate (orchestration)
AI Builder (GPT models) (intelligence layer)
PCF Controls + Teams integration (user experience)

At its core, the accelerator:

Generates AI summaries + next actions
Stores them in Dataverse (persistent & reusable)
Extends capabilities through modular add-on packs

👉 These add-ons transform AI from a helper into an operational engine.

🔷 Architecture Overview

The solution follows a layered enterprise architecture model:

1. Trigger Layer

Case create/update
Email, chat, or call events
SLA checkpoints

2. Orchestration Layer

Power Automate flows
Dataverse plugins
Optional Copilot Studio agents

3. AI Processing Layer

AI Builder prompts (summarization, classification)
Sentiment detection
Risk prediction

4. Data Layer

Dataverse entities (Case, Account, Knowledge Base)
AI-enriched fields
Analytics tables

5. Experience Layer

Model-driven apps
PCF widgets
Teams Adaptive Cards
Power BI dashboards

👉 This architecture allows scalable AI enrichment across the entire CRM lifecycle

🔷 The Real Innovation: Modular Add-On Packs

The real differentiation is not the base AI capability.

It is the introduction of eight independently deployable add-on packs.

🔹 Key Add-On Categories

Add-On	Capability
PCF Widgets	Visual AI insights (risk radar, similar cases)
Predictive Engine	SLA & escalation prediction
Teams Integration	AI insights pushed to Teams
Customer 360	Persona + churn intelligence
Knowledge Intelligence	Self-improving KB loop
Multilingual AI	Cross-language support
Voice & Omnichannel	Call/chat AI summarization
Agentic Automation	AI takes action automatically

👉 These packs address 10+ gaps not covered by D365 Copilot

🔷 Deep Dive: Key Differentiators

1. From Reactive to Predictive AI

Instead of reacting to issues:

SLA breach risk is calculated in real time
Escalation probability is predicted before it happens
Supervisors receive proactive alerts

👉 This reduces escalations by up to 25–40%

2. Visual AI Experience (PCF Controls)

Instead of text-heavy UI:

Radar charts show case complexity & risk
Similar case panels enable faster resolution
Coaching tickers keep guidance always visible

👉 This dramatically improves agent usability and adoption

3. Self-Improving Knowledge Base

A major gap in most systems:

Knowledge is consumed but never improved

This solution:

Detects KB gaps automatically
Generates AI-drafted knowledge articles
Enables continuous learning

👉 Leads to 3× growth in KB coverage

4. From AI Suggestion → AI Action

Most AI stops at suggestion.

This accelerator evolves into Agentic AI:

Stage	Capability
AI Informs	Summary + Insights
AI Suggests	Recommendations
AI Drafts	Email + KB articles
AI Acts	Tasks, routing, execution
AI Orchestrates	Multi-agent automation

👉 AI starts doing the work — not just guiding it

🔷 Business Impact

Organizations adopting this accelerator can expect:

✅ 90% reduction in case triage time
✅ 40% reduction in misrouted cases
✅ 60–70% improvement in handling efficiency
✅ Faster agent onboarding (less than a day)
✅ Reduced dependency on Copilot licensing

👉 All achieved using existing Power Platform investments

🔷 Future Roadmap

The current solution sets the foundation for:

✅ Copilot Studio multi-agent orchestration
✅ Advanced ML-based predictions
✅ Vector-based knowledge retrieval (RAG)
✅ Integration with Microsoft Fabric for intelligence
✅ Autonomous CRM workflows

👉 The evolution leads toward fully AI-driven service operations

🔷 Why This Matters

This is not just another AI demo.

It represents a shift from:

❌ “AI helps agents”
➡️ ✅ “AI becomes part of the operation”

And does so using:

No custom AI infrastructure
No additional Copilot licensing
Native Microsoft ecosystem

🔷 Call to Action

If you’re working on Dynamics 365, Power Platform, or AI-driven CRM solutions:

✅ Start with AI enrichment
✅ Extend with predictive capabilities
✅ Move toward agentic automation

This accelerator pattern can help you move faster, scale better, and deliver measurable value.

🔷 Closing Thought

“The future of CRM is not AI assisting users —
it is AI transforming how work gets done.”

🔷 References & Further Reading

The concepts described in this CRM Copilot Agent Accelerator are built on capabilities available across Microsoft Dynamics 365, Dataverse, Power Platform, Azure AI, and Copilot technologies.

Official Microsoft references:

📌 Dynamics 365 Customer Service

📌 Microsoft Dataverse

📌 Power Automate

📌 AI Builder

📌 Microsoft Copilot Studio

📌 Power Apps Component Framework (PCF)

📌 Microsoft Teams Integration

📌 Omnichannel & Conversational AI

📌 Customer Insights & Customer 360

Dynamics 365 Customer Insights Documentation

📌 Knowledge Management

Knowledge Management in Dynamics 365 Customer Service

📌 Power BI Analytics

Power BI Documentation

📌 Azure AI & Generative AI

📌 Retrieval-Augmented Generation (RAG)

📌 Microsoft Fabric

Microsoft Fabric Documentation

📌 Power Platform Architecture & Well-Architected Guidance

These resources provide the foundational building blocks for implementing AI-assisted, predictive, and agentic experiences across Dynamics 365 Customer Service and the broader Microsoft Power Platform ecosystem.

Troubleshooting Azure Container Apps and Jobs for .NET and Django Workloads

BhaktiRath95 — Tue, 09 Jun 2026 07:58:44 GMT

Introduction

Deploying to Azure Container Apps feels like a huge step forward — you get serverless containers, automatic scaling, built-in ingress, and managed environments without managing Kubernetes directly. But when something goes wrong and your container refuses to start, or your Container App Job silently fails, it can feel like debugging inside a black box.

This first part of our four-part series walks through the most common deployment and startup failures you will hit when running .NET and Django applications on Azure Container Apps and Container App Jobs. We cover what the real error looks like, why it is happening under the hood, and what you need to do to fix it — step by step.

The Real-World Problem: "My Container App is stuck in a restart loop and I have no idea why"

This is probably the most common thing engineers report when they first move workloads to Azure Container Apps. The deployment finishes successfully, the revision shows as active, but the app never becomes healthy. In the Azure portal it cycles between `Running` and `Degraded`, and in the logs you see cryptic exit codes or — even worse — nothing at all.

The root causes almost always fall into one of these buckets:

The container exits immediately because the process crashes on startup (misconfiguration, missing secrets, unhandled exceptions).
The health probe fails because the app takes too long to start or is listening on the wrong port.
A Container App Job never completes because it times out or the job process exits with a non-zero code.

Let us walk through each of these in detail.

Scenario 1: Your .NET Application Crashes at Startup

What You See

Your Container App revision goes into a restart loop. You check the Log Analytics workspace and see something like this:

ContainerAppConsoleLogs_CL | where ContainerAppName_s == "my-dotnet-api" | where TimeGenerated > ago(30m) | project TimeGenerated, Log_s | order by TimeGenerated desc

The output shows:

Unhandled exception. System.InvalidOperationException: Unable to resolve service for type 'MyApp.Data.AppDbContext'

at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService

...

Application is shutting down.

Or even more commonly with Entity Framework Core:

fail: Microsoft.EntityFrameworkCore.Database.Connection

An error occurred using the connection to database 'mydb' on server 'myserver.database.windows.net'.

System.Net.Sockets.SocketException: Connection refused

Why This Happens

When .NET 6+ applications start up, they run the entire `WebApplication.Build()` pipeline before accepting traffic. If any registered service — like a database context — cannot be constructed or if the connection string is missing or wrong, the application throws an unhandled exception and the process exits with a non-zero code. Container Apps detects this exit and restarts the container. This cycle repeats indefinitely.

The most frequent trigger is missing or incorrectly named environment variables and secrets. In local development you rely on `appsettings.Development.json` or `user secrets`, but in Container Apps those files are not present unless you explicitly copy them into the image (which you should never do for secrets).

Step-by-Step Fix

Step 1 — Verify your secrets and environment variables are configured correctly.

In the Azure portal, navigate to your Container App → Configuration → Secrets and Environment variables. Make sure every value your app reads from IConfiguration is defined here.

From the CLI you can inspect and update them like this:

# Add or update a secret reference az containerapp secret set --name my-dotnet-api --resource-group my-rg --secrets "connectionstring=Server=myserver.database.windows.net;..." # Reference that secret as an environment variable az containerapp update --name my-dotnet-api --resource-group my-rg --set-env-vars "ConnectionStrings__DefaultConnection=secretref:connectionstring"

Step 2 — Make sure your .NET app reads configuration correctly.

The naming convention that trips up almost everyone: Azure Container Apps uses double underscores (`__`) to represent the colon (`:`) separator in .NET configuration keys. So `ConnectionStrings:DefaultConnection` becomes `ConnectionStrings__DefaultConnection` as the environment variable name.

// This reads from "ConnectionStrings__DefaultConnection" env var automatically builder.Services.AddDbContext<AppDbContext>(options => options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

Step 3 — Add a startup health check that gives meaningful feedback.

Configure a liveness probe with a generous initial delay to avoid a container being killed before it has had time to start:

# In your Container App YAML configuration probes: - type: Liveness httpGet: path: /health port: 8080 initialDelaySeconds: 30 periodSeconds: 10 failureThreshold: 5 - type: Readiness httpGet: path: /health/ready port: 8080 initialDelaySeconds: 15 periodSeconds: 5 failureThreshold: 3

Add the corresponding health endpoint in your .NET app:

// Program.cs builder.Services.AddHealthChecks() .AddSqlServer( builder.Configuration.GetConnectionString("DefaultConnection")!, name: "database", tags: new[] { "ready" }); app.MapHealthChecks("/health"); app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = check => check.Tags.Contains("ready") });

Step 4 — Pull the raw container logs using the CLI to see exactly what happened before the container exited:

az containerapp logs show --name my-dotnet-api --resource-group my-rg --type console --tail 50

Scenario 2: Your Django Application Fails to Start

What You See

Your Django app deploys, the container starts, but within seconds it exits. In the logs you see one of these common errors:

django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

Or:

django.db.utils.OperationalError: could not connect to server: Connection refused

Is the server running on host "localhost" (127.0.0.1) and accepting

TCP/IP connections on port 5432?

Or the static files problem that catches almost everyone:

[Errno 2] No such file or directory: '/app/staticfiles'

Why This Happens

Django validates its configuration eagerly when the WSGI/ASGI server starts. If `SECRET_KEY` is not set, if `ALLOWED_HOSTS` does not include the container's hostname or the ingress FQDN, or if `DEBUG=True` is set in a configuration branch that requires a proper database, Django refuses to serve any requests.

The static files error comes up because many teams forget to run `python manage.py collectstatic` as part of the container image build process. The `STATIC_ROOT` directory simply does not exist at runtime.

Step-by-Step Fix

Step 1 — Set required Django environment variables in your Container App.

az containerapp secret set --name my-django-app --resource-group my-rg --secrets "django-secret-key=your-very-secret-key-here" "db-password=your-db-password" az containerapp update --name my-django-app --resource-group my-rg --set-env-vars "DJANGO_SECRET_KEY=secretref:django-secret-key" "DEBUG=False" "ALLOWED_HOSTS=my-django-app.happyfield-abc123.eastus.azurecontainerapps.io" "DATABASE_URL=secretref:db-password"

Step 2 — Run `collectstatic` during Docker image build, not at runtime.*

This is a very common mistake. Static files should be baked into the image, not generated when the container starts. Update your `Dockerfile`:

FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . # Collect static files at build time with a dummy SECRET_KEY RUN SECRET_KEY=build-time-placeholder python manage.py collectstatic --noinput EXPOSE 8000 CMD ["gunicorn", "myproject.wsgi:application", "--bind", "0.0.0.0:8000", "--workers", "2", "--timeout", "120"]

Step 3 — Make sure Gunicorn is configured correctly for Container Apps.

The most important thing to verify is that Gunicorn is binding to `0.0.0.0` and not `127.0.0.1`. Container Apps expects the application to listen on all interfaces so that the ingress layer can reach it. Also make sure the port matches what you defined in your Container App's ingress target port:

# Set ingress to match Gunicorn's bind port az containerapp ingress update --name my-django-app --resource-group my-rg --target-port 8000 --type external

Step 4 — Handle database migrations safely.

Never run `python manage.py migrate` as part of your container startup command. If you have multiple replicas, all of them will try to run migrations simultaneously, which can corrupt your schema. Instead, use a Container App Job to run migrations as a pre-deployment step:

# Create a one-time Container App Job to run migrations az containerapp job create --name django-migrate-job --resource-group my-rg --environment my-aca-env --trigger-type Manual --replica-timeout 300 --image myregistry.azurecr.io/my-django-app:latest --command "python" --args "manage.py" "migrate" --env-vars "DJANGO_SECRET_KEY=secretref:django-secret-key" "DATABASE_URL=secretref:db-password" # Execute the migration job before deploying the new revision az containerapp job start --name django-migrate-job --resource-group my-rg

Scenario 3: Your Container App Job Fails Silently or Times Out

What You See

You trigger a Container App Job — maybe it is a nightly data processing job, a scheduled report generator, or a cleanup task — and in the Azure portal the execution shows as Failed with no helpful error message. Or it shows as Running for an unusually long time and then transitions to Failed with a timeout error.

Why This Happens

Container App Jobs have a `replicaTimeout` property. If your job process does not complete within that window, Azure Container Apps kills it and marks the execution as failed. This is different from Container Apps (services) where the container keeps running. Jobs are expected to run to completion and exit with code `0`.

The silent failure happens when your job process exits with a non-zero exit code but does not write anything to `stdout` or `stderr`. Container Apps records the exit code but has no log content to show you.

Step-by-Step Fix

Step 1 — Make your job emit logs to stdout explicitly.

Every print statement, every log line should go to `stdout` or `stderr`. In Python:

import sys import logging logging.basicConfig( level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s", handlers=[logging.StreamHandler(sys.stdout)] ) logger = logging.getLogger(__name__) def main(): logger.info("Job starting") try: # your job logic here process_data() logger.info("Job completed successfully") sys.exit(0) except Exception as e: logger.error(f"Job failed with error: {e}", exc_info=True) sys.exit(1)

In .NET:

// Use ILogger which writes to stdout by default in containers public class MyJob { private readonly ILogger<MyJob> _logger; public MyJob(ILogger<MyJob> logger) { _logger = logger; } public async Task RunAsync(CancellationToken cancellationToken) { _logger.LogInformation("Job starting at {Time}", DateTimeOffset.UtcNow); try { await ProcessDataAsync(cancellationToken); _logger.LogInformation("Job completed successfully"); } catch (Exception ex) { _logger.LogError(ex, "Job failed"); throw; // Let the process exit with non-zero code } } }

Step 2 — Set an appropriate replica timeout and retry count.

Be realistic about how long your job takes in production, then add a buffer:

az containerapp job update --name my-processing-job --resource-group my-rg --replica-timeout 1800 # 30 minutes --replica-retry-limit 2 # Retry twice before marking as failed

Step 3 — Check job execution history and logs.

# List recent job executions and their status az containerapp job execution list --name my-processing-job --resource-group my-rg --output table # Get logs for a specific execution az containerapp job execution show --name my-processing-job --resource-group my-rg --job-execution-name my-processing-job-abc123

From Log Analytics:

ContainerAppConsoleLogs_CL | where ContainerAppName_s == "my-processing-job" | where TimeGenerated > ago(24h) | project TimeGenerated, Log_s, ContainerName_s | order by TimeGenerated desc

Summary: Your Startup Troubleshooting Checklist

Before you dig into complex diagnostics, run through this checklist whenever a Container App or Job fails to start:

Are all required environment variables and secrets defined and correctly referenced?
Is the application listening on `0.0.0.0` and on the port that matches the ingress target port?
Does the Dockerfile copy everything needed for the app to run (migrations, static files, etc.)?
Are health probes configured with enough initial delay for the app to start?
For jobs: is the replica timeout long enough, and does the process exit with code 0 on success?
Is the container registry accessible from the Container Apps environment (managed identity or registry credentials configured)?
Are the resource allocations (CPU and memory) sufficient for the application to start without OOM-killing?

References and Sample Resources

Use these resources for deeper implementation details and production-ready patterns.

Azure Container Apps docs (core)

.NET and Django references

Sample repositories

What's Next

In Part 2 of this series, we move past startup failures and look at what happens after your app is running — the frustrating world of cold starts, scaling delays, and startup latency spikes that make your application feel slow under real production traffic.

Part of the series: Troubleshooting Azure Container Apps in Production

Next: Part 2 — From Slow to Snappy: Performance Tuning Cold Starts and Scaling Delays in Azure Container Apps

Getting Secrets Out of YAML: Implementing Azure Key Vault CSI Driver on AKS with Workload Identity

anandranjan — Mon, 08 Jun 2026 07:00:00 GMT

Why This Pattern Matters
The Problem with Secrets in YAML
What We Wanted to Achieve
Architecture Overview
How the Flow Works
Implementation Prerequisites
Step-by-Step Implementation
Understanding the YAML Components
Secret Rotation and Reloaders
Common Pitfalls and Troubleshooting
Security and Operational Benefits
Key Takeaways
Microsoft Documentation References
Final Thoughts

Why This Pattern Matters

Most Kubernetes environments start with good intentions around secret management.

Over time, however, many AKS deployments gradually evolve toward patterns like:

value: "#{SomeSecret}#"

A pipeline substitutes the value during deployment, the application works, and the pattern spreads across services.

The problem is that this quietly turns:

deployment pipelines
rendered manifests
release artifacts
CI/CD logs

into part of the secret distribution path.

Even if the secret originates from Azure Key Vault, the value itself still travels through multiple systems before reaching the pod.

That creates:

unnecessary exposure risk
difficult rotation workflows
broader operational blast radius
audit complexity

This article walks through a cleaner runtime-based model using:

Azure Key Vault CSI Driver
AKS Workload Identity
Managed Identity federation

where workloads authenticate directly to Key Vault without secret values ever appearing in YAML or pipelines.

The Problem with Secrets in YAML

The traditional pattern usually looks like this:

env: - name: APPLICATION_SECRET value: "#{ApplicationSecret}#"

At deployment time:

The pipeline retrieves the secret
The placeholder is replaced
Kubernetes receives the rendered manifest

Operationally, this means:

the pipeline temporarily possesses the secret
the rendered YAML contains the secret
logs or artifacts may accidentally retain it
secret rotation requires redeployment

This often exists because:

Key Vault secret names don't match application configuration names
teams want backward compatibility
changing application code is expensive

The Azure Key Vault CSI Driver solves this by introducing a runtime mapping layer instead of a pipeline substitution layer.

What We Wanted to Achieve

The design goals were simple:

No secret values in YAML
No secret substitution in CI/CD pipelines
Runtime-only secret retrieval
Identity-based authentication
Support for mapping Key Vault names to application config names
Minimal or zero application code changes
Secure secret rotation workflows

Architecture Overview

+----------------------+ | AKS Application | | Pod | +----------+-----------+ | v +----------------------+ | Kubernetes | | ServiceAccount | +----------+-----------+ | v +----------------------+ | AKS Workload | | Identity Webhook | +----------+-----------+ | v +----------------------+ | Federated Managed | | Identity | +----------+-----------+ | v +----------------------+ | Azure Key Vault | +----------+-----------+ | v +----------------------+ | CSI Driver Mount | | (/mnt/secrets-store) | +----------+-----------+ | v +----------------------+ | Application Reads | | Secret at Runtime | +----------------------+

How the Flow Works

When a pod starts:

The pod uses a Kubernetes ServiceAccount
AKS Workload Identity injects an OIDC token
The Secrets Store CSI Driver uses that token
Azure validates the federated identity relationship
The Managed Identity receives access to Key Vault
Secrets are retrieved at runtime
Secrets become available

This entire process happens dynamically during pod startup.

No secret values pass through:

YAML manifests
pipeline variables
Helm values
release artifacts

The pod authenticates as itself.

Implementation Prerequisites

Before implementation, ensure:

Requirement - Purpose

AKS Cluster - Runtime environment
Azure Key Vault - Centralized secret store
OIDC Enabled - Required for federation
Workload Identity Enabled - Enables identity injection
Managed Identity - Authentication mechanism
Secrets Store CSI Driver - Runtime secret retrieval

Implementation

The implementation can generally be divided into two phases:

Platform Setup - One-time AKS and Azure configuration performed by platform/infrastructure teams
Application Onboarding - Per-service configuration performed by application teams

This separation is important because most of the complexity exists only once at the platform layer. After the foundational setup is complete, onboarding additional workloads becomes significantly simpler and more repeatable.

The examples below are intended to demonstrate the overall implementation pattern and architecture flow. Exact implementation details may vary depending on:

organizational RBAC models
networking restrictions
Key Vault access configuration
GitOps/Helm workflows
cluster governance policies
AKS versions and add-on configurations

Phase 1 — Platform Setup

1. Enable Required AKS Capabilities

A typical implementation starts by enabling the AKS capabilities required for identity federation and runtime secret retrieval.

These usually include:

OIDC Issuer - Enables identity federation between AKS and Microsoft Entra ID
Workload Identity - Injects federated identity tokens into workloads
Azure Key Vault CSI Driver - Retrieves secrets securely at runtime

Example Azure CLI commands:

az aks update \ --resource-group <resource-group> \ --name <aks-cluster> \ --enable-oidc-issuer \ --enable-workload-identity

az aks enable-addons \ --addons azure-keyvault-secrets-provider \ --resource-group <resource-group> \ --name <aks-cluster>

Most organizations validate these capabilities before onboarding workloads.

2. Configure a Managed Identity

Each workload or application typically receives its own User-Assigned Managed Identity.

This follows least-privilege principles and helps reduce operational blast radius between services.

Example:

az identity create \ --name workload-identity \ --resource-group <resource-group>

The identity is then granted access to retrieve secrets from Azure Key Vault.

Common approaches include:

Azure RBAC role assignments
Azure AD group-based access
Key Vault access policies (legacy environments)

Typical required permissions include:

Get
List

for secrets stored in the vault.

3. Configure Federated Identity Trust

Workload Identity relies on federation between:

AKS
Kubernetes ServiceAccounts
Microsoft Entra ID
Managed Identities

A Federated Identity Credential establishes this trust relationship.

The implementation usually maps:

a Kubernetes namespace
a Kubernetes ServiceAccount
an AKS OIDC issuer

to a specific Managed Identity.

Example structure:

system:serviceaccount:<namespace>:<serviceaccount>

This configuration is one of the most important parts of the setup because federation mismatches are a very common source of authentication failures.

4. Prepare Kubernetes Namespaces and ServiceAccounts

Application namespaces and ServiceAccounts are typically created before workload onboarding begins.

The ServiceAccount acts as the identity boundary for the workload.

Example:

apiVersion: v1 kind: ServiceAccount metadata: name: workload-sa namespace: application-namespace annotations: azure.workload.identity/client-id: "<managed-identity-client-id>"

This annotation links the Kubernetes ServiceAccount to the Azure Managed Identity.

Phase 2 — Application Onboarding

Once the platform capabilities are available, application onboarding becomes significantly simpler.

5. Define Secret Retrieval Using SecretProviderClass

The SecretProviderClass resource defines:

which Key Vault secrets should be retrieved
how those secrets should be exposed inside Kubernetes

Example:

apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: workload-kv-secrets namespace: application-namespace spec: provider: azure parameters: usePodIdentity: "false" clientID: "<managed-identity-client-id>" keyvaultName: "<keyvault-name>" tenantId: "<tenant-id>" objects: | array: - | objectName: application-secret objectType: secret secretObjects: - secretName: workload-secret type: Opaque data: - key: APPLICATION_SECRET objectName: application-secret

A few important concepts exist here:

objects - Defines which Key Vault secrets to retrieve
secretObjects - Optionally syncs secrets into Kubernetes Secrets

This separation allows:

Key Vault secret naming
Kubernetes secret naming
application environment variable naming

to remain independent from one another.

That flexibility is one of the biggest advantages of the CSI Driver approach.

6. Update Workloads to Use Workload Identity

Application deployments are then updated to:

use the correct ServiceAccount
enable Workload Identity
mount the CSI volume
optionally consume synced Kubernetes Secrets

Typical workload changes include:

Enable Workload Identity

labels: azure.workload.identity/use: "true"

Without this label:

token injection does not occur
workload federation fails

Attach the ServiceAccount

serviceAccountName: workload-sa

Mount the CSI Volume

volumes: - name: secrets-store csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "workload-kv-secrets"

volumeMounts: - name: secrets-store mountPath: "/mnt/secrets-store" readOnly: true

Even if the application ultimately consumes secrets through environment variables, the CSI volume still needs to be mounted because the Kubernetes Secret synchronization occurs only after a successful mount.

Consume Secrets Inside the Application

Applications can consume secrets:

directly as mounted files
or through synced Kubernetes Secrets using secretKeyRef

Example:

env: - name: APPLICATION_SECRET valueFrom: secretKeyRef: name: workload-secret key: APPLICATION_SECRET

One of the biggest operational advantages here is that applications usually require little or no code change.

The application still reads:

environment variables
configuration values
mounted files

The underlying secret delivery mechanism changes — not the application contract.

7. Validate the Integration

Once deployed, teams typically validate:

pod startup success
successful CSI volume mounts
secret retrieval from Key Vault
environment variable injection
workload authentication

Common validation activities include:

inspecting mounted secret paths
checking pod logs
reviewing CSI Driver logs
confirming Key Vault access logs
validating Workload Identity token injection

Most implementation issues generally fall into one of these categories:

Federated Identity Credential mismatches
missing workload labels
Key Vault permission issues
incorrect ServiceAccount mappings
missing CSI volume mounts

Operational Recommendation

For production environments, many organizations also add:

automated secret rotation handling
restart controllers such as Stakater Reloader
readiness probes
rolling deployment strategies
monitoring and alerting around secret retrieval failures

These additions help ensure secret updates can occur safely with minimal or zero downtime.

Understanding the YAML Components

The implementation consists of three connected resources:

ServiceAccount - Identity binding
SecretProviderClass - Secret retrieval definition
Deployment - Secret consumption

Think of them as a chain:

ServiceAccount ↓ Managed Identity ↓ SecretProviderClass ↓ CSI Driver ↓ Deployment

If any naming mismatch exists, secret retrieval fails.

Consistency matters.

Secret Rotation and Reloaders

One major operational advantage is secret rotation.

Previously:

Update secret
Update pipeline variable
Redeploy application

Now:

Rotate secret in Key Vault
CSI driver refreshes mounted content
Application consumes updated value

For applications using environment variables:

pod restarts are usually required

Many teams use:

Stakater Reloader

to automatically restart workloads when secrets change.

Combined with:

multiple replicas
readiness probes
rolling deployments

this enables zero-downtime secret refresh workflows.

Common Pitfalls and Troubleshooting

Federated Credential Subject Mismatch

Most common issue.

The subject must exactly match:

system:serviceaccount:<namespace>:<serviceaccount>

Missing Workload Identity Label

This label is mandatory:

azure.workload.identity/use: "true"

Without it:

no token injection occurs

Missing CSI Volume Mount

Even if using:

secretKeyRef

the CSI volume must still be mounted.

Otherwise:

Kubernetes Secret sync never occurs

Key Vault Permission Issues

Federation success does not guarantee Key Vault authorization.

The Managed Identity still requires:

Get
List

permissions on the vault.

Environment Variables Do Not Auto-Refresh

Secrets mounted as files can refresh dynamically depending on application behavior.

Environment variables do not.

If using:

secretKeyRef

consider:

Reloader controllers
rolling restart strategies
readiness probes

to safely consume rotated secrets.

Security and Operational Benefits

After migration, several improvements become immediately visible:

No secret values in YAML
No secret values in pipelines
Runtime-only secret resolution
Per-service identity isolation
Centralized audit visibility in Azure
Easier secret rotation
Reduced operational blast radius
Cleaner DevSecOps posture

Most importantly:

The deployment pipeline stops being part of the secret distribution mechanism.

Key Takeaways

Secrets should never flow through deployment pipelines
Workload Identity removes the need for static credentials
CSI Driver enables runtime secret retrieval directly from Key Vault
SecretProviderClass allows clean secret name mapping
File-based secret consumption is more secure than environment variables
Secret rotation becomes operationally simpler and safer

Microsoft Documentation References

Official Microsoft guidance:

AKS Workload Identity https://learn.microsoft.com/azure/aks/workload-identity-overview
Azure Key Vault Provider for Secrets Store CSI Driver https://learn.microsoft.com/azure/aks/csi-secrets-store-driver
Secrets Store CSI Driver https://secrets-store-csi-driver.sigs.k8s.io/
Federated Identity Credentials https://learn.microsoft.com/entra/workload-id/workload-identity-federation

These references are extremely useful when troubleshooting federation, RBAC, or CSI mounting issues.

Final Thoughts

The Azure Key Vault CSI Driver + Workload Identity pattern fundamentally changes how secrets flow through AKS environments.

Instead of distributing secrets through:

YAML
CI/CD systems
deployment artifacts

secrets remain protected behind:

identities
runtime authorization
centralized access control

The initial setup is slightly more involved than pipeline substitution, but once the platform foundations are established, onboarding additional workloads becomes lightweight and repeatable.

This is one of the highest-leverage security improvements available for Kubernetes platforms because it removes an entire category of secret exposure risk without requiring major application rewrites.

Secrets belong to identities — not deployment manifests.

#Azure #AKS #Kubernetes #DevSecOps #CloudSecurity #KeyVault #WorkloadIdentity #PlatformEngineering

Harness-Driven Agents: Secure Podcast Pipeline in Hyperlight MicroVM Sandbox

kinfey — Thu, 04 Jun 2026 14:12:41 GMT

The moment the agent reached for rm -rf

For most of 2024 and 2025, "agents" were a demo word. By 2026 they are something you run — autonomously, in a loop, executing code they wrote themselves a second ago.

I was watching one work late one night. I had given it a goal, a handful of tools, and the freedom to write and run its own Python. For twenty minutes it was magic: read a file, reason about it, write a script, run it, inspect the output, correct itself, try again. Then it produced this:

import shutil shutil.rmtree("/") # "cleaning up temporary files"

It was trying to be helpful — it had decided the workspace was cluttered and wanted a clean start. The "workspace," as far as that process was concerned, was my entire machine.

I killed it in time. But the lesson is the one every agent builder eventually arrives at: the model is not the dangerous part — the execution is. A chatbot that answers wrong is annoying. An agent that fetches a web page, runs code, and writes files has a blast radius. The bounding box has to come from infrastructure, not from a system prompt.

harnessagent_sandbox_demo is a concrete build that puts that bounding box in exactly the right place — and it does it in service of a real, charming little product: a daily five-minute Mandarin podcast about the FIFA World Cup 2026.

The scenario: a daily World Cup podcast, written by agents

Strip away the infrastructure for a second and look at what this thing actually does.

Every day it produces a fresh Mandarin podcast script about the FIFA World Cup 2026. Three LLM agents run in sequence:

SearchAgent — goes out and gathers the day's World Cup news.
ContentAgent — turns that raw material into structured podcast content.
GenScriptAgent — writes the final, readable five-minute script.

The output is two text files — one in Simplified Chinese, one in Traditional Chinese:

./outputs/<YYMMDD>/<YYMMDD>.simple.zh.txt ./outputs/<YYMMDD>/<YYMMDD>.tranditional.zh.txt

That's the whole product. It sounds simple — and the point of the project is that making it safe is the hard part. SearchAgent has to reach the open internet. All three agents write and run code. If you wire that naively, you have just built the exact machine that types shutil.rmtree("/") for you. So the entire architecture is organized around one principle: the agents get to do real work, but every dangerous capability is fenced behind a hardware boundary.

Why the obvious sandboxes fall short for agents

An agent is defined by an act-observe-correct loop running untrusted, model-generated code over and over. That single property breaks most conventional isolation choices.

Option	Why it falls short for agents
No sandbox	One rm -rf, one leaked .env, one rogue network call — the blast radius is the whole machine.
Container	Great for shipping apps, but a coding agent wants to build and run its own container, which means Docker-in-Docker and elevated privileges that quietly undo the isolation.
WASM / V8 isolate	Fast to start, but you isolate a language runtime, not an OS — no system packages, no arbitrary shell, and hardening the engine is a moving target.
Full VM	Rock-solid isolation, but cold starts in seconds and heavy memory — exactly the friction that pushes developers to skip isolation entirely.

Each option trades away safety, speed, or compatibility. A podcast pipeline that runs every day, spinning agents up and down, needs all three at once:

A real environment — to fetch URLs, run shells, call tools.
A hard boundary — so a bad step can't reach the host.
Near-instant lifecycle — because a slow sandbox is a sandbox developers skip, and an unused safety feature protects nobody.

The MicroVM answer, embedded as a library: Hyperlight

A MicroVM gives each workload its own kernel and a hardware-enforced boundary — the isolation strength of a full VM — stripped down to start in milliseconds and tear down just as fast. Misbehave inside, and you hit a wall; there is no path back to the host. And it is disposable by design: when an agent goes off the rails, you delete the sandbox and reopen in milliseconds, with nothing to clean up.

Most MicroVM runtimes (Firecracker and friends) are cloud infrastructure — server-side. Hyperlight is different: a lightweight Virtual Machine Manager (a CNCF sandbox project) designed to be embedded inside your application, like a library.

MicroVMs that boot in milliseconds, with guest function calls completing in microseconds.
No guest kernel, no OS — the guest is a purpose-built no_std Rust/C binary. Nothing in there to attack.
Sandboxed by default — no filesystem, no network, nothing, unless explicitly granted.
Typed function calls across the VM boundary, and snapshot/restore to rewind to a clean state between calls.
Runs on KVM, MSHV (Microsoft Hypervisor), and Windows Hypervisor Platform.

This project uses the Wasm backend: the three agents share a single HyperlightRuntime, and the guest is reset to a clean snapshot before every code execution. That detail is what makes a daily, many-step pipeline cheap — you capture the sandbox state once and rewind to it, instead of rebuilding a VM hundreds of times.

Agent = Model + Harness

The community has converged on a simple equation: Agent = Model + Harness.

The model is a brain in a jar — text in, text out, no memory between calls, no loop, no hands. It can express the intent to call a tool; it cannot actually call it.

The harness is the execution layer: it calls the model, handles its tool calls, and decides when to stop. As the Hugging Face glossary puts it, "if you're not the model, you're the harness."

That reframes the safety problem precisely. When my agent emitted shutil.rmtree("/"), the model deleted nothing — it merely suggested. The harness would have run it. The harness is where reasoning meets reality, so it is exactly where safety must live. The question stops being "how do I make the model safer?" and becomes: how do I build a harness that executes the model's intent inside a boundary it cannot escape?

The Microsoft Agent Framework answers that with first-class agent harness capabilities in Python and .NET, and it ships with one security note stated plainly:

For local shell execution, we recommend running this logic in an isolated environment and keeping explicit approval in place before commands are allowed to run.

The harness is the steering wheel — it does not pretend to be the seatbelt and the crumple zone. For that, it points you outward: run this somewhere isolated. Hyperlight is that isolated somewhere. This project snaps the two pieces together.

The architecture: two planes, one bridge

Here is the heart of the design. Two planes run together every episode:

An orchestration plane on the host — the WorkflowBuilder graph, the LLM clients, and the deterministic save step.
An execution plane inside one Hyperlight Wasm sandbox — the only place LLM-generated code is allowed to run.

The single bridge between them is one call: call_tool("fetch_url", ...).

The mapping to layers:

Layer	Component	Role
Model	Azure AI Foundry via FoundryChatClient (AzureCliCredential)	The reasoning brain behind each harness agent
Agent runtime	Microsoft Agent Framework create_harness_agent	Drives the model, advertises skills, handles tool calls, decides when to stop
Orchestration	WorkflowBuilder graph	prepare → SearchAgent → adapt → ContentAgent → adapt → GenScriptAgent → save_scripts
Code execution	CodeAct provider	Runs model-written code via the one execute_code tool — inside the MicroVM, never on the host
Isolation	Hyperlight Wasm MicroVM	One shared HyperlightRuntime; clean snapshot restored before every execute_code
Host tool	fetch_url (sandbox/podcast_tools.py)	The only network path; urllib + a BBC-only allow-list
Persistence	save_scripts Executor	Deterministic, no LLM — parses two fenced blocks and writes the two output files

The four invariants that make it safe

The README is explicit about what the diagram guarantees. These four invariants are the whole security argument.

The model never sees the network.Its only tool isexecute_code. Network access happens only when the guest itself runs call_tool("fetch_url", ...) from inside the sandbox. The model cannot reach the internet directly — it can only ask the guest to, and the guest can only reach BBC.
One sandbox per run, snapshot per call.All three agents share the sameHyperlightRuntime. Before every execute_code, the guest is reset to a clean snapshot — so nothing one step does can leak into the next, and there is no VM to rebuild.
Two counter paths — and why there are two.Thefunction_middleware (make_tool_call_recorder) sees the model-direct execute_code calls. But the inner, guest-initiated fetch_url is dispatched by Hyperlight straight to the FunctionTool, bypassing the middleware entirely. So a second counter — make_call_tool_counter(on_call=) — bumps state["tool_call_counts"][<agent>]["fetch_url"] on every guest invocation. Two observation points, because the architecture has two genuinely different call surfaces.
Deterministic save — no LLM in the persistence step.GenScriptAgentonly emits text. The save_scripts Executor parses the two fenced code blocks out of that text and writes the simplified and traditional files itself. There is no model in the loop when bytes hit disk, so the output path is fully predictable.

Now let's look at the real code surface

The README documents the API the demo is built on. The snippets below reflect that surface.

1. Install and environment

pip install agent-framework-hyperlight --pre

# Hyperlight needs a hypervisor: KVM on Linux, WHP on Windows. macOS is not yet supported. # The model runs on Azure AI Foundry; FoundryChatClient authenticates via AzureCliCredential. az login export HYPERLIGHT_PYTHON_GUEST_PATH="/path/to/python_guest"

2. A harness agent that carries only a stub — skills do the rest

Each of the three agents is built with create_harness_agent + FoundryChatClient. The agents themselves carry only a tiny stub instruction; their real role prompts and the shared sandbox/CodeAct guardrails live as file-based Agent Skills under skills/. The harness's built-in SkillsProvider advertises those SKILL.md packages, and the model loads them at runtime via load_skill.

from agent_framework import create_harness_agent from agent_framework.foundry import FoundryChatClient from azure.identity import AzureCliCredential # Model on Azure AI Foundry — not Azure OpenAI directly. client = FoundryChatClient(credential=AzureCliCredential()) # The agent carries a tiny stub. Its real persona — "you gather World Cup # news", "you write the script" — lives in a SKILL.md package under skills/, # advertised by the harness SkillsProvider and pulled in via load_skill. search_agent = create_harness_agent( chat_client=client, name="SearchAgent", instructions="You are a harness agent. Load your skill, then begin.", )

3 The CodeAct surface: one tool the model can see

This is the CodeAct pattern from 02-agents/context_providers/code_act/code_act.py. The model sees exactly one tool — execute_code. Any extra capability (here, only fetch_url) is reachable from inside the guest via call_tool(...).

# What the MODEL sees and writes — one script, not ten tool round-trips: # # # inside execute_code, running in the Hyperlight Wasm guest: page = call_tool("fetch_url", url="https://www.bbc.com/sport/football/world-cup") # # ... parse page["BODY"], pull out today's stories ... print(top_stories) # # execute_code is the ONLY tool on the model's surface. call_tool("fetch_url", ...) is reachable only from inside the sandbox.

4. The one host tool, with a BBC-only allow-list

fetch_url lives on the host (sandbox/podcast_tools.py). It is the single bridge across the boundary, and it is deliberately narrow.

import urllib.request from urllib.parse import urlparse ALLOWED_DOMAINS = {"bbc.com", "www.bbc.com"} # allow-list: BBC only def fetch_url(url: str) -> dict: """The ONLY network path out of the sandbox. Host-side, allow-listed.""" host = urlparse(url).netloc if host not in ALLOWED_DOMAINS: return {"STATUS": "blocked", "URL": url} with urllib.request.urlopen(url, timeout=20) as resp: body = resp.read(8192).decode("utf-8", "ignore") # BODY capped at ~8 KB return { "STATUS": "ok", "URL": url, "TITLE": _extract_title(body), "DESCRIPTION": _extract_description(body), "LINKS": _extract_links(body), "BODY": body, }

Notice what this buys you: even if SearchAgent writes hostile code, the worst it can do over the network is read BBC, 8 KB at a time. The allow-list is host-side and the model never sees it — it cannot be prompt-injected away.

5. Wiring the graph and the deterministic save

from agent_framework import WorkflowBuilder workflow = ( WorkflowBuilder() .add_node("prepare", prepare) .add_node("SearchAgent", search_agent) .add_node("adapt_1", adapt) .add_node("ContentAgent", content_agent) .add_node("adapt_2", adapt) .add_node("GenScriptAgent", genscript_agent) .add_node("save_scripts", save_scripts) # deterministic Executor, NO LLM .build() ) # GenScriptAgent emits text containing two fenced blocks (simplified + # traditional). save_scripts parses them and writes the files itself — # there is no model in the persistence step. await workflow.run() # -> ./outputs/<YYMMDD>/<YYMMDD>.simple.zh.txt # -> ./outputs/<YYMMDD>/<YYMMDD>.tranditional.zh.txt

6. The payoff

Run that shutil.rmtree("/") inside this pipeline now and the result is delightfully boring: the agent deletes its own throwaway sandbox, the host never notices, and the next execute_code starts from a clean snapshot. Two things to call out:

Snapshot/restore means every code execution starts from a clean, reusable baseline — capture state once, rewind between calls, instead of rebuilding the whole VM. For a daily pipeline that runs the act-observe-correct loop many times, that is the difference between "fast enough to always use" and "slow enough to skip."
Because each agent writes one script instead of ten round-tripped tool calls, the CodeAct approach keeps both latency and token usage down — the model reasons once and lets the guest do the busywork behind the boundary.

Where it fits, and the one idea to keep

harnessagent_sandbox_demo lives inside Multi-AI-Agents-Cloud-Native — a gallery of patterns for running agent systems safely on Azure: A2A multi-agent orchestration, the Kubernetes sidecar pattern, hardened pipelines, and a sibling sample that runs Copilot agents on AKS inside Kata Containers MicroVMs at the pod level.

And the README is explicit that this design is cloud-native: running it in-cluster on AKS changes nothing about the architecture — the same WorkflowBuilder graph, the same Hyperlight sandbox, the same deterministic save_scripts executor. The local build and the in-cluster build are the same shape.

The two MicroVM samples are two ends of one spectrum. The Kata sample puts the boundary around the whole pod — a deployment topology. This Hyperlight demo pulls the boundary all the way into the agent process itself — the sandbox becomes a library call. Same question — where do you place the hardware boundary in an agent stack? — answered at two different altitudes.

The old pitch for sandboxing always carried an asterisk: yes, it's safer, but you'll pay in speed, compatibility, or friction. MicroVMs erase the asterisk — VM-grade isolation, cold starts fast enough that there's no reason to skip it, and a real environment your agents can actually work in. Enough of a real environment, in fact, to write you a World Cup podcast every morning.

The one idea to internalize: the harness decides, the MicroVM contains. Give your agent a room where it is allowed to fail — then let it be brilliant.

References

Project: harnessagent_sandbox_demo · Multi-AI-Agents-Cloud-Native
Hyperlight: hyperlight-dev/hyperlight · hyperlight-dev/hyperlight-sandbox
Agent Framework: Agent Harness in Microsoft Agent Framework
Background: Why MicroVMs (Docker) · Harness vs. Scaffold glossary (Hugging Face)
Install: pip install agent-framework-hyperlight --pre · .NET: dotnet add package Microsoft.Agents.AI.Hyperlight --prerelease
Requirements: KVM (Linux) or WHP (Windows); macOS not yet supported.

Foundry Toolkit for VS Code at //build: Hosted Agents End-to-End, a Smarter Toolbox, and More

leoyao — Thu, 04 Jun 2026 07:00:00 GMT

We’re excited to share what’s new for Foundry Toolkit for Visual Studio Code at //build 2026. Since going generally available, the toolkit has kept moving fast, and this release is a big one. The headline: a complete, end-to-end Hosted Agent experience, scaffold, run, deploy, and observe without ever leaving VS Code. On top of that, we’ve expanded the Toolbox with native enterprise integrations and shipped a wave of LangGraph samples so every developer has a clear path from idea to production. From your first prompt to a production-grade, observable agent, Foundry Toolkit meets you where you are.

Hosted Agents, End to End

Building an agent is the easy part; getting it from a first draft to a production-grade, observable service is what matters. This release makes the full Hosted Agent lifecycle available in VS Code, and it follows the way you actually work — scaffold, run, deploy, observe.

Scaffold — start from a rich set of samples

Hosted Agent creation now opens with a refreshed scaffolding experience and a rich sample selection, so you start from a working, framework-appropriate template instead of a blank file. Creation is smarter, too: we auto-select your subscription when there’s only one, gate tabs more clearly, and tightened spacing for a cleaner setup flow.

New Hosted Agent scaffolding dialog with the rich sample picker open

Run (F5) — inspect as you build

Press F5 and your agent runs locally with the Agent Inspector, now aligned with the rest of the extension and featuring Copilot SDK visualization so you can see what the Inspector visualizes as the agent executes. It’s the fastest loop from change to verification before anything leaves your machine.

Deploy — a new UX and new ways to ship

Different teams ship differently, so deployment got a refreshed UX and two new options for Hosted Agents:

ZIP Code Deploy: Package your agent source as a ZIP and deploy it directly to Microsoft Foundry Agent Service.

Bring-Your-Own-Image (BYOI): Already have a pre-built container in your own Azure Container Registry? Deploy straight from it.

Hosted Agent deploy dialog showing "ZIP code deploy" and "Bring-your-own-image (ACR)" side by side.

Observe — know it works in production

Once deployed, the full observability story is now available:

Hosted Agent Tracing: Inspect end-to-end traces of Hosted Agent invocations directly from VS Code — tool calls, delegation chains, and timing for real debugging instead of guesswork.

Continuous Evaluation Settings: A new page to configure ongoing evaluation for deployed Hosted Agents, so quality is measured continuously — not just at ship time.

Evaluations Node: One-click access to evaluation runs and results right from the Foundry project tree.

Hosted Agent trace view showing a span tree of tool calls and timings.

A Smarter, More Connected Toolbox

What it is, and why it matters

A Toolbox is how your agent gets its capabilities — the curated set of tools, knowledge sources, and integrations it can call at runtime. Instead of hand-wiring each connection, you assemble a Toolbox once and your agent consumes it consistently across local runs and production. The result: agents that can act on real enterprise data and systems, with the connections managed in one place.

From what to how: create, connect, consume

Create: Start a new Toolbox from the Foundry Toolkit sidebar “Tools Catalog” and pick the capabilities your agent needs.

Connect: Configure and wire in enterprise systems through native, first-class connections once, and use it for all your agents.

Consume: Reference the Toolbox from your Hosted Agent so its tools are available the moment the agent runs, locally (F5) and once deployed.

New this release

Building on that flow, the Toolbox is now richer and more enterprise-ready:

WorkIQ as a Built-in Tool: A first-class WorkIQ experience powered by A2A connections — no MCP fallback required. End-to-end toolbox creation with WorkIQ works out of the box.

Fabric IQ (OneLake Catalog) Integration: Connect your agents to Microsoft Fabric OneLake catalogs directly from the Toolbox.

Toolbox Guardrails: Apply content-safety guardrails to your Toolbox for safer agent execution.

Faster discovery: A new Toolbox Search Toggle and Agent Tool Multi-Select let you find and wire in multiple tools in a single action.

Redesigned Tools Catalog, including WorkIQ and Fabric IQ tiles.

LangGraph Reaches Parity

LangGraph developers, this one is for you. We’ve added five new Hosted Agent samples that bring LangGraph to full parity with the Agent Framework Responses learning path — so you get an equivalent, end-to-end walkthrough no matter which framework you prefer:

MCP — tool loading from a remote MCP server (defaults to GitHub Copilot MCP) via MultiServerMCPClient.

Workflows — a custom StateGraph chaining three specialized LLM nodes: slogan writer, legal reviewer, and formatter.

Files — local filesystem tools plus the Foundry-Toolbox code_interpreter working over session-uploaded files.

Human-in-the-Loop — a StateGraph that drafts a proposal and pauses for approval via langgraph.types.interrupt.

Observability — GenAI OpenTelemetry tracing with enable_auto_tracing(); spans, metrics, and logs flow to Application Insights.

We’ve also refreshed the existing bring-your-own LangGraph samples against the new hosting layer (chat with local tools, Foundry-managed Toolbox loading, and SSE-streamed multi-turn sessions backed by a MemorySaver checkpointer), so every sample reflects how Hosted Agents work today.

Workflow visualization of the LangGraph human-in-the-loop sample paused at an approval node.

Polish Across the Board

A release is more than headline features. This one also includes a redesigned Prompt Builder “Improve an Instruction” dialog for faster iteration, fixes for MCP toolbox tool icons, clearer ZIP-deploy error surfacing, and assorted Agent Builder and Playground regression fixes — the whole experience feels tighter end to end.

Get Started Today

Install: Foundry Toolkit on the VS Code Marketplace

Quick Start: Follow our getting-started tutorial to build your first Hosted Agent

Deep Dive: Explore the documentation, samples, and LangGraph parity walkthroughs

Join the Community

Share your projects, file issues, or suggest features on our GitHub repository. We can’t wait to see what you build.

Welcome to the next chapter of AI development!

DevOps for Microsoft Hosted Agents: From Terraform Apply to Production-Grade Agent Delivery

LZhang — Wed, 03 Jun 2026 19:34:50 GMT

A companion piece to Infrastructure as Code for AI: Building and Deploying Microsoft Hosted Agents with Terraform.

Just announced — source-code deploy (preview). Foundry has just added a second Hosted Agent deploy path alongside the container path this post covers. Instead of a container image, you upload a .zip of your source plus a requirements.txt (Python 3.13 / 3.14) or a .csproj (.NET 10), and the Agent Service either builds dependencies for you (remote_build) or runs a prebuilt bundle (bundled). The version definition uses code_configuration instead of container_configuration — the two are mutually exclusive on a given version. Versioning is content-addressable on the zip's SHA-256, so the dedup behaviour described below still applies. Required roles shift slightly: deploying the agent needs Foundry Project Manager at project scope, and the platform-assigned agent identity gets Foundry User (both handled automatically by azd and the Foundry VS Code Toolkit). The DevOps loop in this post — immutable versions, eval gating, manifest-driven promotion, traffic-split canary, per-version observability — transfers directly; only the build-and-push stage changes (no Dockerfile, no ACR for remote_build). The container path covered here remains fully supported and is still the right choice if you need custom base images, system packages, or non-Python/.NET runtimes. Full details: Deploy a hosted agent from source code (preview).

What this post assumes. It describes recommended enterprise DevOps patterns on top of Microsoft Foundry Hosted Agents. Some patterns — evaluation gating, traffic-based rollout, manifest-driven promotion — are best practices and may not be enforced by the platform itself. Hosted Agents and several related capabilities (A2A, certain deployment and routing controls) are in preview and may evolve.

TL;DR

Terraform provisions the platform: Foundry account, project, model deployment, ACR, App Insights, RBAC.

DevOps pipelines ship agent versions, not source branches — the deploy artifact is a container image digest plus an immutable version spec.

Evaluation should be treated as a release gate, not a dashboard. Quality regressions should fail the build the same way unit-test failures do.

Traffic split between versions is the rollout and rollback primitive. Rollback typically avoids rebuilding or redeploying artifacts.

Observability is sliced per version — during canary, two versions serve simultaneously and aggregate metrics lie.

The Delivery Pipeline at a Glance

  Terraform  ───►  Foundry project (AIServices) + model deployment + ACR + App Insights
                              │
   PR opened                  ▼
      └─►  docker build  ───►  push to ACR  ───►  capture image digest
                                                              │
                                                              ▼
                                            Foundry SDK: create agent version
                                            (image digest + cpu/mem + env + protocols)
                                                              │
                                                              ▼
                                                   Evaluation gate  ────►  fail → stop
                                                              │
                                                              ▼ pass
                                          Promote via manifest → staging → prod
                                                              │
                                                              ▼
                                       Traffic-split canary (0% → 10% → 100%)
                                                              │
                                                              ▼
                                       App Insights: per-version latency, cost,
                                       sampled quality, sandbox sizing

Infrastructure as Code gets the platform stood up. It does not, on its own, ship an agent. The gap between terraform apply succeeding and a customer-facing agent reliably serving requests in production is where DevOps lives — and for Microsoft Hosted Agents on Microsoft Foundry, that gap has its own shape.

A Hosted Agent is not a prompt and a tool list. It is your own code, packaged as a container image, pushed to Azure Container Registry, and deployed to a Foundry project. The Foundry Agent Service pulls the image, provisions an isolated execution environment per agent session, assigns the agent its own dedicated Microsoft Entra ID (agent identity), and exposes a dedicated endpoint. An agent supports up to four protocols, any of which can be combined in a single deployment:

Responses (.../protocols/openai/responses) — OpenAI-compatible chat-style API. Implemented in the container.
Invocations (.../protocols/invocations) — arbitrary JSON in / arbitrary JSON out for webhook receivers and non-conversational workloads. Implemented in the container.
A2A (.../protocols/a2a, preview) — the open Agent2Agent protocol for agent-to-agent delegation across frameworks and vendors. Surfaced on its own endpoint path by the platform.
Activity — the Teams / M365 channel protocol. The platform bridges Responses to Activity automatically when an agent is published to a Microsoft 365 channel.

Microsoft manages the runtime, scaling, session state, and lifecycle. You ship the image and the version definition.

Important — Foundry version compatibility. Hosted Agents are supported on the new Microsoft Foundry project resource model (azurerm_cognitive_account_project under a Cognitive Services account of kind = "AIServices"). The older Azure AI Foundry Hub model (azurerm_ai_foundry / azurerm_ai_foundry_project, kind = "Hub") — the Azure ML–derived workspace surface — does not expose Hosted Agent capabilities. They are two distinct Azure resource types with different APIs. Everything in this post assumes the new Foundry project.

That shape drives three things every DevOps loop for Hosted Agents has to handle:

The deploy artifact is a container image plus an immutable agent version. A version snapshots the image digest, CPU/memory, environment variables, and protocol configuration. To change anything, you create a new version. The platform supports weighted traffic between versions, which is your blue/green and canary primitive.
The agent identity is created for you, per agent. You don't pick one or wire managed-identity references manually. Each agent is assigned a dedicated Microsoft Entra ID (agent identity) at deploy time; RBAC to downstream resources is granted to that identity.
Quality is non-deterministic. Two terraform apply runs against the same configuration produce identical resources. Two agent runs against the same input can produce different outputs. Your pipeline has to gate on evaluation, not only on tests passing and HTTP 200s.

This post lays out an end-to-end DevOps loop on top of that shape: how to structure the repository, what runs in CI versus CD, how to gate releases on evaluation, how to promote across environments, how to use version traffic split for safe rollouts and instant rollback, and what observability is worth wiring beyond the defaults.

A Quick Tour of Microsoft Foundry

If you've spent more time in Azure OpenAI or AI Studio than in Foundry, a short orientation helps before the DevOps patterns make sense.

Microsoft Foundry is Microsoft's unified platform for building, evaluating, deploying, and operating AI applications and agents. It consolidates what used to be spread across Azure OpenAI, Azure AI Studio, and the AI Hub model into a single resource and a single portal at ai.azure.com. Three pieces are worth knowing up front.

The resource model

Foundry is built on two Azure resources:

Foundry account — an azurerm_cognitive_account with kind = "AIServices", project_management_enabled = true, a custom_subdomain_name, and a managed identity. This is the top-level container: it holds your model deployments (Azure OpenAI and the broader Foundry model catalog), connections to backing services, and the Foundry-managed Toolbox MCP endpoint.
Foundry project — an azurerm_cognitive_account_project under that account. A project is the scope for agents, evaluations, conversation history, indexes, and per-app connections. One project per app or per environment is the usual shape.

This is the new Foundry model — and it is the only model that supports Hosted Agents. The older Azure AI Foundry Hub (azurerm_ai_foundry + azurerm_ai_foundry_project, kind = "Hub") is a separate Azure ML–derived workspace and cannot host Hosted Agents. The two surfaces look superficially similar in the portal but are distinct Azure resource types with different APIs and feature sets. If a tutorial, sample, or piece of Terraform you find online creates an azurerm_ai_foundry Hub, it is targeting the classic surface and the Hosted Agents APIs (/agents, agent versions, traffic split, dedicated endpoints) will not be available against it. To use Hosted Agents you must provision a new Foundry account + project as described above. There is no in-place upgrade from a Hub.

What Foundry gives you

A Foundry project is more than a container. Out of the box it provides:

A model catalog and deployment surface — Azure OpenAI models (GPT-4.1, GPT-4o, o-series, embeddings), plus open and partner models, all deployed and invoked through the same project endpoint with the same auth model.
Two agent execution modes — prompt-based agents (defined entirely by instructions + tool configuration in the portal, suitable for conversational assistants) and Hosted Agents (your own containerized code, the subject of this post).
A managed Toolbox — a project-level MCP endpoint that exposes Foundry-curated tools (Code Interpreter, Web Search, Azure AI Search, OpenAPI, custom MCP, A2A) with consolidated auth. Hosted Agent code connects to the Toolbox using standard MCP client libraries.
First-class evaluation — datasets, graders (similarity, LLM-as-judge, safety, groundedness), and evaluation runs as a built-in concept, not a bolt-on.
Built-in tracing — OpenTelemetry traces from agents land in a linked Application Insights resource automatically. No manual instrumentation needed to get the basics.
Per-agent identity — when you deploy a Hosted Agent, the platform creates a dedicated Microsoft Entra ID (agent identity) for it and gives it a dedicated endpoint. RBAC to downstream resources is granted to that identity.

How the pieces line up for Hosted Agents

For the rest of this post, the mental model is:

Resource group
└── Foundry account (Cognitive Services, kind=AIServices)
    ├── Model deployments (e.g. gpt-4.1)
    └── Foundry project
        ├── Hosted Agent: customer-support
        │   ├── Version v1 (image digest A, 100% traffic)
        │   └── Version v2 (image digest B, 0% traffic — canary)
        ├── Hosted Agent: webhook-handler
        ├── Evaluations
        ├── Connections (ACR, AI Search, Key Vault…)
        └── Toolbox (MCP)

Terraform provisions the account, project, model deployments, ACR, App Insights, and RBAC. Hosted Agents — images, versions, traffic weights — are managed through azd or the Foundry SDK. That boundary is what the rest of this post automates.

The minimal Terraform shape

For Hosted Agents you need the new-model shape instead. The skeleton below is the minimum that lets you deploy a Hosted Agent on top of it — storage, Key Vault, monitoring, networking, and OIDC for CI live alongside for more details see Infrastructure as Code for AI: Building and Deploying Microsoft Hosted Agents with Terraform | Microsoft Community Hub.

# Foundry account (new model — required for Hosted Agents)
resource "azurerm_cognitive_account" "foundry" {
  name                       = "ai-${local.name}"
  resource_group_name        = azurerm_resource_group.main.name
  location                   = azurerm_resource_group.main.location
  kind                       = "AIServices"
  sku_name                   = "S0"
  project_management_enabled = true
  custom_subdomain_name      = "ai-${local.name}"   # required for AAD auth

  identity {
    type = "SystemAssigned"
  }
}

# Model deployment the agent will call
resource "azurerm_cognitive_deployment" "gpt" {
  name                 = "gpt-4.1"   # stable name — agents pin to this
  cognitive_account_id = azurerm_cognitive_account.foundry.id

  model {
    format  = "OpenAI"
    name    = "gpt-4.1"
    version = "2025-04-14"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 10
  }
}

# Foundry project — the scope for Hosted Agents, evals, conversations
resource "azurerm_cognitive_account_project" "main" {
  name                 = "proj-${local.name}"
  cognitive_account_id = azurerm_cognitive_account.foundry.id
  location             = azurerm_resource_group.main.location

  identity {
    type = "SystemAssigned"
  }
}

# Container registry the agent image is pushed to and pulled from
resource "azurerm_container_registry" "acr" {
  name                = "acr${replace(local.name, "-", "")}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  sku                 = "Standard"
  admin_enabled       = false   # use RBAC, not admin user
}

# The project's managed identity needs to pull the agent image
resource "azurerm_role_assignment" "project_acr_pull" {
  scope                = azurerm_container_registry.acr.id
  role_definition_name = "AcrPull"   # use Container Registry Repository Reader if the ACR has ABAC enabled
  principal_id         = azurerm_cognitive_account_project.main.identity[0].principal_id
}

A few things worth calling out:

kind = "AIServices" + project_management_enabled = true + custom_subdomain_name are what make this a new-model Foundry account. Omit project_management_enabled and azurerm_cognitive_account_project will not provision; omit custom_subdomain_name and you lose the Foundry endpoint shape that Entra-authenticated access depends on.
azurerm_cognitive_account_project is the new-Foundry project resource. Do not use azurerm_ai_foundry_project — that targets the Hub model and does not host agents.
Keep the model deployment name stable. Agent code (and your agent.yaml) pins to the deployment name, not the model version. Changing the version is safe; changing the name forces a new agent version.
The project MI needs ACR pull, not push. CI pushes the image (via its own identity); the platform pulls it on the project's behalf when the agent runs.
ABAC-enabled ACR is supported but requires --source-acr-auth-id [caller] on az acr build in your CI script — a common gotcha.
A note on the provider. Everything above uses the hashicorp/azurerm provider. Foundry's surface evolves quickly, and you will occasionally hit a property or child resource that AzureRM hasn't caught up with yet — project connections, capability hosts, and some newer agent-related fields are common examples. When that happens, reach for azure/azapi: use azapi_update_resource to patch a missing property on an AzureRM-owned resource, and azapi_resource for resources AzureRM doesn't model at all. Keep AzureRM as the default and use AzAPI as a targeted gap-filler, so you don't fork ownership of mainstream resources.

The Hosted Agent Delivery Loop

A working delivery loop has five stages. Each maps to a specific artifact, a specific tool, and a specific failure mode.

Stage	Artifact	Tool	Primary failure mode
Infra provisioning	Terraform state	`terraform apply`	Quota, RBAC propagation, ACR not reachable
Image build & push	OCI image in ACR (ACR must remain publicly reachable today)	`docker build` / `az acr build`	Image too large, base image CVEs
Agent version create	Immutable version (image digest + config)	`azd` or Foundry SDK	Bad env var, wrong protocol declared
Evaluation	Eval dataset + grader	Foundry evaluators	Quality / safety regression
Traffic shift & observe	Version weights, App Insights traces	Foundry SDK + Azure Monitor	Silent quality decay, sandbox over/under-sizing

The first stage is where the prior post left off. The remaining four are this post.

Infra provisioning assumes the standard pattern: terraform plan runs on every PR as a review gate (posted as a PR comment) and terraform apply runs only on merge to the environment branch. Everything below assumes the platform is already applied.

Repository Shape

A repository that supports the loop end-to-end looks roughly like this:

agent-platform/
├── infra/                          # Terraform from the prior post (AIServices + project)
│   ├── modules/foundry-project/
│   └── environments/
│       ├── dev.tfvars
│       ├── staging.tfvars
│       └── prod.tfvars
├── agents/
│   ├── customer-support/
│   │   ├── Dockerfile
│   │   ├── src/                    # Agent code (Python or C#)
│   │   ├── agent.yaml              # Version spec: image, cpu/memory, protocols, env
│   │   ├── evals/
│   │   │   ├── dataset.jsonl
│   │   │   └── graders.yaml
│   │   └── README.md
│   └── webhook-handler/
│       └── ...
├── scripts/
│   ├── deploy_agent_version.py     # Build → push → create version → optional weight shift
│   ├── run_evals.py
│   └── promote_version.py          # Shifts traffic between versions
└── .github/workflows/
    ├── infra.yml                   # Terraform plan/apply
    ├── agent-pr.yml                # Build, push to ACR, deploy candidate version, run evals
    └── agent-release.yml           # Promote a tested version to staging / prod

Two deliberate choices. First, infrastructure and agents live in the same repo but in separate top-level directories with separate pipelines. They have different cadences and different reviewers. Second, each agent is its own folder with its own Dockerfile, code, version spec, and eval suite. A single PR touches one agent's directory cleanly; a code-review diff stays focused.

The Agent Version as the Deploy Unit

A Hosted Agent is deployed as a version. A version is immutable — once created it captures:

the container image digest (not just the tag — the digest, so it cannot drift),
CPU and memory allocation for the per-session sandbox (e.g. 1 vCPU / 2 GiB),
the container protocols the image implements — responses, invocations, or both,
environment variables passed to the container at runtime,
any other version-scoped configuration (e.g. base model deployment name).

The container's container_protocol_versions only declares responses and/or invocations — the two protocols the container itself implements. A2A (preview) is surfaced by the platform on its own endpoint path, and Activity is bridged from Responses automatically when the agent is published to a Microsoft 365 channel. Under the hood, agent versions run on Azure Container Apps with VM-isolated sandboxes, which is also why you may see the term revision in some Container Apps–surfaced APIs and limits — a Hosted Agent version corresponds to one such revision.

To change any of those, you create a new version. The platform keeps the old one and shifts traffic between them by weight. This is the primitive you use for canary rollouts and for rollback — both reduce to a traffic-weight change, not a redeploy.

An agent.yaml per agent makes the version reproducible from source:

# agents/customer-support/agent.yaml
name: customer-support
container:
  image: ${ACR_LOGIN_SERVER}/customer-support  # digest resolved at deploy time
  cpu: 1
  memory: 2Gi
protocols:                                 # container_protocol_versions
  - responses
  # add `invocations` here if the container also handles webhook-style payloads
env:
  # The platform automatically injects FOUNDRY_PROJECT_ENDPOINT,
  # AZURE_AI_MODEL_DEPLOYMENT_NAME, and APPLICATIONINSIGHTS_CONNECTION_STRING
  # — you only set what's specific to your agent.
  LOG_LEVEL: info
metadata:
  owner: support-team
  source_commit: ${GITHUB_SHA}

scripts/deploy_agent_version.py is the executable form of this spec. Its job per agent is:

Build the container image (docker build locally, or az acr build server-side for ABAC ACRs).
Push to ACR and capture the resulting image digest — not the :latest tag.
Resolve environment variables from the target environment's config.
Call the Foundry SDK to create a new agent version pinned to that digest.
Emit a deployment-manifest.json containing the agent name, version ID, image digest, source commit SHA, and the eval dataset hash used.

One gotcha: the platform deduplicates. A create version call with no change to the version parameters (same image digest, same env, same CPU/memory, same protocols) will not produce a new version object. Write the script to treat "no new version returned" as success and reuse the existing version ID in the manifest, not as a failure to retry.

That manifest is the cross-pipeline contract. PR pipelines produce one. Promotion pipelines consume one. Rollback consumes a previous one.

Evaluation as a Release Gate

Foundry ships evaluators (datasets, graders, evaluation runs) as a first-class platform feature. Whether to block a release on their results is a team decision, not a platform mandate — but it is the recommended pattern for any agent serving real users. A pipeline that promotes an agent because the image built, the container started, and the version was created with HTTP 200 will eventually ship a regression that an integration test cannot catch. Treat the eval suite the way you treat unit tests: failures stop the pipeline.

A minimal but honest evaluation setup has three pieces.

A reference dataset. Twenty to fifty representative scenarios is enough to start. Each row is an input plus either a reference answer, a set of must-include facts, or a rubric. Store as JSONL alongside the agent:

{"id":"refund-1","input":"How do I get a refund for order 12345?","must_include":["return window","14 days","original payment method"]}
{"id":"escalate-1","input":"This is the third time my package is late.","rubric":"Agent should acknowledge, apologize, offer escalation, not promise compensation."}

Graders. Foundry's evaluators library ships templates — exact match, similarity, LLM-as-judge for rubric scoring, and built-in safety and groundedness graders. Pick what matches your dataset shape. LLM-as-judge is the workhorse for open-ended responses; pin its model deployment explicitly so the grader itself does not drift between runs.

Thresholds. Decide what "passing" means before the first run. A common pattern:

Hard floor on safety / groundedness — any regression fails the build.
Relative threshold on quality — no more than X% drop versus the last known-good version.
Absolute floor on must-include coverage — for example ≥ 90%.

Wire it into the PR pipeline:

# .github/workflows/agent-pr.yml (excerpt)
- name: Build, push, and create candidate version
  run: |
    python scripts/deploy_agent_version.py \
      --agent customer-support \
      --project $EVAL_PROJECT \
      --version-suffix pr-${{ github.event.number }} \
      --traffic 0   # create the version, do not route traffic yet

- name: Run evaluations against candidate endpoint
  run: |
    python scripts/run_evals.py \
      --agent customer-support \
      --version pr-${{ github.event.number }} \
      --baseline last-known-good \
      --fail-on-regression

The PR creates a candidate version with zero traffic weight against a long-lived "eval" Foundry project, runs evaluations against the candidate version's dedicated endpoint, and then deletes the candidate version on PR close. A standing eval project beats a per-PR Foundry project — provisioning a project per PR is slow and adds RBAC overhead that does not earn its keep.

Environment Promotion

Three environments is the floor: dev, staging, prod. Each is its own Foundry project, ideally its own Foundry account in its own resource group. What promotes between them is the image digest and the version spec — not source code, and not "redeploy from main."

A workable model:

dev — every push to a feature branch builds an image and creates a dev version. Loose evaluation thresholds. Used for human poking and end-to-end debugging.
staging — merges to main create a staging version. Full eval suite, strict thresholds. Same sandbox sizing, same env vars, same protocols as prod.
prod — manually approved promotion from staging. Promotion script reads the staging manifest, finds the image digest that passed, and creates the prod version pointed at that exact digest. No rebuild.

The "same digest" rule is the recommended pattern for safe promotion. If staging passed evaluations on customer-support@sha256:abc… running gpt-4.1, prod should get that exact image. Re-building from main in the prod pipeline reintroduces the risk you spent staging trying to eliminate — a different base-image patch level, a different transitive dependency, a different build clock — even though nothing in your source changed.

GitHub Actions environments make the approval concrete:

jobs:
  promote-prod:
    needs: deploy-staging
    environment: production    # requires reviewer approval
    runs-on: ubuntu-latest
    steps:
      - name: Create prod version from staging manifest
        run: |
          python scripts/deploy_agent_version.py \
            --agent customer-support \
            --project $PROD_PROJECT \
            --from-manifest staging-manifest.json \
            --traffic 10   # canary at 10%

The canary weight is the second half of safe promotion: create the prod version, give it a small fraction of traffic, watch the App Insights traces, then shift the rest with promote_version.py.

Traffic-Split Rollout and Instant Rollback

Weighted version traffic changes the rollback model entirely. Rollback typically avoids rebuilding or redeploying artifacts — the previous version is still there, ready to take traffic.

A typical canary flow:

Create new version v42 at 0% traffic. Endpoint exists; no production calls reach it.
Shift to 10%. Observe for an hour or a day, depending on traffic volume.
Shift to 50%, then 100%. Old version stays at 0% but is not deleted.
After a stability window (commonly a week), delete the previous version to free quota.

Rollback is the reverse: shift weights back to the previous version. It is a control-plane call, not a deploy. The agent's endpoint URL does not change, sessions in flight continue on whichever version they started on, and new sessions land on whatever the weights say.

Two consequences worth internalizing:

Keep at least the last two known-good versions live. Rollback is only as fast as your ability to flip weights to a version that already exists.
Do not skip the canary step under deadline pressure. A 0%→100% cutover gives you the same blast radius as a non-canaried deploy. The platform supports incremental rollout; use it.

For a destructive change — a removed protocol, a renamed agent, an env var the previous version cannot tolerate — rollback may not be safe. Forward-fix is the answer. Identify those changes in PR review and require an explicit "rollback path: forward-fix" note in the PR.

Handling Model Version Changes

A model deployment bump is the highest-blast-radius runtime change you can make to a Hosted Agent: the agent's behaviour on every input can shift. Treat it like a dependency upgrade.

Open a PR that changes only the AZURE_AI_MODEL_DEPLOYMENT_NAME (or the model version on the deployment, via Terraform).
Build a new image if needed, create a new agent version, run the full eval suite at 0% traffic.
Run a larger regression dataset if you have one.
Require a human reviewer who is not the PR author.
Promote through staging, then canary in prod for at least one business day before shifting full traffic.

If the new model is faster or cheaper, the temptation is to skip steps. Don't. A quality regression in prod almost always costs more than a careful upgrade.

The Terraform side is small: openai_model_version is a variable on the azurerm_cognitive_deployment. Terraform recreates the deployment if the version changes. The Hosted Agent picks up the new deployment the next time it calls the model — if you kept the deployment name stable, which is your contract with the agent code. If you change the deployment name as well, the agent needs a new version that knows the new name.

Observability That Actually Tells You Something

The platform injects an Application Insights connection string into every Hosted Agent container as an environment variable. Agents that use the protocol libraries emit OpenTelemetry traces by default. That gives you per-request latency, token counts, tool invocations, and conversation IDs out of the box. That is the floor. Add to it:

Custom span attributes on every request. Agent name, agent version ID, image digest (short), model deployment name. Without these, post-incident analysis cannot tell you which version was live when a problem started — especially during a traffic-split rollout where two versions are serving simultaneously.
Quality signal capture. Sample a percentage of production conversations into a queue for offline grading. Run the same graders you used in CI against that sample on a schedule. This is your drift detector for response quality.
Sandbox right-sizing signals. Hosted Agents bill on the CPU/memory you allocate per session. Oversizing multiplies cost by your concurrency. Track CPU and available memory inside the sandbox and compare against the version's allocation — if peaks stay below ~50%, the next version should drop a tier; if they push above ~70%, raise it. Right-sizing is a per-version decision because versions are immutable.
Per-version error and latency. Slice every standard metric by version ID. A canary that looks fine in aggregate can be quietly worse than the previous version on specific request shapes.
Cost dimensions. Tag traces with customer_id or tenant_id if you have multi-tenancy. Aggregating session cost by tenant in App Insights is straightforward once the dimension is on the span.
Alerts on shape, not just rate. A doubling in average response length or a sudden drop in tool invocation frequency often precedes a quality regression that error-rate alerts will miss entirely.

A weekly "agent health" report in your team channel — pulling these App Insights queries together — beats a perfect dashboard nobody opens.

A Pragmatic Maturity Path

Most teams cannot build the whole loop on day one. A reasonable order:

Infrastructure in Terraform. AIServices account, project, model deployment, ACR, App Insights, role assignment so the project MI can pull from ACR.
First agent deployed manually with azd. Just to prove the round trip end to end.
agent.yaml plus a deploy script that builds, pushes by digest, and creates a version. One environment.
Three environments with manual promotion by manifest.
A 20-row eval dataset with one grader, run on every PR. Advisory only at first.
Eval as a blocking gate. Thresholds tuned from the advisory phase.
Canary rollout via traffic split. Versions held live for a stability window before deletion.
Production sampling into offline evaluation. Drift detection.
Model version upgrade playbook. Documented, exercised once on a low-risk agent.
Tested rollback via weight shift. The first time you discover a rollback bug should not be during an incident.

Each step is independently useful. Skipping ahead — particularly to step 6 without time in step 5 — produces thresholds that block legitimate changes and erode trust in the pipeline.

Where This Is Heading

The platform is moving. A few things to watch as you build:

Declarative Hosted Agent versions in Terraform. AzureRM coverage of Hosted Agents and agent versions is expanding. Parts of the deploy script will collapse into Terraform as that lands. The script-driven approach in this post is the bridge, not the destination.
Continuous evaluation as a first-class platform feature. Sampling production traffic into scheduled evals — what you wire by hand today — is moving into the Foundry control plane.
Multi-agent composition over A2A. As the A2A endpoint moves from preview to general availability and more frameworks ship A2A clients, multi-agent workflows become a first-class deployment shape. The DevOps loop extends — version pinning between agents, eval at the workflow level, observability across the agent graph — but the manifest grows accordingly.
Toolbox-managed tool surfaces. As more tool integrations move behind the project Toolbox MCP endpoint, the agent image gets smaller and the tool configuration becomes a project-level concern. That changes what belongs in agent.yaml versus what belongs in Terraform.

The throughline: the more the platform absorbs, the more your job shifts from wiring plumbing to defining policy. What "good" means for your agent, what the quality floor is, who can approve a model upgrade, how fast you can roll back. Those decisions do not get automated away. The pipeline just makes them executable.

Conclusion

Terraform provisions the Foundry project, model deployment, ACR, and observability. The DevOps loop on top of it — container builds pinned by digest, immutable agent versions, evaluation as a release gate, manifest-driven promotion across environments, traffic-split canary and rollback, and observability sliced by version — gets Hosted Agents to production and keeps them there.

Build it incrementally. Treat the image digest and the version spec as the deploy artifact, not the source branch. Make evaluation a check the pipeline cares about. Use version weights as your rollout and rollback primitive. And design for the day the platform absorbs the next layer of plumbing, so that when it does, your work moves up the stack instead of getting thrown away.

Token economics–driven architecture: hybrid models, AI Runway, AKS Kata MicroVM, MCP

kinfey — Wed, 03 Jun 2026 07:00:00 GMT

1. The moment the bill arrived

For most of 2024 and 2025, "Agents" were a demo word. In 2026 they are a line item on the cloud invoice.

Every major model provider — OpenAI, Anthropic, Google, Mistral, DeepSeek, and even the in-cluster open-weights serving stacks — now bills by the token. Input tokens, output tokens, cached tokens, reasoning tokens, tool-call tokens. The unit price has come down. The number of tokens an autonomous agent burns through has gone up by an order of magnitude.

The slide deck I keep coming back to is module 02 of the Enterprise Agent Workshop — Token Economics and Cost Control. The short version: an agentic system is not a chat app. A chat app emits one model call per user turn. An agent emits a model call to plan, another to pick a tool, another to interpret the tool result, another to decide the next step, and another to summarize — and then it loops. Multiply by tools that themselves invoke models. Multiply again by retries and reflection.

The bill is no longer "what does the model cost per million tokens." The bill is "what does my architecture cost per user request."

This post is about an architecture that answers that question on purpose — and that does it without giving up the security properties an enterprise actually needs. The blueprint lives in this repo, BYOT_Dev: a four-agent SDLC tower (Requirements → Code → Test → Deploy) running on AKS, each agent boxed inside its own Kata MicroVM, each one exposing tools to GitHub Copilot Chat over the Model Context Protocol, and all of them sharing a single on-cluster small-language-model endpoint served by AI Runway

2. Why agentic workloads inflate the token bill

Three forces compound:

Autonomy multiplies call count. A user typing "build me a URL shortener" produces one prompt at the IDE. By the time a 4-agent pipeline has clarified requirements, generated code, written tests, and produced a Kubernetes manifest, you have spent 30–200 model calls — most of them invisible to the user.
Reasoning eats output tokens. Modern reasoning models think before they speak. That hidden chain-of-thought is billed. A 5-line answer might charge you for 3,000 reasoning tokens.
Context inflation. Every tool result is re-injected into the next call. A 50 KB code review answer becomes the context of the next refactor turn. Costs grow super-linearly with conversation depth.

You can't out-prompt-engineer this. The only durable mitigation is architectural — and it has three levers:

Lever	What it means in practice
Model tiering	Use a small, cheap model for narrow tasks; reserve the frontier model for orchestration and judgement.
Placement tiering	Place each model where it's cheapest to run: on-cluster CPU for tiny SLMs, on-cluster GPU for mid-size models, cloud APIs for frontier reasoning.
Protocol tiering	Use a standard like MCP so the expensive orchestrator can hand off subtasks to the cheap workers without lock-in.

The architecture this post describes pulls all three levers at once.

3. The mental model: frontier brain, small-model hands

Look at this picture:

spec: image: ghcr.io/kaito-project/aikit/llama3.2:1b model: { id: "kaito/llama3.2-1b", source: huggingface } engine: { type: llamacpp } provider: name: kaito overrides: resource: instanceType: Standard_D4s_v3 preferredNodes: ["aks-nodepool1-21523631-vmss000001"] nodeSelector: { agentpool: nodepool1 } resources: { cpu: "2", memory: "4Gi" } scaling: { replicas: 1 }

AI Runway then takes care of:

selecting the engine (llamacpp for CPU, vllm or dynamo for GPU);
selecting the provider (kaito today, others coming);
pulling the model image from the AIKit catalog;
exposing an OpenAI-compatible Service at http://llama3-2-1b-cpu.airunway-models.svc:80/v1.

A note on the CPU-only example. This repo deliberately uses CPU + Llama-3.2-1B to prove the architecture can run on the cheapest node SKU available. In production you should not assume CPU is always right. The right answer is scenario-driven:

Scenario	Suggested placement
High-volume, narrow, latency-tolerant task (e.g. "expand a requirement into bullet points")	On-cluster CPU SLM (1B–3B) — what this repo demonstrates
Code generation, refactoring, multi-file reasoning	On-cluster GPU mid-model (7B–14B) via KAITO vllm, on an AKS GPU pool that auto-scales from zero
Privacy-sensitive enterprise data, must not leave the cluster	On-cluster GPU, possibly with confidential compute
Frontier reasoning, planning, judging tool output	The Copilot seat's already-included frontier model, called sparingly via MCP — not a second pay-per-token endpoint you have to provision

AI Runway makes that choice a YAML edit, not a refactor. The point of the abstraction is optionality — the right to change your mind about token economics quarter by quarter without rewriting agents.

5. Hybrid scaling: all inference on AKS, planning on the Copilot tokens you already pay for

The single biggest token-economics mistake an enterprise can make right now is treating model placement as a binary — "all in the cluster" or "all on a pay-per-token cloud API." Real workloads are neither. The pattern that actually saves money has two ingredients, and both of them are already on your invoice:

AKS that you already provisioned. A small CPU node pool for the steady-state workload, plus a GPU node pool that scales from zero when the small pool can't keep up. Same cluster, same Kata isolation, one invoice line.
The Copilot seat the developer already pays for. Copilot Chat's frontier model has its own token allowance baked into the seat. Use that allowance — not a separately provisioned cloud inference endpoint — to do the planning that drives the cheap AKS workers via MCP.

That is the whole "hybrid." No external Foundry endpoint, no second per-token meter for inference. Just AKS capacity that grows when you need it + a frontier brain you already pay for.

The agent traffic split is roughly:

~85% of agent calls are short, narrow, predictable — "expand this requirement", "format this YAML", "summarize this diff". A 1B–3B model on a CPU node answers these in seconds; the bill is the node, not the token.
~15% are heavier — multi-file refactors, long-context reasoning, the 400-line FastAPI generation. They need a 7B–14B model on a GPU.
Planning and judgement on top of all of it are done by the Copilot seat's frontier model, which the user is paying for whether you build BYOT or not.

Lever A — a GPU node pool on the same AKS cluster, scaled 0 → N

Keep the always-on tiny-cpu ModelDeployment for steady state. Add a second AI Runway ModelDeployment for the mid model on a GPU node pool that is created at size zero and managed by the AKS Cluster Autoscaler

az aks nodepool add \ --cluster-name $CLUSTER --resource-group $RG \ --name gpupool \ --node-vm-size Standard_NC24ads_A100_v4 \ --node-count 0 --min-count 0 --max-count 4 \ --enable-cluster-autoscaler \ --node-taints sku=gpu:NoSchedule \ --workload-runtime KataVmIsolation

# airunway/modeldeployment-mid-gpu.yaml (sketch) spec: image: ghcr.io/kaito-project/aikit/qwen2.5:7b engine: { type: vllm } provider: name: kaito overrides: resource: instanceType: Standard_NC24ads_A100_v4 nodeSelector: { agentpool: gpupool } tolerations: [{ key: sku, operator: Equal, value: gpu, effect: NoSchedule }] resources: { cpu: "4", memory: "32Gi", nvidia.com/gpu: "1" } scaling: { replicas: 0, maxReplicas: 4 }

The key trick is replicas: 0 plus an autoscaler min-count 0. When nobody is asking the mid model anything, no GPU node is running and no GPU node is billed. The first request causes AI Runway to scale to 1, which triggers the Cluster Autoscaler to provision a GPU node, which gets scheduled with Kata Pod Sandboxing intact. When traffic dies down, both the replica and the node go back to zero. All of this is inside AKS — the agents never leave the cluster to find a GPU.

Lever B — reuse the Copilot frontier tokens you already pay for

This is the lever most token-cost writeups miss. Every developer using BYOT already has a Copilot seat. That seat carries a frontier-model token allowance which Copilot Chat consumes the moment the user types into the chat. The orchestration loop in docs/workflow.md — "plan which tool to call next, read the tool's output, summarize the result" — is paid out of that allowance, not out of a new inference endpoint you provision.

This means:

You do not stand up a separate cloud OpenAI / Foundry deployment for "the smart model." The smart model is already on the user's screen.
You do not put a per-token meter on the agent-to-frontier path. The frontier is upstream of your agents — it calls them via MCP, not the other way around.
The only per-token spend the architecture introduces is what Copilot itself charges against the seat, which is independent of how many BYOT agents you stand up.

The net effect: the parts of the workload that are expensive per token (planning, judgement) run on tokens the company already buys; the parts that are cheap to compute (long-form generation) run on AKS compute you already pay for as node hours.

How the agents pick between the two AKS backends

The Agent Framework client in https://github.com/kinfey/Multi-AI-Agents-Cloud-Native/blob/main/code/BYOT_Dev/agents/app/airunway_client.py takes its base_url and model from the ConfigMap. Three strategies, in increasing sophistication:

Per-role static binding. byot-requirements and byot-test (cheap, narrow) get AIRUNWAY_BASE_URL=tiny-cpu. byot-code and byot-deploy (heavier generation) get AIRUNWAY_BASE_URL=mid-gpu. One ConfigMap, one rollout.
Try-then-scale-up inside the tool. Each tool tries tiny-cpu first; if the answer is too short, fails a quality check, or times out, it retries against mid-gpu. The small model handles the easy 85%; the GPU pool handles only the 15% that actually needed it.
AI Gateway in front of both. Put Azure API Management as an AI Gateway in front of the two AI Runway services. The agent talks to one URL; the gateway does semantic caching, token budgeting, and load-aware routing between tiny-cpu and mid-gpu. Both backends remain in your AKS — the gateway only routes.

A back-of-envelope token saving

Assume one Copilot Chat session through the BYOT tower fires 30 model calls at the lower agents. If those 30 went to an external frontier API at, say, $5 / million output tokens with an average 2 K output per call, that is $0.30 / session in additional lower-tier model spend — stacked on top of what Copilot Chat already charges the seat for planning.

With the hybrid AKS + seat-tokens pattern:

25–26 calls (~85%) → tiny-cpu on a CPU node that is already running for the always-on agents → ≈ $0 marginal
4–5 calls (~15%) → mid-gpu, billed as GPU node hours only while AI Runway has scaled up, and amortised across every concurrent BYOT user that lands on the same node → ≈ $0.02–0.05
Planning / judgement → already inside the Copilot seat allowance the developer is paying for → $0 additional

A $0.30-per-session pay-per-token outcome collapses toward ≈ $0.02–0.05 of pure AKS compute, and the GPU bill returns to zero when nobody is asking hard questions. That is the lever. The reason it works is that AI Runway gives the agents a single in-cluster front door, AKS gives the cluster elastic GPU capacity it doesn't pay for while idle, and Copilot Chat brings its own pre-paid frontier brain.

6. Kata MicroVM: the hardware-level helmet for agentic code

Cost is one half of the agentic-workload problem. The other half is what happens inside the box you put the agent in.

Earlier this year I published Giving the Copilot SDK Agent a "hardware-level helmet" using Kata microVM on AKS. The argument, compressed:

A traditional container is an apartment with shared roof — the host Linux kernel. For a hand-written service the tenant is predictable. For an agent, the tenant is the model, deciding at runtime which shell command to run, which file to read, which npx package to install. That's a new threat model. Container namespaces aren't sized for it. You want a dedicated guest kernel per Pod — a microVM.

Kata Containers is the integration layer that gives Kubernetes microVMs. AKS ships it as Pod Sandboxing with the kata-vm-isolation RuntimeClass on top of Hyper-V — created automatically when the node pool is provisioned with --workload-runtime KataVmIsolation.

In BYOT_Dev every agent Pod sets:

spec: runtimeClassName: kata-vm-isolation containers: - name: agent securityContext: runAsNonRoot: true readOnlyRootFilesystem: true capabilities: { drop: ["ALL"] } seccompProfile: { type: RuntimeDefault }

…and AKS does the rest. The Pod boots a real Hyper-V microVM, with its own guest kernel, before the container even starts. Verifying it is one command:

kubectl -n agents exec deploy/byot-requirements -- uname -r # compare with the kernel on the node — they differ → microVM confirmed

The repository also pins one agent per node via podAntiAffinity on kubernetes.io/hostname, so the four agents live on four physically distinct Kata hosts — a model escape in one cannot reach the others through a shared host kernel, because there is no shared host kernel.

The connection to token economics is this: the moment you trust a cheap on-cluster model to run agent loops on real customer code, the security envelope has to be stronger than a normal container, not weaker. Kata is the thing that makes "cheap" and "safe" not a trade-off. And because AKS Pod Sandboxing applies the same way to the CPU pool, the GPU pool, and any future node pool you add for burst, the hybrid placement story above does not weaken the isolation story — every Pod, on every tier, still boots its own guest kernel.

7. MCP: how GitHub Copilot Chat actually drives this tower

The final piece is the protocol. The agents inside the Kata MicroVMs are useless unless something can call them. The "something" the user already has open is GitHub Copilot Chat in VS Code.

The Model Context Protocol is the standard Copilot Chat (and almost every other serious agentic IDE) speaks to remote tool servers. In this repo each role exposes its tools via FastMCP over Streamable HTTP — see agents/app/main.py and the per-role tool sets in agents/app/roles/.

Service exposure is a small but important detail. The repo uses type: LoadBalancer for each role's Service — see k8s/services.yaml — because:

kubectl port-forward does not work against Kata Pods (the listener lives inside the microVM, not in the host sandbox netns);
kubectl proxy works but pins Copilot to localhost and requires a long-running local process;
a LoadBalancer gives each agent a public Azure IP the IDE can hit directly.

Once the four LoadBalancer IPs are in .vscode/mcp.json, Copilot Chat in agent mode sees four MCP servers — byot-requirements, byot-code, byot-test, byot-deploy — and the user can simply say:

"Use the byot tower to take this idea — a URL shortener with click analytics — from requirements through deployment."

What happens under the covers (docs/workflow.md):

Copilot's frontier model plans the sequence. Frontier tokens spent: small, but smart.
It calls byot-requirements.gather_requirements({"idea": "URL shortener…"}) over MCP. No frontier tokens; the cluster-side Llama-3.2-1B does the work.
It calls byot-code.implement_from_requirements({...}). Same — cluster-side small model.
It calls byot-test.generate_test_plan({...}). Same.
It calls byot-deploy.generate_k8s_manifest({...}). Same.
Copilot's frontier model reads the four results and presents a coherent summary to the user. Frontier tokens spent: small.

The expensive model decided what to do five times. The cheap model did the actual long-form generation four times. That is the token-economics win, and the only reason it's possible without lock-in is that MCP is an open standard.

8. Reading the architecture as a budget statement

Translate the picture into a unit-cost table — now with the hybrid tiers explicit:

Layer	Where the cost lives	What controls it
User input + IDE planning	Copilot seat (per-user subscription)	Already paid — flat rate
Frontier orchestration tokens	Copilot seat token allowance — already included, used for MCP planning, no separate endpoint	Number of agent rounds Copilot does
Tool-call traffic	Azure LoadBalancer egress	Negligible at this scale
tiny-cpu inference (steady state, ~85%)	AKS CPU node hours (1× D4s_v3 in this demo)	Replicas, model size, batch size
mid-gpu inference (autoscaled, ~15%)	AKS GPU node hours on the same cluster, only while replicas > 0	Cluster Autoscaler / Karpenter min=0 max=N, scale-to-zero
Hardware isolation	AKS Pod Sandboxing (Kata) — same node hours	Whether you turn it on (you should)
Provider swap-out	AI Runway YAML	A kubectl apply

Three things to notice.

First, most of the per-request variable cost has moved from a token meter to a node meter. CPU hours are easier to forecast, easier to chargeback, and easier to cap than per-call token spend. You know how many D4s_v3 cores you're paying for; you do not know in advance how many tokens a frontier model will decide it needs.

Second, GPU capacity is no longer a fixed bet, and it never leaves AKS. The GPU node pool sits at zero nodes until AI Runway needs it, and when it does, it scales up inside the same cluster under the same Kata RuntimeClass — no second region, no second tenancy, no second per-token bill.

Third, the frontier brain is reused, not re-bought. The planning and judgement that drives the whole tower runs on the Copilot seat token allowance the developer already pays for. There is no separate "smart model" cloud endpoint provisioned by BYOT, so there is no second per-token meter to babysit.

And because Kata Pod Sandboxing is included in AKS and applies the same way on the CPU pool and the GPU pool, the security cost on top of the compute cost is zero.

That is what makes this architecture cost-aware and elastically-scalable and safety-aware at the same time. Those three used to be a trade-off. They no longer are.

9. Six commands, end-to-end

For completeness, the repo's run order (README.md):

# 0. one-time prereqs: az login, kubectl, helm, docker, aks-preview az login # 1. provision AKS with Kata + ACR + AzureLinux bash infra/01-create-aks-kata.sh # 2. install AI Runway controller + KAITO provider (pinned to v0.5.0) bash infra/02-install-airunway.sh # 3. deploy Llama-3.2-1B on CPU via AI Runway ModelDeployment bash infra/03-deploy-qwen.sh # 4. build & push the single agent image to ACR bash infra/04-build-push-agents.sh # 5. deploy the 4 Kata-isolated MCP agents bash infra/05-deploy-agents.sh # 6. print the public MCP endpoints for GitHub Copilot bash infra/06-show-mcp-endpoints.sh

Drop the printed IPs into .vscode/mcp.json, open Copilot Chat, and you have a fully working, hardware-isolated, cost-aware agentic tower talking to a small model on a CPU node — driven by the frontier model the user is already paying a seat for. Add the mid-gpu ModelDeployment on a scale-to-zero GPU node pool alongside it whenever your traffic justifies the next tier; the agents and the Copilot integration don't change, and nothing leaves AKS.

10. Wrapping up: the through-line

Let me trace it one more time:

Token economics is the new SLO. Agentic workloads multiply model calls; every call has a price. Architecture, not prompts, is what bends the curve.
Tier your models, tier your placement. Frontier reasoning at the top; small models for the bulk work; on-cluster CPU for steady state and on-cluster GPU for the heavy 15%.
Mix AKS compute with the Copilot tokens you already pay for. Don't add a second pay-per-token cloud endpoint for inference. The heavy compute belongs on an AKS GPU pool that scales from zero; the planning belongs on the Copilot seat allowance the developer already has. That combination both saves tokens (no new per-token meter) and scales elastically (the cluster grows only when AI Runway asks).
AI Runway makes placement a YAML edit. Today's CPU Llama is tomorrow's GPU Qwen on the same cluster. Same agent code.
Kata MicroVM is non-negotiable for agentic code. The tenant is the model. The roof must be your own. AKS Pod Sandboxing makes it turnkey — and it applies the same way on the CPU pool and the GPU pool.
MCP is the bridge. GitHub Copilot Chat is already an MCP client. Expose the cheap workers as MCP tools and the frontier brain calls them — burning the seat tokens, not new tokens.
The reference build is in this repo. Six commands, four agents, one tiny CPU model, full microVM isolation, real Copilot Chat integration — and a hybrid scaling path you can layer on without changing the agents or leaving AKS.

In the agentic era, a container is not just a box for your application — it is a box for uncertainty and for tokens. The microVM hardens the box; AI Runway lets you slide the model in and out of the box, between CPU and GPU nodes in the same cluster, without rewriting anything; MCP lets the user's expensive IDE drive the cheap box from the outside on tokens already on its tab. That is the through-line.

Build the tower. Watch the bill.

GitHub Action for Deploying Hosted Agents

j_folberth — Wed, 03 Jun 2026 07:00:00 GMT

Introduction

With Microsoft's introduction to Hosted Agents comes a next logical question. How to implement this? Organizations need a method that is quick, repeatable, and requires minimal adjustments to their existing tooling and processes. Thus, we will walk through how to deploy a Hosted Agent through a repeatable GitHub Action.

If this is new to you this blog is a follow up to Deploying Foundry Hosted Agents via REST API | Microsoft Community Hub.

Before You Start

This action assumes the following are already in place in the workflow that calls it:

An existing Microsoft Foundry project with a deployed model.
A container image already pushed to Azure Container Registry (ACR).
An identity with the **Foundry User** role on the Foundry project. See [hosted agent permissions](https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/hosted-agent-permissions) for the full permissions reference.
A runner with `az`, `jq`, and `python3` installed. This is true on `ubuntu-latest`; if you self-host, install them explicitly.
azure/login configured in the caller workflow **before** this action runs.

⚠️ *Identity prerequisite This action assumes `azure/login` has already run in the caller workflow and that the resulting identity holds a Foundry data-plane role (e.g., Foundry User). Without that, `az account get-access-token` will fail before the REST call is made.

Requirements

Grounding ourselves in our requirements to implement the deployment processes, in the quickest way that leverages minimal adjustments and a repeatable process, we will leverage GitHub Action and Bash. The Bash script will take a series of arguments that will be used to call the REST API.

The action requires four inputs: `project_endpoint`, `agent_name`, `image`, and `model_deployment_name`. The example pipeline wires these from the outputs of a preceding IaC step, but the action itself takes plain strings. These strings can come from any tool that can hand them off as workflow inputs. This keeps it flexible and limits adjustments to existing CI/CD processes.

If interested, one can use the Azure Developer CLI (`azd up`) command which is documented via Microsoft official examples and MS Learn. This blog chose not to cover this as the majority of enterprise customers already have tooling they are leveraging other than `azd`.

Also, one could use the `azure.ai.projects` library to create an agent. This blog made the decision not to go down this route as not all organizations have adopted the philosophy of allowing application code to create underlying compute infrastructure. Additionally, some organizations desire teams outside of developers to control and set the size of the Micro VM (referred to as the "sandbox" in the Foundry docs) that the Hosted Agent is running on.

If your organization does not use GitHub Actions this step should be duplicatable in Azure DevOps leveraging the Bash task.

Deployment Steps

For us to do this appropriately let's take a step back and evaluate a CI/CD workflow for an Agent whose definition is stored in a container.

Ideally a pipeline should follow steps outlined in CI/CD for AI Agents on Microsoft Foundry. Those pipelines typically take the shape build/push → IaC → update agent → smoke test. For our purposes, since we are hyper-focusing on the Hosted Agent Deployment via REST API we are going to focus on the repeatable GitHub Action of deploying the agent. To emphasize this our workflow will focus on the step called "Update agent — Foundry data plane POST `agents/NAME/versions`".

Based on organization preference, I can understand the need to break out the update agent step into a separate workflow. We traditionally don't recommend this as keeping everything in one pipeline means one set of failures to triage, one history to read, and one CI/CD surface to keep current. but This action though is structured to support a split if your release process requires it.

Hosted Agent REST Deployment Action

This is the crux of why the article exists. If you've followed my style of repeatable DevOps process for YAML Pipelines, this action follows similar principles. We will parametrize with defaults to empower minimal configuration while also optimizing for flexibility. To view the full example check out the Update Foundry Agent action .

The Inputs, Outputs, and `runs:` blocks shown below all live in a single file: `.github/actions/update-agent/action.yml`.

Inputs

Here are those parameters with descriptions and defaults:

inputs: project_endpoint: description: Foundry project endpoint URL required: true agent_name: description: Name of the hosted agent required: true image: description: Full container image reference (registry/name:tag) required: true model_deployment_name: description: Name of the AI model deployment required: true cpu: description: CPU allocation for the agent container required: false default: '0.25' memory: description: Memory allocation for the agent container required: false default: '0.5Gi'

Verify the latest sandbox sizes at hosted-agents#sandbox-sizes There is also guidance on right-sizing your Micro VMs. At the time of this writing here are the available combinations:

Outputs

We should output values that make sense for subsequent steps in the workflow. Every instance that calls this action may not use them, but it's always good to expose non-secret values just in case.

In our case we are creating a new version of the agent, so let's output that agent version:

outputs: agent_version: description: Version ID returned by the Foundry data plane value: ${{ steps.post.outputs.agent_version }}

`agent_version` is the version identifier returned by the data plane. Capture this in your pipeline (artifact, release tag, etc.) so you have an audit trail and a target to re-deploy against if a future version needs to be rolled back. Subsequent steps in the workflow can reference it via `${{ steps.<step-id>.outputs.agent_version }}`.

Action

The action will need to map our environment variables being passed into the input as the first step. After that we will need to get an access token from Azure so we can then call the REST API endpoint.

Once we have this, we will need to prepare the body of our call. Verify against the API for all valid properties. For our example I chose not to set `rai_config` (Responsible AI overview) and `tools` (function/tool bindings) to keep things simple.

runs: using: composite steps: - name: Post agent version to Foundry data plane id: post shell: bash env: PROJECT_ENDPOINT: ${{ inputs.project_endpoint }} AGENT_NAME: ${{ inputs.agent_name }} IMAGE: ${{ inputs.image }} MODEL_DEPLOYMENT_NAME: ${{ inputs.model_deployment_name }} CPU: ${{ inputs.cpu }} MEMORY: ${{ inputs.memory }} run: | FOUNDRY_TOKEN=$(az account get-access-token \ --resource "https://ai.azure.com/" \ --query accessToken -o tsv) AGENT_REQUEST_BODY=$(jq -n \ --arg cpu "$CPU" \ --arg memory "$MEMORY" \ --arg model "$MODEL_DEPLOYMENT_NAME" \ --arg image "$IMAGE" \ '{ definition: { kind: "hosted", container_protocol_versions: [{protocol: "responses", version: "1.0.0"}], cpu: $cpu, memory: $memory, environment_variables: {AZURE_AI_MODEL_DEPLOYMENT_NAME: $model}, image: $image

⚠️ **Heads up on logs.** The line that echoes `HTTP ${HTTP_STATUS}: $(cat /tmp/agent_response.json)` dumps the full response body to the job log. If your request body contains sensitive `environment_variables`, the API may return them in the response, where they will appear in plain text in the workflow log. Either scrub the response before echoing, or echo only the `version` field on success.

A 2xx response confirms the data plane accepted the new agent version. Confirming the agent behaves as intended is a separate step. This is done typically with a smoke test against the deployed agent in a later workflow job.

If something goes wrong the most common failures are:

401/403- `azure/login` didn't run, the identity is missing a Foundry data-plane role, or the wrong subscription is selected. Check the `azure/login` step and confirm the identity holds **Foundry User** (or higher) on the Foundry project (see the *Before You Start* callout above).
404 - wrong `project_endpoint`, or the agent named in `agent_name` does not yet exist on the project. The agent must exist before posting a new version.
400 - body or model issue: invalid `cpu` / `memory` shape, a required field missing, or `model_deployment_name` pointing at a deployment that isn't reachable from this project.

Calling the Action

So now that we have the action, how can we scale this across multiple workflows? Simple, we just need to pass in the required parameters. Here is an example, with a stubbed `deploy-iac` step so can the outputs passed into the action as inputs:

- name: Deploy Bicep infrastructure id: deploy-iac uses: ./.github/actions/deploy-bicep with: environment_name: ${{ inputs.environment_name || 'main' }} location: ${{ inputs.location || 'swedencentral' }} - name: Update agent uses: ./.github/actions/update-agent with: project_endpoint: ${{ steps.deploy-iac.outputs.project_endpoint }} agent_name: ${{ inputs.agent_name }} image: ${{ steps.deploy-iac.outputs.acr_endpoint }}/${{ inputs.image_name }}:${{ inputs.image_tag }} model_deployment_name: ${{ steps.deploy-iac.outputs.model_deployment_name }}

And just to show we can call the same action multiple times here are two examples that do just that: Deploy (Bicep) and Deploy (Terraform).

Conclusion

The composite action shown above gives organizations what the introduction called for: a quick, repeatable way to deploy a Hosted Agent that requires minimal adjustments to the GitHub Actions tooling and processes already in use. With it wired into a workflow, deploying a new Hosted Agent version becomes a standard step in your pipeline.

Deploying Foundry Hosted Agents via REST API

j_folberth — Wed, 03 Jun 2026 02:52:44 GMT

Introduction

At Microsoft Build `26 it was announced that Microsoft Foundry Hosted will soon be generally available. This blog post will outline the deployment process and requirements for creating hosted agents in Foundry. The audience for this blog is designed to range from beginner to more advanced users. We will focus on deploying a Hosted Agent from a local machine using the REST API backed by the Responses protocol.

The key thing to understand upfront: deploying a Hosted Agent is ultimately a data plane operation against the Foundry Service API, not an infrastructure deployment. However, it depends on carefully orchestrated infrastructure, identity, and access configuration being in place first. This post walks through each layer.

The companion repository for this post is available at github.com/JFolberth/simple-hosted-agent-responses and contains the full Infrastructure as Code (IaC) templates, deployment scripts, and agent source code referenced throughout.

Background

There's a school of thought that to fully understand and control something you must properly name it. Given this context let's level set on terminology on both what an Agent is and specifically when and how an Agent is defined inside Microsoft Foundry.

For our Definition of an Agent we will leverage the definitions provided and backed by Microsoft Agent Framework. At a process level an Agent can be defined as:

For those that want to go another level this can translate into:

Notice anything missing from these documents? There is no mention of where the Agent is hosted in terms of compute. This is open ended by design as Microsoft believes in providing the flexibility to leverage any part of the agentic technology stack for agent hosting.

We can bring our own compute such as Azure Function, Azure Container App (ACA), App Service, Hosted Virtual Machine, AWS Lambda, etc., to make a call to the Foundry. This is an agentic architecture but is not the same as a Foundry Hosted Agent. This is the key architectural element that is important to understand what makes Foundry Hosted Agents different.

As of this writing there are only three types of Foundry Agents: Prompt, Workflow (preview), and Hosted:

We are going to focus on this last type, Hosted Agents. The compute for Hosted Agents is managed entirely by Foundry through per-session Micro VMs, lightweight, isolated compute units that scale to zero when idle and resume automatically on the next request.

When building agentic applications with open-source frameworks you typically manage many cross-cutting concerns: containerization, web server setup, security, memory persistence, scaling, instrumentation, and version rollbacks. Hosted Agents solve these by letting you bring your own code and framework while the platform handles everything else.

Choose a Hosted Agent over a prompt-based agent when you need to use any framework (Agent Framework, LangGraph, Semantic Kernel), control compute resources like CPU and memory, run stateful workloads that persist files across turns, or use custom protocols for webhook receivers and non-conversational processing.

Each deployed agent automatically receives its own dedicated Microsoft Entra ID and endpoint with no manual identity configuration required. For the full comparison see What are hosted agents?

For us to fully understand the deployment process we first need to understand the required infrastructure, access, and tooling to deploy a Hosted Agent.

Requirements

A Hosted Agent can be deployed via either REST API calls or through the azd CLI. We are going to focus on the REST API process.

To deploy a Hosted Agent via REST API calls the following software needs to be available:

az cli

docker

python (not needed but preferred in our walkthrough)

The following infrastructure and sub resources are required to deploy Hosted Agents:

Microsoft Foundry (Microsoft.CognitiveServices/accounts)

Microsoft Foundry Project (Microsoft.CognitiveServices/accounts/projects)

Azure Container Registry (ACR) (Microsoft.ContainerRegistry/registries)

Microsoft Foundry Model Deployment (Microsoft.CognitiveServices/accounts/deployments)

Application Insights (optional) (Microsoft.Insights/components)

Log Analytics (optional) (Microsoft.OperationalInsights/workspaces)

From an access perspective the following roles will need to be assigned/assignable:

Contributor (provision the infrastructure)

Foundry Project Manager (Optional-Ability to view agent in Foundry Portal)

Foundry User (Data plane access is needed to create a new agent)

AcrPull (Foundry Project needs to pull the container from ACR)

AcrPush (To deploy the image to ACR)

Log Analytics Reader (Foundry Project access to read Log Analytics, optional if using)

To elaborate further the Managed Identity of the Foundry Project will need to have: AcrPull, Log Analytics Reader, Foundry User. Ideally this access provisioning would be contained within your Infrastructure as Code (IaC) processes.

The other access would be required for the user or service principal handling the deployment. These roles would be: Contributor, Foundry Project Manager, Foundry User, AcrPush. In production environments, outside the Contributor role, these roles should be provisioned after the infrastructure has been deployed. Justification being they are one time assignments and it is not a best practice to define the service principal access as part of your template. Luckily, we have a script that will handle all of this for you.

High Level Steps

First, the infrastructure will need to be deployed, either directly in the portal or ideally through an IaC template. After successful completion we will need to grab the Managed Identity of the Foundry Project, Azure Container Registry endpoint, Model(s) deployed, and Foundry Project endpoint. We will need this information for deploying our agent and granting the executing user the ability to view the Agent in Foundry.

After the IaC deployment we will grant the user Foundry Project Manager. We chose the Foundry Project Manager role here as not only will it give permission to deploy an agent it will also give the deploying user access to the Foundry Portal to view the agent. If viewing the agent is not a requirement then Foundry User should suffice.

Since the IaC deploys our ACR we will need to build and push our container image. At the time of this writing the image must be built with --platform linux/amd64 as the Foundry Micro VM runtime only supports amd64 architecture. See the hosted agent container requirements for details.

docker build --platform linux/amd64 -t <acr-login-server>/<image-name>:<tag> . az acr login --name <acr-name> docker push <acr-login-server>/<image-name>:<tag>

For the Foundry Micro VM runtime to pull your container image at agent startup, a connection must exist between your Foundry Project and the Azure Container Registry. This is configured as a project-scoped connection of category ContainerRegistry that references the ACR login server and uses the project's Managed Identity for authentication (no stored credentials).

The IaC templates in the companion repo handle this automatically, but if provisioning manually you will need to create this connection via the Foundry project connections API or through the Foundry Portal. For more details see Connections in Microsoft Foundry and Configure container registry connections.

The last step is the most important, deploying the agent. At the end of the day a Hosted Agent is actually not an infrastructure change, rather a data plane operation against the Foundry Service API. We pass in the kind as hosted, specify the container image, and provide the CPU and memory allocation for our Micro VMs.

curl -X POST \ "<project-endpoint>/agents/<agent-name>/versions?api-version=2025-11-15-preview" \ -H "Authorization: Bearer $(az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv)" \ -H "Content-Type: application/json" \ -d '{ "kind": "hosted", "configuration": { "container": { "image": "<acr-login-server>/<image-name>:<tag>" }, "resources": { "cpu": "0.25", "memory": "0.5Gi" }, "protocolVersion": "1.0.0", "environmentVariables": { "AZURE_AI_MODEL_DEPLOYMENT_NAME": "<model-deployment-name>" } }, "metadata": { "enableVnextExperience": "true" } }'

Conclusion

As we covered, deploying a Hosted Agent comes down to a single REST API call, but getting to that point requires the right infrastructure, roles, and container image in place.

Infrastructure first: Your Foundry project, ACR, and model deployments must exist before agent creation. Role assignments, particularly Foundry User for the project managed identity and AcrPull for image access, must be in place before proceeding.

The REST call is the agent: Unlike traditional container deployments where you manage compute directly, the POST to the Foundry data plane is what brings your agent to life. You specify the container image, CPU/memory allocation, and protocol version. Foundry handles the rest, provisioning per-session Micro VMs that scale to zero when idle.

You own only the agent logic: The platform manages the runtime, session isolation, identity, and scaling. This is the fundamental shift from self-hosted patterns. Less operational overhead in exchange for letting Foundry control the compute layer.

To see a complete, working implementation including Bicep and Terraform IaC templates, deployment scripts with proper RBAC handling, and a Python agent using the Responses protocol, visit the companion repository at github.com/JFolberth/simple-hosted-agent-responses. The repo includes a dev container with all prerequisites pre-installed, making it possible to go from clone to deployed agent in a single azd up command.

Infrastructure as Code for AI: Building and Deploying Microsoft Hosted Agents with Terraform

Lee_Stott — Tue, 02 Jun 2026 14:22:33 GMT

AI agents are no longer experimental. Teams are shipping production-grade agents that retrieve information, call APIs, reason over documents, and orchestrate multi-step workflows at scale. Microsoft Foundry's Hosted Agents service gives you a fully managed runtime for those agents, built on top of the Microsoft Foundry Agent Service, with Microsoft handling the infrastructure, scaling, and runtime lifecycle.

The challenge is that provisioning this infrastructure by hand or clicking through the portal, running one-off CLI commands, or relying on undocumented shell scripts, simply does not scale. It introduces configuration drift, makes reproducing environments painful, and creates real governance risk as teams grow.

This post walks through how to provision and manage the Azure infrastructure required to run Microsoft Hosted Agents using Terraform. You will leave with working configuration, a clear understanding of the resource model, and practical guidance on where Terraform can take you all the way and where you will need to supplement with the Azure CLI or the Microsoft Foundry Agent Service SDK.

What Are Microsoft Hosted Agents?

Microsoft Hosted Agents are AI agents deployed and managed within Microsoft Foundry. Microsoft Foundry is Microsoft's unified platform for building, evaluating, and deploying AI applications and agents. It provides:

A managed compute runtime — Microsoft provisions and scales the infrastructure so you do not manage VMs or containers.
An agent execution environment — agents are defined with instructions, tools (code interpreter, Bing grounding, Azure AI Search, function calling), and a backing model endpoint.
Deep Azure integration — identity via Microsoft Entra ID, secrets via Azure Key Vault, storage via Azure Blob, tracing via Azure Monitor and Application Insights.
A project-scoped model — each Microsoft Foundry project encapsulates an agent's resources, connections, and deployments within a logical boundary.

The "Hosted" distinction matters. You are not running agent code on your own Kubernetes cluster or App Service. Microsoft manages the runtime. Your responsibility is to provision the surrounding infrastructure correctly: the Microsoft Foundry resource, the project, the model deployment, the identity configuration, and the monitoring resources that back it all.

That boundary — the infrastructure you own — is exactly what Terraform manages well.

Why Terraform for Hosted Agent Deployments?

Infrastructure as Code (IaC) is not a new idea, but its importance grows as AI deployments become more complex. Here is why Terraform is a strong choice for Microsoft Foundry deployments specifically:

Repeatability: A Terraform configuration produces the same infrastructure every time. Staging mirrors production. Disaster recovery is a terraform apply away.
Governance: Infrastructure definitions live in version control alongside application code. Changes are reviewable, auditable, and reversible. This satisfies most enterprise change-management requirements.
Scale: Spinning up per-customer or per-team agent environments using Terraform workspaces or module instantiation is far more manageable than manual provisioning.
State management: Terraform tracks the actual state of your Azure resources. It detects drift and reconciles it declaratively.
Ecosystem: The AzureRM provider is mature, actively maintained by HashiCorp and Microsoft, and covers the majority of Azure services including the Microsoft Foundry resources.

Architecture Overview

Before writing any Terraform, it helps to understand the resource hierarchy in Microsoft Foundry and how each layer maps to an Azure resource type.

The Foundry Resource Hierarchy

Microsoft Foundry uses a two-level hierarchy:

1. Foundry Account (azurerm_cognitive_account, kind: AIServices) — The top-level AI Services resource. It provides the model endpoint, manages agent execution, and acts as the logical boundary for all projects beneath it. You must set project_management_enabled = true and provide a custom_subdomain_name to enable project creation. In ARM terms this is a Microsoft.CognitiveServices/accounts resource.

2. Foundry Project (azurerm_cognitive_account_project) — A child resource scoped within the Foundry Account. Each project has its own agents, model deployments, connections, and data assets. In production, you typically have one project per application, product team, or environment.

Figure 1: The Microsoft Foundry resource hierarchy. A single Foundry Account (Cognitive Services, kind AIServices) acts as the top-level container, with Projects scoped beneath it — one per application, team, or environment.

Supporting Resources

The following Azure resources make up a complete Hosted Agents deployment:

Microsoft Foundry Account (AI Services): A single azurerm_cognitive_account of kind AIServices serves as both the Foundry Account and the model endpoint host. Model deployments (e.g. gpt-4.1) are provisioned via azurerm_cognitive_deployment within this account.
Log Analytics Workspace + Application Insights: Provides observability for agent traces, request logs, and metrics.
User-Assigned Managed Identity: Grants the Foundry Account and Projects access to Azure resources without stored credentials.
Role Assignments (RBAC): Wires the managed identity to the Foundry Account with least-privilege Cognitive Services permissions.

Figure 2: Supporting infrastructure map. The managed identity holds least-privilege RBAC grants to the Microsoft Foundry Account (AI Services) — enabling model access and project management — all within the same resource group.

Reference Architecture (Described)

A production-ready layout separates concerns across two resource groups: one for shared infrastructure (networking, monitoring) and one for the Microsoft Foundry Account and its projects. The Foundry resource group houses the azurerm_cognitive_account (kind: AIServices) resource and the azurerm_cognitive_account_project instances. The shared resource group holds Log Analytics and Application Insights. A user-assigned managed identity spans both, holding RBAC grants to each backing service.

For a dev/test environment you can collapse both into a single resource group. For production, the separation makes cost attribution, access control, and lifecycle management cleaner.

Prerequisites

Accounts and Permissions

An active Azure subscription with the Owner or Contributor + User Access Administrator roles at the subscription or resource group level (role assignments require elevated permission).
Foundry access enabled in your subscription. In some tenants you may need to accept terms or request quota for Azure OpenAI.
Azure OpenAI quota for the model you intend to deploy (e.g. gpt-4.1). Request this via the Azure portal under Quotas in Azure OpenAI Studio.

Local Tools

Terraform CLI ≥ 1.9 — Install guide
Azure CLI ≥ 2.60 — Install guide
A code editor (VS Code with the HashiCorp Terraform extension and the Azure Terraform extension is a strong combination).

Authentication

For local development, authenticate via the Azure CLI. The AzureRM Terraform provider picks this up automatically:

az login
az account set --subscription "<your-subscription-id>"

For CI/CD pipelines, use a service principal with AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID environment variables, or — preferably — a workload identity federation (federated credentials) to avoid storing long-lived secrets. GitHub Actions supports OIDC-based workload identity natively.

Terraform Fundamentals for Hosted Agents

Provider Configuration

The hashicorp/azurerm provider is your primary dependency. The new Microsoft Foundry resources (azurerm_cognitive_account with kind = "AIServices" and azurerm_cognitive_account_project) require version 4.x of the provider. Pin your version to avoid unexpected breaking changes:

terraform {
  required_version = ">= 1.9"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
}

provider "azurerm" {
  features {
    key_vault {
      purge_soft_delete_on_destroy = false
    }
    resource_group {
      prevent_deletion_if_contains_resources = true
    }
  }
  subscription_id = var.subscription_id
}

The features block is required even when empty. The Key Vault setting prevents accidental secret loss during terraform destroy. The resource group setting adds an extra safety net in production.

State Management

Never use local state for shared or production environments. Store state in Azure Blob Storage with state locking via Azure Blob lease:

terraform {
  backend "azurerm" {
    resource_group_name  = "rg-terraform-state"
    storage_account_name = "sttfstate<unique>"
    container_name       = "tfstate"
    key                  = "ai-agents/prod.tfstate"
  }
}

Create the state storage account and container before running terraform init. A bootstrap script or a separate Terraform workspace dedicated to state management are both valid approaches.

Known Limitations and Workarounds

Terraform coverage of Foundry is improving rapidly but is not yet complete. You should be aware of the following gaps as of mid-2025:

Agent definitions are not in Terraform: The actual agent (its system prompt, instructions, tool configuration, and model binding) is created via the Azure AI Agent Service SDK or the Foundry portal, not via Terraform. Terraform provisions the infrastructure; your application code or a post-provisioning script creates the agent.
Connections: Some connection types within a Foundry Project (e.g. Azure AI Search, custom connections) may require the Azure CLI or the Foundry SDK. Verify coverage in the AzureRM provider docs before assuming Terraform handles them.
Model deployments: azurerm_cognitive_deployment covers OpenAI model deployments and is well-supported. Use this to deploy your model before referencing it from the agent.
Private networking: If you need private endpoints for your Foundry Account, additional VNet, subnet, and DNS zone resources are required. This post focuses on the public networking path; private networking is a follow-on topic.

Step-by-Step Implementation

The following sections build up a complete Terraform configuration. The recommended project structure is a flat module layout for a single environment, with a separate modules/ai-foundry/ directory when you need to reuse the pattern across environments.

ai-agents-infra/
├── main.tf
├── variables.tf
├── outputs.tf
├── versions.tf
└── terraform.tfvars

1. Variables

Define variables first. Parameterising from the start avoids hard-coded values that create technical debt when you replicate the configuration for staging or production:

# variables.tf

variable "subscription_id" {
  type        = string
  description = "Azure subscription ID."
}

variable "location" {
  type        = string
  default     = "eastus"
  description = "Azure region for all resources."
}

variable "environment" {
  type        = string
  default     = "dev"
  description = "Environment label (dev, staging, prod)."
}

variable "project_name" {
  type        = string
  description = "Short name for the project. Used in resource naming."
}

variable "openai_model_name" {
  type        = string
  default     = "gpt-4.1"
  description = "Azure OpenAI model to deploy for the agent."
}

variable "openai_model_version" {
  type        = string
  default     = "2025-04-14"
  description = "Model version to deploy."
}

variable "openai_sku_capacity" {
  type        = number
  default     = 10
  description = "Tokens-per-minute capacity (in thousands) for the deployment."
}

2. Resource Group and Core Infrastructure

A single resource group keeps things simple for dev. In production, consider splitting as described in the architecture section above.

# main.tf — Resource group and naming locals

locals {
  name_prefix = "${var.project_name}-${var.environment}"
  tags = {
    environment = var.environment
    project     = var.project_name
    managed_by  = "terraform"
  }
}

resource "azurerm_resource_group" "main" {
  name     = "rg-${local.name_prefix}"
  location = var.location
  tags     = local.tags
}

3. Supporting Services

Provision Log Analytics and Application Insights for agent observability and diagnostics. Unlike the legacy Hub-based architecture, the azurerm_cognitive_account (kind AIServices) does not require a dedicated Storage Account or Key Vault as provisioning dependencies.

# main.tf — Monitoring infrastructure

data "azurerm_client_config" "current" {}

# Log Analytics Workspace (required by Application Insights)
resource "azurerm_log_analytics_workspace" "main" {
  name                = "law-${local.name_prefix}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  sku                 = "PerGB2018"
  retention_in_days   = 30
  tags                = local.tags
}

# Application Insights for agent observability
resource "azurerm_application_insights" "main" {
  name                = "appi-${local.name_prefix}"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  workspace_id        = azurerm_log_analytics_workspace.main.id
  application_type    = "web"
  tags                = local.tags
}

4. User-Assigned Managed Identity

A managed identity allows the Foundry Account and its projects to authenticate to Azure services without stored credentials. This is a security best practice and is required for several Microsoft Foundry features.

# main.tf — Managed identity for the Microsoft Foundry Account

resource "azurerm_user_assigned_identity" "foundry" {
  name                = "id-${local.name_prefix}-foundry"
  resource_group_name = azurerm_resource_group.main.name
  location            = azurerm_resource_group.main.location
  tags                = local.tags
}

5. Microsoft Foundry Account and Model Deployment

In the current Microsoft Foundry architecture, a single azurerm_cognitive_account of kind AIServices serves as both the Foundry Account and the model endpoint host. Set project_management_enabled = true and provide a globally unique custom_subdomain_name to enable Foundry Project creation beneath it.

# main.tf — Microsoft Foundry Account (AI Services)

resource "azurerm_cognitive_account" "foundry" {
  name                       = "aisa-${local.name_prefix}"
  resource_group_name        = azurerm_resource_group.main.name
  location                   = azurerm_resource_group.main.location
  kind                       = "AIServices"
  sku_name                   = "S0"
  project_management_enabled = true
  custom_subdomain_name      = "${replace(local.name_prefix, "-", "")}foundry"
  tags                       = local.tags

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.foundry.id]
  }
}

# Deploy the model within the Foundry Account
resource "azurerm_cognitive_deployment" "agent_model" {
  name                 = var.openai_model_name
  cognitive_account_id = azurerm_cognitive_account.foundry.id

  model {
    format  = "OpenAI"
    name    = var.openai_model_name
    version = var.openai_model_version
  }

  sku {
    name     = "Standard"
    capacity = var.openai_sku_capacity
  }
}

Note on quota: The capacity value is in thousands of tokens per minute. A value of 10 means 10,000 TPM. If terraform apply fails with a quota error, reduce this value or request a quota increase via the Azure portal.

Note on custom_subdomain_name: This must be globally unique across all Azure AI Services accounts. If provisioning fails with a conflict error, adjust the suffix (e.g. append a random string using the random_string resource).

6. Foundry Project

Create a Foundry Project beneath the Foundry Account provisioned in Step 5. Each project scopes its own agents, model connections, and data assets. Use one project per application or team.

# main.tf — Microsoft Foundry Project

resource "azurerm_cognitive_account_project" "agent_project" {
  name                 = "proj-${local.name_prefix}-agents"
  cognitive_account_id = azurerm_cognitive_account.foundry.id
  location             = azurerm_resource_group.main.location
  display_name         = "Agent Project - ${var.project_name}"
  description          = "Hosted agents project for ${var.project_name}"

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.foundry.id]
  }

  tags = local.tags
}

7. RBAC Role Assignments

Grant the managed identity the permissions it needs. This is the area most commonly misconfigured in manual deployments. Terraform makes it explicit and auditable.

# main.tf — RBAC assignments

# AI Services: Foundry identity needs Cognitive Services OpenAI User to call model endpoints
resource "azurerm_role_assignment" "foundry_openai" {
  scope                = azurerm_cognitive_account.foundry.id
  role_definition_name = "Cognitive Services OpenAI User"
  principal_id         = azurerm_user_assigned_identity.foundry.principal_id
}

# AI Services: Foundry identity needs Cognitive Services Contributor to manage projects
resource "azurerm_role_assignment" "foundry_contributor" {
  scope                = azurerm_cognitive_account.foundry.id
  role_definition_name = "Cognitive Services Contributor"
  principal_id         = azurerm_user_assigned_identity.foundry.principal_id
}

# Optional: grant your own principal the Azure AI Developer role on the Foundry Account
# so you can create and manage agents from your local machine or CI pipeline
resource "azurerm_role_assignment" "developer_account" {
  scope                = azurerm_cognitive_account.foundry.id
  role_definition_name = "Azure AI Developer"
  principal_id         = data.azurerm_client_config.current.object_id
}

8. Outputs

Export the values your application and post-provisioning scripts will need:

# outputs.tf

output "resource_group_name" {
  value = azurerm_resource_group.main.name
}

output "foundry_account_id" {
  value = azurerm_cognitive_account.foundry.id
}

output "ai_foundry_project_id" {
  value = azurerm_cognitive_account_project.agent_project.id
}

output "foundry_endpoint" {
  value = azurerm_cognitive_account.foundry.endpoint
}

output "openai_deployment_name" {
  value = azurerm_cognitive_deployment.agent_model.name
}

output "managed_identity_client_id" {
  value = azurerm_user_assigned_identity.foundry.client_id
}

10. Example terraform.tfvars

# terraform.tfvars — do NOT commit this file if it contains sensitive values
subscription_id      = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
location             = "eastus"
environment          = "dev"
project_name         = "contoso-agents"
openai_model_name    = "gpt-4.1"
openai_model_version = "2025-04-14"

openai_sku_capacity = 10

Figure 3: Terraform deployment workflow. State is stored in an Azure Blob Storage backend, enabling team collaboration and preventing concurrent apply conflicts.

Deploying and Validating the Agent Infrastructure

Running the Deployment

# 1. Initialise — downloads provider plugins and configures the backend
terraform init

# 2. Validate syntax and configuration
terraform validate

# 3. Preview what will be created (review carefully before applying)
terraform plan -out=tfplan

# 4. Apply the plan
terraform apply tfplan

A full initial apply typically takes 8–15 minutes. The Foundry Account (AI Services) provisioning is the longest step. The model deployment may also take a few minutes to reach a ready state — Terraform handles this with implicit dependency ordering, but you may see brief retries in the output.

Verifying the Deployment

After apply completes, verify each resource is in a healthy state:

# Confirm the resource group and its resources exist
az resource list --resource-group "rg-contoso-agents-dev" --output table

# Check the Foundry Account (AI Services) is in a Succeeded state
az cognitiveservices account show \
  --name "aisacontosoagentsdevfoundry" \
  --resource-group "rg-contoso-agents-dev" \
  --query "properties.provisioningState"

# Confirm the model deployment is ready
az cognitiveservices account deployment show \
  --resource-group "rg-contoso-agents-dev" \
  --name "aisacontosoagentsdevfoundry" \
  --deployment-name "gpt-4.1" \
  --query "properties.provisioningState"

Navigate to the Microsoft Foundry portal and confirm your Foundry Account and Project appear. At this point you can create an agent manually in the portal to validate that the model endpoint is reachable and the identity chain works correctly before automating agent creation.

Common Deployment Issues

Quota exceeded on model deployment: Reduce openai_sku_capacity or request a quota increase in the Azure portal under Azure OpenAI → Quotas.
Resource name conflicts: The custom_subdomain_name on the Foundry Account must be globally unique. Use the random_string Terraform resource to append a unique suffix if needed.
Role assignment propagation delay: RBAC changes can take 1–2 minutes to propagate. If the Foundry Account cannot access resources immediately after apply, wait a moment and retry.
project_management_enabled not set: If azurerm_cognitive_account_project fails with an error about project management, ensure project_management_enabled = true and custom_subdomain_name are set on the parent azurerm_cognitive_account.
azurerm_cognitive_account_project not found: Ensure your AzureRM provider version is ~> 4.0 or later. Run terraform init -upgrade if you previously initialised with an older version.

Creating an Agent After Infrastructure Provisioning

Terraform has provisioned the platform. Now you need to create the agent itself. This is done via the Azure AI Agents SDK (available for Python, C#, JavaScript, and Java) or the Foundry portal.

The following Python snippet demonstrates creating a basic agent programmatically after Terraform apply. It uses the outputs from Terraform directly:

import os
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

# These values come from Terraform outputs
project_connection_string = os.environ["AI_PROJECT_CONNECTION_STRING"]
model_deployment = os.environ["OPENAI_DEPLOYMENT_NAME"]

client = AIProjectClient.from_connection_string(
    credential=DefaultAzureCredential(),
    conn_str=project_connection_string,
)

# Create the hosted agent
agent = client.agents.create_agent(
    model=model_deployment,
    name="customer-support-agent",
    instructions=(
        "You are a helpful customer support assistant. "
        "Answer questions accurately and concisely. "
        "If you are unsure, say so rather than guessing."
    ),
)

print(f"Agent created: {agent.id}")

Figure 5: Agent runtime architecture. The Foundry Project hosts the Agent Service, which routes requests to the GPT-4.1 model endpoint and optionally invokes tool integrations (Code Interpreter, File Search, Azure Functions, or custom tools).

The project connection string is available from the Foundry portal (Project → Overview → Project connection string) or can be constructed from Terraform outputs. Refer to the Azure AI Agents quickstart for the full SDK setup.

Operational Considerations

Lifecycle Management

Terraform's declarative model means updates are incremental by default. To update the OpenAI model version, change openai_model_version in your .tfvars file and run terraform plan to confirm the change before applying. Terraform will delete and recreate the cognitive deployment in-place — be aware this causes brief downtime for the model endpoint.

To destroy a complete environment:

terraform destroy

The prevent_deletion_if_contains_resources feature on the resource group will block destruction if any untracked resources exist, which is a useful safety net in production.

Handling Configuration Drift

Drift occurs when Azure resources are modified outside of Terraform (portal changes, CLI scripts, other automation). Detect drift with:

terraform plan -refresh-only

This reports the difference between the Terraform state and the actual resource state without making changes. Schedule this as a drift-detection job in CI to catch out-of-band changes early.

Environment Isolation

Use Terraform workspaces or separate state files per environment:

# Create and switch to a staging workspace
terraform workspace new staging
terraform workspace select staging
terraform apply -var-file="environments/staging.tfvars"

Alternatively, use a directory-per-environment layout (environments/dev/, environments/prod/) with a shared module in modules/ai-foundry/. The directory layout is more explicit and easier to navigate in a team setting.

Cost Control

Set a low openai_sku_capacity in dev (e.g. 1 = 1,000 TPM) to limit accidental spend.
Tag all resources with environment and project tags (the locals.tags block handles this) to enable cost attribution in Azure Cost Management.
Use the Azure Pricing Calculator to estimate monthly costs before deploying to production. The Azure AI Services account (model token usage), Log Analytics, and Application Insights are the primary cost drivers.
Consider destroying dev environments overnight using a scheduled CI job that runs terraform destroy and terraform apply on a schedule.

CI/CD Integration

Automating Terraform via GitHub Actions is straightforward. The following workflow runs plan on pull requests and apply on merge to the main branch:

# .github/workflows/terraform.yml

name: Terraform Deploy

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  id-token: write   # Required for OIDC workload identity federation
  contents: read
  pull-requests: write

env:
  ARM_CLIENT_ID:       ${{ secrets.AZURE_CLIENT_ID }}
  ARM_TENANT_ID:       ${{ secrets.AZURE_TENANT_ID }}
  ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
  ARM_USE_OIDC:        "true"

jobs:
  terraform:
    runs-on: ubuntu-latest
    environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}

    steps:
      - uses: actions/checkout@v4

      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "~1.9"

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        run: terraform plan -out=tfplan -var-file="environments/dev.tfvars"

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve tfplan

Figure 4: CI/CD pipeline using GitHub Actions with OIDC workload identity federation. No long-lived secrets are stored — the runner exchanges a JWT for a short-lived Azure token before each Terraform run.

Use OIDC workload identity federation to avoid storing long-lived service principal secrets in GitHub. This is the recommended authentication method for GitHub Actions deployments to Azure.

Best Practices

Modular Terraform Design

Once you have a working flat configuration, extract the Foundry resources into a reusable module. A module boundary around the Hub, Project, OpenAI account, and RBAC assignments lets you stamp out new agent environments with a single module call and a new .tfvars file.

# environments/staging/main.tf
module "agent_platform" {
  source = "../../modules/ai-foundry"

  project_name         = "contoso-agents"
  environment          = "staging"
  location             = "eastus"
  subscription_id      = var.subscription_id
  openai_model_name    = "gpt-4.1"
  openai_model_version = "2025-04-14"
  openai_sku_capacity  = 30
}

Parameterisation and Environment Configs

Never hard-code subscription IDs, tenant IDs, or region names in main.tf.
Keep environment-specific values in environments/<env>.tfvars files and commit them to source control (they are config, not secrets).
Store actual secrets (service principal credentials, API keys for third-party connections) in Azure Key Vault or GitHub Secrets — not in .tfvars files.

Versioning Models and Agent Configurations

Treat your openai_model_version and agent instructions as versioned artefacts. When Microsoft releases a new model version, create a pull request that updates the variable value, runs a plan, and documents the expected change. This creates a clear history of when model versions changed and who approved the change.

Logging and Monitoring

Enable diagnostic settings on the Azure OpenAI account to route request logs and metrics to your Log Analytics workspace.
Use Application Insights to capture agent traces from the Azure AI Agents SDK (it integrates with OpenTelemetry).
Set up Azure Monitor alerts on OpenAI account errors (4xx/5xx rates) and Log Analytics ingestion failures.

Responsible AI Considerations

Enable Azure OpenAI content filtering on your deployment. Terraform supports this via the content_filter block in azurerm_cognitive_deployment where the policy allows.
Define a clear system prompt that sets agent behaviour boundaries and instructs the agent to decline harmful requests.
Log and review agent conversations during early deployment. Microsoft Foundry includes evaluation tools for assessing agent response quality and safety.
Apply least-privilege RBAC throughout — the role assignments in this post follow that principle.

Conclusion and Next Steps

You now have a complete, repeatable Terraform configuration for provisioning the Azure infrastructure required to run Microsoft Hosted Agents via Microsoft Foundry. The key takeaways:

Terraform manages the infrastructure layer effectively — the Foundry Account, Project, model deployment, identity, and RBAC.
Agent definitions themselves are provisioned via the Azure AI Agents SDK or the Foundry portal as a post-Terraform step.
State management, parameterisation, and modular design are non-negotiable for team environments.
OIDC-based workload identity is the right authentication model for CI/CD pipelines.
Drift detection, environment isolation, and cost tagging are operational necessities, not optional extras.

Where to Go Next

Add Azure AI Search: Extend the Foundry Project with an Azure AI Search connection and enable the Search tool on your agent for Retrieval-Augmented Generation (RAG).
Private networking: Add private endpoints for the Foundry Hub and OpenAI account to lock down ingress to your VNet.
Multi-region deployment: Instantiate the Terraform module twice with different regions and use Azure Traffic Manager or Front Door to route requests.
GitOps for agents: Store agent definitions (system prompts, tool configurations) as YAML or JSON in your repository and use a CI pipeline to apply them via the Azure AI Agents SDK on every merge, creating a fully declarative agent deployment pipeline.
Evaluation pipelines: Use Microsoft Foundry's built-in evaluation capabilities to run automated quality and safety assessments on every new model version or prompt change.

References

Building and Operating a Microsoft Foundry Hosted Agent with GitOps and GitHub Tasks

Lee_Stott — Tue, 02 Jun 2026 07:00:00 GMT

The Gap Between Prototype and Production

Most AI engineering teams can build a working agent in a day. The hard part is not building it; the hard part is operating it. Prompts drift. Tool configurations change without review. Deployments happen from someone's laptop. There is no audit trail, no rollback plan, and no consistent way to promote a change from a development environment to production.

GitOps closes that gap. By treating your agent definition, configuration, and infrastructure as version-controlled source code, you get the same delivery discipline that software engineering teams have applied to application code for years. Every change is reviewed, every deployment is automated, and every environment state is traceable to a specific commit.

This post shows you how to apply GitOps principles to a Microsoft Foundry Hosted Agent using GitHub as the source of truth and GitHub Tasks and Actions as the automation layer. The result is a repeatable, governed, production-ready delivery model for AI agents.

What Is a Microsoft Foundry Hosted Agent?

Microsoft Foundry is Microsoft's platform for building, deploying, and operating AI applications and agents. A Hosted Agent is an agent runtime managed by the Foundry platform rather than self-hosted by your team. You supply the agent logic, configuration, and tools; Foundry handles the runtime lifecycle, scaling, and managed infrastructure.

In practical terms, a Foundry Hosted Agent is a containerised agent application. You package your agent code, prompt definitions, tool bindings, and environment configuration into a container image. Foundry deploys and manages that container within a Foundry project, connected to models, tools, and observability infrastructure that the platform provides.

Teams choose Hosted Agents over self-hosting because:

The platform manages runtime infrastructure, patching, and scaling
Integration with Azure AI models, managed identity, and observability is built in
You can focus engineering effort on agent logic rather than cluster management
Foundry projects provide environment and resource isolation without requiring you to provision and manage separate Azure resources for each environment

Hosted Agents are a good fit when your team wants strong operational support with minimal platform overhead, when you need clear separation between environments, and when your agents depend on Azure AI capabilities such as Azure OpenAI Service, Azure AI Search, or Model Context Protocol integrations.

Why GitOps Matters Specifically for AI Agents

GitOps is straightforward for stateless web services: the code changes, the pipeline runs, the container is deployed. AI agents are more complex because there are multiple distinct artefacts that all affect agent behaviour:

System prompts and instruction files
Tool definitions and external integrations
Model selection and configuration (temperature, max tokens, safety settings)
Model Context Protocol (MCP) server definitions
Orchestration logic and agent workflow code
Safety and policy settings
Infrastructure and deployment configuration

Any one of these can change the behaviour of your agent in ways that are difficult to detect without structured review. A prompt change that looks harmless can alter tone, scope, or factual grounding. A tool configuration change can expose data to unintended callers. A model upgrade can shift response quality unpredictably.

Git gives you a single place to version, review, and approve all of these artefacts together. Pull requests give you a structured review gate. Workflow automation gives you validation before anything reaches a deployed environment. Tags and releases give you deployment markers you can roll back to.

The discipline of GitOps turns what is often an ad-hoc AI delivery process into a repeatable engineering practice.

Reference Architecture

The following diagram shows a practical reference architecture for delivering a Microsoft Foundry Hosted Agent through a GitOps model using GitHub.

+---------------------------+
|    GitHub Repository      |
|  /src  /agents  /tools    |
|  /prompts  /infra         |
|  /.github/workflows       |
+---------------------------+
             |
             | Pull Request / Push to main
             v
+---------------------------+
|   GitHub Actions          |
|  1. Validate agent config |
|  2. Lint and scan code    |
|  3. Run unit tests        |
|  4. Build container image |
|  5. Push to registry      |
+---------------------------+
             |
             | Image tag (SHA or semver)
             v
+---------------------------+
|  Azure Container Registry |
|  myregistry.azurecr.io    |
|  my-agent:<sha>          |
+---------------------------+
             |
      +------+------+
      |             |
      v             v
+----------+  +----------+
| Foundry  |  | Foundry  |
| Dev      |  | Test     |
| Project  |  | Project  |
+----------+  +----------+
                   |
         Approval gate (GitHub env)
                   |
                   v
            +----------+
            | Foundry  |
            | Prod     |
            | Project  |
            +----------+
                   |
                   v
+---------------------------+
|  Observability            |
|  Azure Monitor / App      |
|  Insights / Foundry Logs  |
+---------------------------+

Key design decisions in this architecture:

The GitHub repository is the single source of truth for all agent artefacts
No human deploys directly to any Foundry project; all changes flow through automation
Environment promotion requires a GitHub environment approval, creating a governance gate
The container image is built once and promoted across environments; the image is not rebuilt per environment
Secrets are stored in Azure Key Vault and accessed by the Foundry agent at runtime via managed identity

Figure: GitOps delivery pipeline stages from commit to production

Repository Structure

A well-structured repository separates agent logic from infrastructure and tooling from prompts. The following structure works well in practice:

my-foundry-agent/
├── .github/
│   ├── workflows/
│   │   ├── validate.yml        # Runs on every PR
│   │   ├── build-deploy.yml    # Runs on merge to main
│   │   └── rollback.yml        # Manual trigger workflow
│   └── CODEOWNERS              # Review assignments by path
├── src/
│   ├── agents/
│   │   ├── agent.py            # Agent entry point and orchestration
│   │   └── agent_config.json   # Agent metadata and settings
│   ├── tools/
│   │   ├── search_tool.py      # Tool implementations
│   │   └── data_tool.py
│   └── prompts/
│       ├── system.txt          # System prompt (versioned as plain text)
│       └── instructions.txt    # Supplementary instructions
├── tests/
│   ├── unit/                   # Unit tests for tools and logic
│   ├── integration/            # Integration tests against a running agent
│   └── smoke/                  # Post-deployment smoke tests
├── infra/
│   ├── main.bicep              # Foundry project and resource definitions
│   └── environments/
│       ├── dev.parameters.json
│       ├── test.parameters.json
│       └── prod.parameters.json
├── scripts/
│   ├── validate_agent.py       # Config validation script
│   └── smoke_test.py           # Smoke test runner
├── Dockerfile                  # Container image definition
└── docs/
    └── architecture.md         # Architecture and runbook documentation

What belongs where and why:

/src/prompts - System prompts as plain text files. Versioning prompts as files means every change goes through a pull request with a diff review, just as code does.
/src/agents - Agent orchestration logic and configuration. Keeps the entry point and agent metadata co-located.
/src/tools - Tool implementations separated from agent logic. Tool logic changes independently and should be reviewable in isolation.
/infra - Infrastructure as code with per-environment parameter files. Environment-specific values live here, never in source files.
/tests - Three layers of testing: unit tests for tools, integration tests for the full agent, and smoke tests that run against a deployed environment.
/.github/workflows - All automation defined as code. There should be no manual deployment steps that live outside this directory.

GitHub Tasks Across the Delivery Lifecycle

GitHub Tasks and Issues provide the work tracking layer on top of the GitOps delivery model. Used well, they connect the intention behind a change to its implementation and deployment history.

Practical patterns for using GitHub Tasks with agent delivery:

Prompt change task - Open an issue to describe why the system prompt is changing. The pull request that changes system.txt closes that issue, creating a permanent link between the rationale and the diff.
Tool integration task - When adding a new MCP server or external tool integration, create a task that captures the design decision, security review outcome, and test evidence before the pull request is merged.
Model upgrade task - When upgrading the underlying model version, create a task that includes evaluation results and comparison data. The task becomes part of your change audit trail.
Rollback task - If a deployment causes quality regressions, create a task to track the rollback, root cause investigation, and corrective action. Automation can open this task automatically when a deployment fails health checks.
Dependency on approval - GitHub Tasks can be linked to environment approvals in GitHub Actions. A task in a specific milestone or project column can gate a promotion workflow.

The key insight is that GitHub Tasks are not just work management; they are part of your audit trail. A regulatory or security reviewer can follow the chain from a production deployment back through workflow runs, pull request reviews, and the original task that described the intent of the change.

End-to-End GitOps Flow

The following walk-through describes a realistic developer experience for changing an agent prompt and promoting it to production.

A developer opens a GitHub Issue describing the prompt change required and the expected behaviour improvement.
The developer creates a feature branch, edits src/prompts/system.txt, and updates any related unit tests.
A pull request is opened. The validate workflow runs immediately, checking prompt length, configuration schema, and lint rules. Unit tests run against the changed files.
A code reviewer approves the pull request. The CODEOWNERS file ensures that prompt changes require review from the AI engineering team, not just any contributor.
On merge to main, the build workflow runs: the container image is built with the new prompt baked in, tagged with the commit SHA, and pushed to Azure Container Registry.
The deployment workflow deploys the new image to the Foundry Dev project automatically. Integration and smoke tests run against the deployed dev agent.
If tests pass, the workflow pauses at the Test environment gate and requests approval from a named reviewer.
After approval, the same image is deployed to Foundry Test. Smoke tests run again.
A second approval gate controls promotion to Foundry Prod.
If at any point a health check or smoke test fails, the rollback workflow redeploys the previous image tag from the registry. The image tag of the last known-good deployment is stored as a GitHub environment variable.

This flow means that no human ever deploys directly to any environment. Every environment state is traceable to a specific commit, image tag, and workflow run.

Security and Governance

AI agents often have access to sensitive data and external systems. Security and governance cannot be an afterthought.

Identity and Access

Use managed identity for the Foundry Hosted Agent to access Azure resources. Avoid service principal secrets where Microsoft Entra Workload Identity or managed identity is available.
Apply the principle of least privilege: the agent identity should have read access to data sources and limited write access only where the use case requires it.
Tool integrations that require API keys or external credentials should retrieve them from Azure Key Vault at runtime, never from environment variables baked into the image.

Secrets and Configuration

Store secrets in Azure Key Vault. Reference them in your Foundry project configuration using Key Vault references.
Store GitHub Actions secrets using repository or environment-scoped secrets. Never echo secrets in workflow logs.
Separate environment configuration (endpoints, resource names, capacity settings) from agent logic. Use the /infra/environments/ parameter files for this.

Auditability and Review

Enforce pull request reviews for all changes to /src/prompts, /src/agents, and /infra via CODEOWNERS.
Require status checks to pass before merging. Blocked merges prevent untested changes reaching production.
GitHub's workflow run history gives you a complete deployment audit trail. You can answer "what was deployed to prod on Tuesday and who approved it" in seconds.
For regulated environments, consider branch protection rules that require signed commits.

Safe Rollout

Use canary or blue-green patterns where Foundry supports them for high-traffic agents.
Always keep the previous image tag available in the registry. Do not delete images on deployment.
Document and test your rollback procedure before you need it in production.

Observability and Operational Readiness

A deployed agent that you cannot observe is an agent you cannot operate. Build observability in from the start.

What to Monitor

Deployment health - Track whether each Foundry deployment succeeded and the agent is responding. Wire deployment outcomes back to GitHub workflow run status.
Model and tool errors - Log tool call failures, model timeout errors, and safety filter activations. Aggregate these in Azure Monitor or Application Insights.
Latency - Track end-to-end response latency per agent version. A latency increase after a model or prompt change is an early signal of a quality regression.
Token consumption - Monitor token usage per request and per session. Unexpected increases can indicate prompt injection or runaway orchestration loops.
Traceability - Log which agent version handled each request. Correlation between the image tag and request traces is essential for debugging production issues.

Debugging and Alerting

Use structured logging with a consistent schema. Include fields for agent version, session ID, tool called, and outcome.
Set up alerts for error rate thresholds and latency percentiles. Alert before users notice the problem.
For failed agent runs, ensure logs capture the full conversation context (within your data retention policy) so that developers can reproduce and diagnose the failure.

Microsoft Foundry Toolboxes

One of the most important additions to the Foundry platform is Toolboxes, currently in Public Preview. If you have ever seen an agent codebase where three different agents each wire the same search tool with their own credentials and slightly different configurations, you already understand the problem Toolboxes solve.

A Toolbox is a named, versioned bundle of tools managed centrally in Microsoft Foundry. You define the tools once, configure authentication and access centrally, and publish a single MCP-compatible endpoint. Any agent in any runtime consumes that endpoint without per-tool wiring, custom SDK integration, or duplicated credential management.

Figure: Before and after Foundry Toolboxes. Each agent previously managed its own tool connections. With Toolboxes, agents connect to one governed endpoint.

The Four Pillars

Discover (coming soon) - Find approved tools without browsing long catalogues. Reduces duplication by surfacing what already exists before developers build something new.
Build (available today) - Select tools into a named toolbox. Supported types include built-in tools (Web Search, Code Interpreter, File Search, Azure AI Search), MCP servers, Agent-to-Agent (A2A) endpoints, and OpenAPI-defined services.
Consume (available today) - A single MCP-compatible endpoint exposes every tool in the toolbox to any agent runtime. Agents that can speak MCP can use a Foundry Toolbox without any Foundry-specific SDK dependency.
Govern (coming soon) - Centralised authentication and observability applied to every tool call flowing through the toolbox. Security and platform teams get consistent controls without asking developers to bolt governance onto every agent individually.

Toolboxes and GitOps: A Natural Fit

Toolboxes are particularly well-suited to a GitOps delivery model because the toolbox definition is a discrete, versioned artefact. Instead of credentials and tool configuration scattered across agent codebases, the toolbox becomes its own managed entity with its own version history.

The key design property is that the toolbox endpoint URL is stable. When you promote a new toolbox version to be the default, agents consuming the endpoint pick up the update without any code changes. This means you can update tool configuration, add a new MCP server, or rotate credentials in the toolbox without redeploying every agent that uses it.

Figure: Toolbox versioning in a GitOps model. Commits trigger CI validation and deployment of new toolbox versions. The stable endpoint URL allows agents to consume updates without redeployment.

Adding a Toolbox to Your Repository

In your GitOps repository, toolbox definitions belong in /src/tools/toolbox_config.py or as a declarative configuration file checked into version control. The following example creates a toolbox that combines web search, Azure AI Search over internal documentation, and a GitHub MCP server:

# src/tools/toolbox_config.py
# Run this via CI to create or update a toolbox version in Foundry.

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
import os

client = AIProjectClient(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential()
)

toolbox_version = client.beta.toolboxes.create_toolbox_version(
    toolbox_name="customer-feedback-toolbox",
    description="Tools for triaging customer feedback: search, docs, and GitHub.",
    tools=[
        {
            "type": "web_search",
            "description": "Search approved public documentation sites.",
            "custom_search_configuration": {
                "project_connection_id": os.environ["BING_CONNECTION_NAME"],
                "instance_name": os.environ["BING_INSTANCE_NAME"]
            }
        },
        {
            "type": "azure_ai_search",
            "name": "product-manuals-search",
            "description": "Search internal product documentation.",
            "azure_ai_search": {
                "indexes": [
                    {
                        "index_name": os.environ["SEARCH_INDEX_NAME"],
                        "project_connection_id": os.environ["SEARCH_CONNECTION_ID"]
                    }
                ]
            }
        },
        {
            "type": "mcp",
            "server_label": "github",
            "server_url": "https://api.githubcopilot.com/mcp",
            "project_connection_id": os.environ["GITHUB_CONNECTION_ID"]
        }
    ],
)
print(f"Toolbox version created: {toolbox_version.version}")
print(f"MCP endpoint: {toolbox_version.mcp_endpoint}")

To promote a toolbox version to be the default (the endpoint agents use without specifying a version), add this to your deployment workflow:

# Promote toolbox version to default after validation
toolbox = client.beta.toolboxes.update(
    toolbox_name="customer-feedback-toolbox",
    default_version=toolbox_version.version,
)
print(f"Default version is now: {toolbox.default_version}")

The stable endpoint for agents consuming this toolbox is:

https://<your-project>.services.ai.azure.com/api/projects/<project>/toolbox/customer-feedback-toolbox/mcp?api-version=v1

Attaching the Toolbox to Your Hosted Agent

In your agent code, connect to the toolbox via a single MCP tool definition. The agent gains access to every tool in the toolbox without knowing their individual configurations:

# src/agents/agent.py (relevant excerpt)
from agent_framework import MCPStreamableHTTPTool
import httpx, os

toolbox_endpoint = os.environ["FOUNDRY_TOOLBOX_ENDPOINT"]

http_client = httpx.AsyncClient(
    auth=_ToolboxAuth(token_provider),  # Microsoft Entra bearer token
    timeout=120.0,
)

mcp_tool = MCPStreamableHTTPTool(
    name="toolbox",
    url=toolbox_endpoint,
    http_client=http_client,
    load_prompts=False,
)

# Agent now has access to web search, AI Search, and GitHub MCP
# through one tool definition and one authenticated connection.

GitOps Workflow Extension for Toolboxes

Add a dedicated job to your build-deploy workflow to create and promote toolbox versions as part of the same CI/CD pipeline:

  deploy-toolbox:
    name: Deploy Toolbox Version
    needs: validate
    runs-on: ubuntu-latest
    environment: dev
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Azure login (OIDC)
        uses: azure/login@v3
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID_DEV }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Create toolbox version in Foundry
        env:
          FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT_DEV }}
          BING_CONNECTION_NAME: ${{ vars.BING_CONNECTION_NAME }}
          BING_INSTANCE_NAME: ${{ vars.BING_INSTANCE_NAME }}
          SEARCH_INDEX_NAME: ${{ vars.SEARCH_INDEX_NAME }}
          SEARCH_CONNECTION_ID: ${{ vars.SEARCH_CONNECTION_ID }}
          GITHUB_CONNECTION_ID: ${{ vars.GITHUB_CONNECTION_ID }}
        run: python src/tools/toolbox_config.py

Key points to note:

Toolbox configuration is Python code in source control, reviewed through pull requests like any other change
Connection IDs and index names are environment variables from GitHub Actions variables, not hardcoded in the script
The same script runs for dev, test, and prod with different environment variable bindings
Toolbox version promotion is a separate step from agent deployment, so you can update tools independently of the agent container
Because the toolbox endpoint is stable, rolling back a toolbox version does not require rolling back the agent image

Common Pitfalls

Teams adopting this pattern commonly make the following mistakes. Identifying them early saves significant operational pain later.

Treating prompts as unmanaged text. If your system prompt lives in a portal text box rather than a versioned file, you have no history, no review process, and no rollback capability. Move prompts into source control on day one.
Deploying manually from the portal. Even one manual deployment breaks the GitOps contract. Your repository no longer reflects the true state of the environment. Automate everything and remove portal deployment permissions from individuals.
Mixing environment configuration into source files. Hardcoded endpoint URLs or model deployment names in agent_config.json mean your dev and prod configurations diverge at the source level. Use parameter files and environment variables resolved at deployment time.
Poor separation between agent logic and tool logic. When agents and tools are tightly coupled in a single file, a tool change requires a full agent review and redeployment. Keep them separate so they can evolve independently.
Not versioning your Toolbox definition. Defining a Foundry Toolbox interactively through the portal gives you no audit trail and no rollback path. The toolbox configuration script belongs in source control alongside your agent code.
Skipping evaluation before promotion. Deploying a prompt change without running a structured evaluation against a representative test set is how regressions reach production. Build evaluation into the pull request workflow, not just the deployment workflow.
No rollback plan. If your first rollback is unplanned and urgent, it will be slow and stressful. Test your rollback procedure in a non-production environment and document the steps.
Ignoring token and cost signals. AI workloads have variable cost profiles. A change that doubles average token consumption per request may be functionally correct but economically unsustainable. Monitor consumption as a first-class signal.

Example GitHub Actions Workflow

The following workflow runs on pull request validation and on merge to main. It covers the core delivery lifecycle: validate, build, deploy to dev, and smoke test.

# .github/workflows/build-deploy.yml

name: Build and Deploy Foundry Hosted Agent

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

env:
  REGISTRY: myregistry.azurecr.io
  IMAGE_NAME: my-foundry-agent

jobs:

  validate:
    name: Validate Agent Configuration
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Validate agent config schema
        run: python scripts/validate_agent.py

      - name: Run unit tests
        run: pytest tests/unit/ -v

      - name: Lint code
        run: ruff check src/

  build:
    name: Build and Push Container Image
    needs: validate
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    permissions:
      id-token: write
      contents: read
    outputs:
      image_tag: ${{ steps.meta.outputs.version }}
    steps:
      - uses: actions/checkout@v4

      - name: Azure login (OIDC)
        uses: azure/login@v3
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Log in to Azure Container Registry
        run: az acr login --name ${{ env.REGISTRY }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=sha,format=short

      - name: Build and push image
        uses: docker/build-push-action@v7
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}

  deploy-dev:
    name: Deploy to Foundry Dev
    needs: build
    runs-on: ubuntu-latest
    environment: dev
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Azure login (OIDC)
        uses: azure/login@v3
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID_DEV }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Deploy agent to Foundry Dev project
        run: |
          az ai foundry agent deploy \
            --project ${{ vars.FOUNDRY_PROJECT_DEV }} \
            --image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.build.outputs.image_tag }} \
            --environment dev

      - name: Run smoke tests against dev
        run: pytest tests/smoke/ -v --base-url ${{ vars.AGENT_URL_DEV }}

  deploy-test:
    name: Deploy to Foundry Test
    needs: deploy-dev
    runs-on: ubuntu-latest
    environment: test
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Azure login (OIDC)
        uses: azure/login@v3
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID_TEST }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Deploy agent to Foundry Test project
        run: |
          az ai foundry agent deploy \
            --project ${{ vars.FOUNDRY_PROJECT_TEST }} \
            --image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.build.outputs.image_tag }} \
            --environment test

      - name: Run smoke tests against test
        run: pytest tests/smoke/ -v --base-url ${{ vars.AGENT_URL_TEST }}

Key decisions in this workflow:

Validation runs on every pull request, not just on merge. Fast feedback catches problems before review.
The container image is built once and the image tag is passed forward to deployment jobs. The same artefact is promoted across environments.
Authentication uses OIDC federated credentials via azure/login@v3 with id-token: write permissions. No long-lived secrets are stored in GitHub for Azure authentication.
The environment: test directive in the deploy-test job triggers a GitHub environment approval gate. A named reviewer must approve before the job runs.
Smoke tests run after every deployment. A failed smoke test prevents further promotion.

Best Practices Checklist

Use this checklist when adopting the GitOps pattern for a Microsoft Foundry Hosted Agent:

All agent artefacts, including prompts, tool definitions, model configuration, and Toolbox configuration scripts, are committed to source control
No manual deployments to any environment; all changes flow through GitHub Actions workflows
Pull request reviews are enforced for all changes to agent logic, prompts, and infrastructure via CODEOWNERS
Unit tests cover tool logic; integration tests cover end-to-end agent behaviour; smoke tests cover deployed environments
Container images are built once per commit and promoted across environments; images are not rebuilt per environment
Environment configuration (endpoints, resource names) lives in parameter files, never in source code
Secrets are stored in Azure Key Vault and accessed via managed identity at runtime
GitHub environment approval gates control promotion from dev to test to prod
Foundry Toolboxes are used to centralise tool definitions, credentials, and access governance across all agents; the toolbox configuration script is version-controlled and deployed through CI/CD
Toolbox versions are promoted via the update default_version API step in the deployment workflow, not manually through the portal
Latency, error rate, and token consumption are monitored with alerting thresholds
The rollback procedure is documented, automated, and has been tested in a non-production environment
GitHub Issues are used to record the intent behind significant changes and link to the pull requests that implement them
Branch protection rules prevent direct pushes to main and require status checks to pass before merge
The previous image tag is retained in the registry and stored as a GitHub environment variable for rollback

Conclusion

A Microsoft Foundry Hosted Agent is not something you deploy once and forget. Prompts evolve, tools change, models are upgraded, and policy requirements shift. Every one of those changes has the potential to alter agent behaviour in ways that affect users, costs, and compliance posture.

GitOps, implemented through GitHub and GitHub Tasks, gives you the operational discipline to manage that complexity. Source control for all artefacts. Pull request review for every change. Automated validation, build, and deployment. Environment promotion gates. A complete audit trail from task to production. These are not bureaucratic overhead; they are the foundation of reliable, trustworthy AI agent operations.

The teams that operate AI agents well are the ones that treat them like production software from the start. The investment in pipeline, structure, and governance pays back every time a change goes smoothly, every time a rollback takes minutes rather than hours, and every time a security or compliance reviewer can answer their question from a pull request history rather than a support ticket.

Build the discipline in early. Your future self, and your production environment, will benefit from it.

References

Building a GitHub Copilot Agent Usage Dashboard

jometzg — Mon, 01 Jun 2026 19:00:00 GMT

Introduction

Working with organisations that are attempting to create GitHub Copilot custom agents, take-up of these agents by their community becomes important to know. Some questions quickly emerge are "how well are we actually using it?", "which agents are getting used and which have not had that much traction?".

Native metrics provide high-level insights into adoption, but they lack the depth needed to answer more granular questions—such as which agent workflows are most used, or how behaviour evolves over time.

In this post, I’ll walk through how to build an enterprise-grade GitHub Copilot usage dashboard that captures detailed telemetry from VS Code using OpenTelemetry, processes it in Azure Monitor, and visualises insights in Grafana—all using a reproducible, infrastructure-as-code approach. The dashboard can be made available to anyone that needs it.

Architecture

VS Code can be configured to emit metrics using Open Telemetry as a standard. This is a configuration item in VS Code and you essentially point it to an Open Telemetry Collector. The collector is an endpoint that can consume the telemetry.

In this implementation, it is a container image that is hosted in Azure and I have chosen Azure Container Apps (ACA) for this purpose as it is an easy to use managed environment - but it could also run in Azure Kubernetes Service (AKS) with a little more effort.

There is a prebuilt image opentelemetry collector for this and this has been adapted to inject configuration to send the telemetry to Azure Application Insights.

For defining and hosting the dashboard, I have chosen another Azure managed service Azure Managed Grafana

Sample Dashboard

The sample dashboard is one that contains a collection of visualisations derived from the collected data in Application Insights. Azure Managed Grafana allows you to visually author these dashboards or they can be implemented as a JSON file and adapted from there.

Note that the telemetry generated by VS Code gives the location of the users - city, region and country, but does not include any personally-identifying information (PII) and so cannot be used to track individuals. As I understand it, this is by design.

Managed Grafana has its own permission structure, which may then be used to give users access to the dashboard.

Implementation Details

There is a GitHub repo Copilot Usage Dashboard that contains details of how to implement this together with instructions for either "click-ops" 🙂creation or via Terraform. So I suggest you follow the link to my repo to look at the details. In summary, there needs to be in Azure:

Azure Container App (ACA) that hosts the collector - this needs to have public ingress
Azure Container Registry (ACR) that hosts the docker image that is customised via the Dockerfile
Key Vault that hosts the Application Insights connection string that ACA references
Application Insights - this needs to be created with a flag to allow it to work with Grafana data
Log Analytics Workspace that works with Application Insights
Azure Managed Grafana to host the Grafana dashboard

The main thing to bear in mind is that VS Code needs to be configured to emit OpenTelemetry

{ "github.copilot.nextEditSuggestions.enabled": true, "github.copilot.chat.otel.enabled": true, "github.copilot.chat.otel.exporterType": "otlp-http", "github.copilot.chat.otel.otlpEndpoint": "https://<fqdn>" }

where the FQDN is the URL of the public ingress to the Azure Container App.

There is a Dockerfile in this repo that just injects the correct configuration file into the OpenTelemetry collector image. It is this configuration file that tells the collector to emit to Application Insights. It is of the form below:

receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: attributes: actions: - key: environment value: "prod" action: upsert exporters: azuremonitor: connection_string: "${APPLICATIONINSIGHTS_CONNECTION_STRING}" debug: verbosity: detailed service: pipelines: traces: receivers: [otlp] processors: [batch, attributes] exporters: [azuremonitor, debug] metrics: receivers: [otlp] processors: [batch, attributes] exporters: [azuremonitor]

As can be seen above, there is a placeholder for the Application Insights connection string - in the ACA configuration this is an environment variable that then points to a secret which is in key vault.

If all is well, VS Code will emit telemetry to the container image running in ACA and this will use its configuration to send to Application Insights. The Grafana dashboard then using this data.

Troubleshooting

The GitHub repo goes into the detail of troubleshooting, but the overall steps to troubleshoot are:

If there is no data in Grafana, check that Grafana has access to Application Insights
check whether there is telemetry being pushed into Application Insights by looking at the logs and looking for the contents of the table Dependencies. If there is telemetry there, then it is Grafana permissions. If not, look to ACA
Look at ACA logs to see if it is healthy and look to see if there is any logs being received
Use a curl request to send a fake log to ACA (a sample is in the repo) to see if the ACA is accepting logs
Check the connection to Application Insights is correct and is being pulled from key vault or replace the environment variable value with the connection string directly
If all good so far, then it may be that the configuration in VS Code is not correct or in the correct place.

Hopefully the more detailed steps will resolve any issues quickly.

Further thoughts and enhancements

This implementation attempts to build a dashboard showing GitHub Copilot agent usage using a standard set of security controls, but more may be needed. Here is a list of possible enhancements:

A more refined dashboard. This should be easy as there are samples for all sorts of visualisations and few of these may allow more focus on agent and model usage.
the ACA-hosted OpenTelemetry collector has a public-facing ingress. This may need to be locked-down at the network level by address restriction or by a non-public ingress. Care would need to be taken to make sure that this is then visible/reachable to the intended VS Code user audience
The ACA collector endpoint is not authenticated in of itself. This could be achieved at the container level by putting an authenticating proxy in the Dockerfile or at the ACA ingress level. Some investigation would be needed to see how the VS Code configuration could work with this and this may dictate largely what form this authentication can take.
How the VS Code configuration changes can be automated for a user base has not been investigated as part of this work. It is assumed that an organisation may be able to roll-out these changes using their application deployment automation.

Summary

This approach provides a means by which an organisation can track the usage of GitHub Copilot agents (and their models), that is not provided by GitHub Enterprise dashboards. This will provide insights into the take up of custom agents and their underlying models - allowing an organisation to test whether their investments on custom agents are being used effectively. Additionally, the dashboards themselves can easily be rolled-out to a wider community than GitHub Enterprise one.

Claude Code on Microsoft Foundry in VS Code — A Practical Setup Guide (with the gotchas)

LZhang — Mon, 01 Jun 2026 13:40:49 GMT

Enables enterprise-grade governance without changing your developer workflow.

The official Microsoft Learn article (Configure Claude Code for Microsoft Foundry) gets you ~80% of the way there. The remaining 20%—VS Code settings shape, tenant mismatches, and configuration conflicts like "baseURL and resource are mutually exclusive"—is where most setups fail in practice.

This guide walks the full path end-to-end, with the exact JSON that validates, working CLI configuration, and a troubleshooting matrix based on real-world failures. This guidance is based on repeated customer deployments and internal testing across both CLI and VS Code scenarios.

TL;DR

Setup - Deploy claude-sonnet-4-6 (optionally Haiku + Opus) in a supported region - Grant Cognitive Services User + Foundry User - az login --tenant <tenant>, then launch VS Code via code .

Config - CLI: - CLAUDE_CODE_USE_FOUNDRY=1 - ANTHROPIC_FOUNDRY_RESOURCE=<name> - Do NOT set ANTHROPIC_FOUNDRY_BASE_URL at the same time - VS Code: - Use [{ "name": "...", "value": "..." }] format

Validate - claude → /status - Expect: API provider: Microsoft Foundry

Why run Claude Code on Foundry?

Anthropic's Claude Code is a top-tier agentic coding assistant. Running it through Microsoft Foundry instead of Anthropic's public API gives you:

Data residency & compliance: prompts and completions stay inside your Azure tenant.
Entra ID auth: no API keys to rotate; centralized RBAC.
Private networking: works behind VNets/Private Endpoints.
Unified billing & quotas: usage shows up on your Azure invoice and in Foundry monitoring.

Same model, same CLI, enterprise-grade plumbing underneath.

Prerequisites checklist

Requirement	How to verify
Azure subscription with pay-as-you-go billing	`az account show`
Foundry resource in supported regions	Check your region's model availability in Foundry portal
Contributor/Owner on the resource group (for deployments)	Azure Portal → IAM
Cognitive Services User + Foundry User on the resource (for invoking)	Azure Portal → IAM
Azure CLI installed and logged in	`az --version`, `az login`
Claude Code CLI installed	`claude --version`
VS Code (current) with the Anthropic Claude Code extension	Help → About
Windows only: Git Bash (from Git for Windows) or WSL2 — Claude Code's runtime requires a POSIX shell	`bash --version` in Git Bash / WSL

⚠️ Claude models in Foundry are currently available in select regions. Check the Foundry portal model catalog for your region's availability (commonly East US 2 and Sweden Central).

Step 1 — Deploy the Claude models

Claude Code uses three model roles, and it expects a deployment for each:

Role	Default deployment name	Used for
Primary	`claude-sonnet-4-6`	general coding (balanced)
Fast	`claude-haiku-4-5`	quick edits, file reads
Extended thinking	`claude-opus-4-6`	complex reasoning

Deploy at least Sonnet to get started. Add Haiku and Opus when you need them — Claude Code will route automatically. If a role-specific model isn't deployed, Claude Code may fall back or fail depending on the task.

Deployment names in this guide follow the current Claude 4.x naming exposed in Foundry. Exact versions change over time — check the Foundry model catalog in your region for what's currently available.

Foundry Portal: AI Foundry → your project → Build → Models + endpoints → + Deploy model → pick the Anthropic Claude model → Global Standard deployment → name it exactly as above (or remember the name to override later).

To discover the current model version before deploying (replace eastus2 with your Foundry region):

az cognitiveservices model list -l eastus2 `
  --query "[?contains(model.name,'claude')].{name:model.name, version:model.version, format:model.format}" -o table

Azure CLI:

az cognitiveservices account deployment create `
  --name <foundry-resource> `
  --resource-group <rg> `
  --deployment-name claude-sonnet-4-6 `
  --model-name claude-sonnet-4-6 `
  --model-version <version> `
  --model-format Anthropic `
  --sku-name GlobalStandard `
  --sku-capacity 1

✍️ Figure 1: Foundry portal “Models + endpoints” showing the three Claude deployments.

Step 2 — Grant yourself the right roles

This is the #1 source of silent failures. You need both:

Role	Role ID	Purpose
Cognitive Services User	`a97b65f3-24c7-4388-baec-2e87135dc908`	data-plane invocation
Foundry User (formerly Azure AI User)	`53ca6127-db72-4b80-b1b0-d745d6d5456d`	Foundry-native permissions

$me = az ad signed-in-user show --query id -o tsv
$scope = az cognitiveservices account show -n <foundry-resource> -g <rg> --query id -o tsv

# Use role IDs — rename-proof (works whether the display name is "Azure AI User" or "Foundry User")
az role assignment create --assignee $me --role a97b65f3-24c7-4388-baec-2e87135dc908 --scope $scope  # Cognitive Services User
az role assignment create --assignee $me --role 53ca6127-db72-4b80-b1b0-d745d6d5456d --scope $scope  # Foundry User (formerly Azure AI User)

The Foundry RBAC rename (Azure AI User → Foundry User) is rolling out; both role names map to the same role definition (same role ID), depending on tenant rollout state. Use whichever role name your tenant exposes — or use the role IDs above to avoid ambiguity.

Step 3 — Install the Claude Code CLI

Use the official installer from Anthropic (auto-updates in the background):

irm https://claude.ai/install.ps1 | iex
claude --version

If claude isn't on PATH, restart your shell. The installer drops it under %USERPROFILE%\.local\bin.

If your Foundry resource lives in a tenant different from your default, an az login to the wrong tenant produces the cryptic error:

ValueError: Unable to get authority configuration for
https://login.microsoftonline.com/<bad-guid>.
Authority would typically be in a format of
https://login.microsoftonline.com/your_tenant

Fix:

az login --tenant <foundry-tenant-guid>
az account set --subscription <foundry-subscription-guid>
az account show   # confirm tenant & subscription

💡 You can list every tenant you have access to with: az account list --query "[].{name:name, tenantId:tenantId}" -o table

Step 5 — Configure the CLI

Set these in the same PowerShell session you'll launch claude from:

$env:CLAUDE_CODE_USE_FOUNDRY     = "1"
$env:ANTHROPIC_FOUNDRY_RESOURCE  = "<your-foundry-resource-name>"

# Optional: only if your deployment names differ from the defaults
$env:ANTHROPIC_DEFAULT_SONNET_MODEL = "claude-sonnet-4-6"
$env:ANTHROPIC_DEFAULT_HAIKU_MODEL  = "claude-haiku-4-5"
$env:ANTHROPIC_DEFAULT_OPUS_MODEL   = "claude-opus-4-6"

To make them persistent: setx CLAUDE_CODE_USE_FOUNDRY 1 (and so on), then sign out and back in (or restart Explorer). GUI apps like VS Code launched from the Start menu only pick up new user-env vars after the user session refreshes — opening a fresh terminal isn't enough.

🚫 The "mutually exclusive" trap

API Error: baseURL and resource are mutually exclusive

You'll hit this if you set both ANTHROPIC_FOUNDRY_RESOURCE and ANTHROPIC_FOUNDRY_BASE_URL. Pick one:

Most users → ANTHROPIC_FOUNDRY_RESOURCE=<name> (Claude Code builds the URL).
Custom subdomain / private endpoint → use ANTHROPIC_FOUNDRY_BASE_URL instead.

Step 6 — Verify the CLI

claude
> /status

Expected output:

API provider:                 Microsoft Foundry
Microsoft Foundry base URL:   https://<resource>.services.ai.azure.com/anthropic
Microsoft Foundry resource:   <resource>
Model:                        Default (claude-sonnet-4-6)

✍️ Figure 2: /status output confirming API provider: Microsoft Foundry.

If you instead see "Anthropic" or it prompts for an Anthropic login, CLAUDE_CODE_USE_FOUNDRY isn't being inherited — see troubleshooting below.

Step 7 — Configure the VS Code extension

Install Claude Code from the VS Code Marketplace (publisher: Anthropic).

Open user settings.json (Ctrl+Shift+P → Preferences: Open User Settings (JSON)) and add:

"claudeCode.environmentVariables": [
  { "name": "CLAUDE_CODE_USE_FOUNDRY",    "value": "1" },
  { "name": "ANTHROPIC_FOUNDRY_RESOURCE", "value": "<your-foundry-resource-name>" }
]

🪤 Schema gotcha. The MS Learn doc currently shows this as a plain {KEY: VALUE} object under the UI label "Claude Code: Environment Variables". In recent extension versions the actual JSON key is claudeCode.environmentVariables and the value must be an array of {name, value} objects. If you paste the doc's snippet verbatim, VS Code will flag "Missing property name", "Colon expected", "Unknown configuration setting". Use the array form above.

Make the extension see your `az login`

The extension inherits environment & credentials from the process that launches VS Code. After az login:

# In the same PowerShell where az login succeeded:
code .

If VS Code was already running, fully quit it (not just close the window) and relaunch from the terminal. Developer: Reload Window is not enough to refresh inherited Azure CLI credentials.

✍️ Figure 3: settings.json with the claudeCode.environmentVariables array form.

Step 8 — Try it

In VS Code, click the Claude Code (Spark) icon in the sidebar to open the panel. Type:

Summarize the structure of this project.

You should get a response within a few seconds, and the panel should indicate it's routing through Microsoft Foundry. Run /status inside the panel to confirm API provider: Microsoft Foundry if you want certainty.

✍️ Figure 4: Claude Code panel in VS Code responding through Microsoft Foundry.

Troubleshooting matrix

Symptom	Where it shows up	Likely cause	Fix
`API Error: baseURL and resource are mutually exclusive`	`claude` CLI on first request	Both `ANTHROPIC_FOUNDRY_BASE_URL` and `ANTHROPIC_FOUNDRY_RESOURCE` set	Unset one. Prefer `ANTHROPIC_FOUNDRY_RESOURCE`.
`Unable to get authority configuration for https://login.microsoftonline.com/<guid>`	`claude` CLI startup or VS Code panel	Wrong tenant ID in `az login`	`az login --tenant <correct-guid>`; verify with `az account show`
`Failed to get token from azureADTokenProvider: ChainedTokenCredential authentication failed`	VS Code Claude Code panel	Extension didn't inherit `az login` session	Quit VS Code entirely; relaunch with `code .` from the authed shell
`Token tenant does not match resource tenant`	`claude` CLI or VS Code panel	CLI logged into a different tenant than the Foundry resource	`az login --tenant <foundry-tenant>`
`The model <name> is not available on your foundry deployment`	`claude` CLI first use or VS Code model selector	Deployment name mismatch	Either rename the Foundry deployment, or set `ANTHROPIC_DEFAULT_*_MODEL` to the actual name
`401 / 403` on first request	`claude` CLI or VS Code panel	Missing RBAC on the resource	Assign Cognitive Services User and Foundry User on the resource scope
Claude Code prompts for Anthropic login	VS Code Claude Code panel	`CLAUDE_CODE_USE_FOUNDRY` not set in the process	Set the env var before launching `claude` / `code .`
VS Code shows "Unknown Configuration Setting" for `claudeCode.environmentVariables`	VS Code Settings tab	Wrong JSON shape	Use the array of `{name,value}` objects form
`429 Too Many Requests`	`claude` CLI or VS Code panel	TPM/RPM exhausted	Foundry portal → Operate → Quotas; request increase or reduce parallelism
Works in CLI, fails in VS Code extension	VS Code Claude Code panel only	Env vars set per-shell, not visible to GUI VS Code	Use `setx` (persistent user env) or move them into `claudeCode.environmentVariables`
"Model is not available in region"	Foundry portal model deployment step	Foundry resource not in a supported region	Deploy a new Foundry resource in a supported region, or check model availability

Best practices

Auth & secrets - Prefer Entra ID over API keys. If you must use a key for CI, store it as a secret (GitHub Actions secret, Key Vault) — never in settings.json (it may sync via Settings Sync). - Scope RBAC at the resource level, not the subscription, for least privilege.

Project context - Create a CLAUDE.md at your repo root with stack, conventions, and entry-point commands. Claude Code reads it automatically and the quality jump is significant. - Use .claude/rules/*.md for per-area rules (e.g., test conventions, security rules).

Cost & latency - Let Claude Code's auto-routing pick the right role (Sonnet/Haiku/Opus). Don't pin everything to Opus. - Cap context with ANTHROPIC_MAX_TOKENS if you have a strict budget. (Note: not honored by every Claude Code version — check the Claude Code docs for your version.) - Watch token spend in Foundry → Operate → Metrics weekly.

Reliability - For team use, deploy all three model roles even if you don't think you need them — silent role-routing failures are confusing. - Tag your Foundry resource (env=dev|prod, team=...) for chargeback.

Reproducibility - Document the exact env vars and az login --tenant GUID in your team README. - Pin Claude Code CLI version in onboarding docs (claude --version) so new joiners hit the same behavior.

A note on the MS Learn doc

The doc is accurate but skips three things that caused the most friction in real-world deployments:

VS Code extension settings shape — the example uses the UI label as a JSON key and an object instead of the array form the schema actually expects.
Process inheritance — it says "set the env vars" but doesn't emphasize that the VS Code window must be launched from a shell where both az login and the env vars are live. Reloading the window doesn't help.
Mutually exclusive RESOURCE vs BASE_URL — listed in passing, but the error message only appears at first request, after you think everything is configured.

If the Microsoft Learn page is updated, treat this post as a companion — same destination, fewer dead ends.

What you've got now

Claude Code running locally on your machine, talking to your Foundry resource.
Entra ID auth — no API keys to manage.
Full Foundry telemetry, quotas, and billing.
VS Code panel + CLI, both backed by the same setup.

Drop a CLAUDE.md in your repo and start shipping.

When to Use RESOURCE vs BASE_URL

Use RESOURCE (default) - Standard public deployments - No custom networking

Use BASE_URL - Private endpoints - Custom DNS / VNet routing

Never set both.

Agents That Build Agents: A SKILL-first Blueprint with MS Agent Framework & Foundry

kinfey — Sun, 31 May 2026 07:51:50 GMT

The single insight that changes everything

Most "build an AI agent" tutorials collapse two completely different jobs into one tangled mess:

the job of building an agent (writing the code, defining its tools, evaluating it, packaging it), and
the job of running an agent (planning, reasoning, calling tools, remembering users, delivering outcomes).

Once you separate them, modern agent development becomes a clean two-layer architecture:

A Coding Agent sits on top — that's how you produce an agent. A Runtime Agent sits below — that's the agent your business operates. Microsoft Agent Framework is the SDK that ties them together; Microsoft Foundry is the platform both layers publish to and run on.

But the secret ingredient — the thing that turns a generic Copilot into a domain-aware engineer — is the SKILL. SKILL is what the Coding Agent reads before writing a single line. It's how requirements become artifacts that actually match your framework, your conventions, and your fixtures.

This post walks the entire two-layer architecture, in the order you should learn it — with SKILL as the star of Layer 1. We ground every concept in ZavaShop, a fictional global e-commerce company with 5 fulfillment centers, dozens of suppliers, and a CEO who wants one live dashboard for all of it. Both Python and .NET (C#) are first-class — pick the language your team will run in production.

LAYER 1 — The Coding Agent (Build Time)

The Coding Agent is not the agent your customer talks to. It's the agent that constructs the agent your customer talks to. Its output is a bundle of artifacts — code, agent definitions, workflows, skills, connectors, evals, tests, configs, docs — that flow through validation and into Foundry.

Build time has five movements.

Movement 1 — Requirements & Planning

Before the Coding Agent writes a single line, you owe it three things:

A real business pain. Not "let's build an agent." Rather: "Mei, the supervisor at Seattle DC, gets interrupted 60 times a day by stock-level questions."
A list of acceptance criteria. What does "done" look like? "Agent answers stock questions for SKUs in our 10-SKU catalog. P95 latency under 4s. Wrong-tool rate under 5% on the eval set."
The fixtures it'll run on. Real or realistic data — warehouses, SKUs, POs, customers — so the Coding Agent isn't reasoning about a vacuum.

ZavaShop context. The workshop ships workshop/data/ — 5 warehouses, 10 SKUs, 6 POs, 8 suppliers, 5 contracts, 4 customers (3 VIP), 6 orders, 5 carriers, 4 open exceptions. Every artifact the Coding Agent generates is anchored to this shared fixture set, so numbers stay consistent across the entire system.

Movement 2 — The Coding Agent + its SKILL (the star of build time)

This is the movement most teams skip — and it's the one that decides whether your build-time output is professional code or "ChatGPT-shaped" code.

What a Coding Agent actually is

The Coding Agent is GitHub Copilot Chat in Agent Mode, configured with a domain-aware agent definition. In the ZavaShop workshop, it lives at .github/agents/zavashop-coding-agent.agent.md and is activated from the VS Code Agent picker. You start each session with one plain sentence:

"I'm working on the inventory agent in Python — wire up stock and PO lookups against the fixtures, plus a HostedMCPTool for the warehouse handbook."

Notice what's not in that sentence: no library names, no class names, no file paths. The Coding Agent has to fill all of that in. The mechanism it uses is the SKILL.

What a SKILL is

A SKILL is a structured contract that teaches the Coding Agent how to write code in your framework, your conventions, and your domain. It is the most important file in the entire build-time layer — without it, GitHub Copilot is a fluent generalist; with it, it becomes a domain-aware specialist that writes code your tech leads would have written.

Conceptually, a SKILL contains:

Section	Purpose
Scope & when to use	"Use this SKILL for building agents on Foundry / Azure AI — tools, MCP, Toolbox, Skills, Memory, Threads"
Framework idioms	The exact way to construct AzureAIAgentClient, register function tools, wire HostedMCPTool, create a Thread
Code patterns	Reference snippets the Coding Agent imitates — naming, import order, error handling, type hints
Fixture/data contract	How to load workshop/data/, which loaders exist (find_stock, find_po, etc.), where to add sys.path
Anti-patterns	What not to do — don't hardcode the model name, don't write inline mock dicts, don't bypass the data loader
Acceptance heuristics	How to map a LAB's acceptance criteria to runnable checks (eval rows, smoke tests)

A SKILL is versioned with the codebase. When the framework releases a new idiom, you update the SKILL once; every agent built afterwards picks it up automatically. This is the single biggest reason convention drift disappears.

The six SKILLs in the ZavaShop workshop

The workshop ships six SKILLs — three for each language track — and they cover three orthogonal capability surfaces:

Track	SKILL	Use it for
🐍 Python	agent-framework-azure-ai-py	Single agent on Foundry: tools, MCP, Toolbox, Skills, Memory, Threads
🐍 Python	agent-framework-workflows-py	Multi-agent workflows: WorkflowBuilder, executors, HITL, Checkpoint
🐍 Python	agent-framework-agui-py	AG-UI server + client: SSE, frontend/backend tools, shared state, HITL
🟦 .NET	agent-framework-azure-ai-csharp	Same as the Python azure-ai SKILL, for C#
🟦 .NET	agent-framework-workflows-csharp	Same as the Python workflows SKILL, for C#
🟦 .NET	agent-framework-agui-csharp	AG-UI in ASP.NET Core: MapAGUI, AGUIChatClient, HITL

How the Coding Agent uses SKILL

The Coding Agent's loop is SKILL-first, code-second:

The discipline is captured in the workshop's one mantra:

"Read the SKILL first."

It is not optional. Skip it and you're back to generic Copilot output.

Movement 3 — A worked example: from a single sentence to a runnable agent

Let's trace what happens when you sit down in front of the Coding Agent and say:

"I'm working on the inventory agent in Python — wire up stock and PO lookups against the fixtures, plus a HostedMCPTool for the warehouse handbook."

Step 1 — The Coding Agent routes

The Coding Agent's definition (.github/agents/zavashop-coding-agent.agent.md) contains a routing table:

It loads both files into context before doing anything else.

Step 2 — The Coding Agent plans

Using the SKILL's patterns and the LAB's acceptance criteria, it generates a plan:

Every item in the plan traces back to either the SKILL (how) or the LAB README (what). No invention, no guessing.

Step 3 — The Coding Agent generates

Now — and only now — it writes code. Because the SKILL specified the patterns, what comes out is shaped like the rest of your codebase:

# scripts/inventory_agent.py import os import sys from pathlib import Path # Pattern from SKILL: add data folder to sys.path, import named loaders sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "data")) from zava_data import find_stock, find_po # noqa: E402 from agent_framework import HostedMCPTool from agent_framework_azure_ai import AzureAIAgentClient from azure.identity import DefaultAzureCredential def stock_lookup(sku: str, warehouse: str) -> dict: """Return on-hand quantity for a SKU at a warehouse.""" return find_stock(sku, warehouse) def po_lookup(po_id: str) -> dict: """Return the status of a purchase order.""" return find_po(po_id) async def main() -> None: client = AzureAIAgentClient( project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], model_deployment_name=os.environ["FOUNDRY_MODEL"], # gpt-5.5 from env async_credential=DefaultAzureCredential(), ) agent = client.create_agent( name="Zara", instructions=( "You are Zara, the warehouse assistant for Seattle DC (SEA-01). " "Use the tools when asked about stock or POs. " "Answer briefly." ), tools=[ stock_lookup, po_lookup, HostedMCPTool( name="warehouse-docs", url=os.environ["WAREHOUSE_DOCS_MCP_URL"], ), ], ) thread = agent.get_new_thread() print(await agent.run("How many SKU-7421 are at SEA-01?", thread=thread))

Notice the things the SKILL enforced without you having to ask: env-var-driven config, named function tools with docstrings, sys.path data-loader pattern, HostedMCPTool placed alongside function tools, Thread for multi-turn.

Step 4 — The Coding Agent validates

The SKILL also told it how to validate. The Coding Agent runs:

a smoke test against fixtures (SKU-7421 @ SEA-01 → 312),
the eval set (eval_queries.jsonl) — was the right tool called? did the answer contain the expected fact?
a red-team probe round.

It reports back: "3/3 acceptance criteria pass. Eval score 5/5. Red-team: no successful prompt injections."

Step 5 — Done

What landed in your repo is not just a script. It's an artifact bundle — code + agent definition + tools + eval rows + a one-page README — that matches the way your team writes agents. That bundle is what flows into the next three movements.

Movement 4 — Agent Artifacts (the outputs)

A well-instructed Coding Agent produces eight kinds of artifact. Together they make up "an agent" in the deployable sense:

Artifact	What it is	Why it matters
Source code	The Agent / Workflow program	Versioned, reviewable, diffable
Agent definitions	Name, instructions, tool list	The "personality" — independently editable
Workflows	WorkflowBuilder graphs	Multi-agent orchestration as code
Skills	Named, packaged behaviors	Reusable capabilities — one Skill, many agents
Connectors	MCP servers, Toolbox registrations	Where the agent reaches into the world
Evals	eval_queries.jsonl and harness	Regression target for every prompt change
Tests & configs	Unit tests, .env schema, deployment manifests	Reproducibility
Documentation	READMEs, runbooks	The agent your future self can operate

Don't confuse two senses of "skill" here. A SKILL file (uppercase, in .github/skills/) instructs the Coding Agent at build time. An Agent Skill (a Foundry concept) is a named runtime capability the Runtime Agent calls. Both names are deliberate — Layer 1's SKILL produces, among other artifacts, Layer 2's Skills.

Movement 5 — Validation

Before any artifact reaches Foundry, four gates run:

Tests — unit + integration. Did find_stock("SKU-7421", "SEA-01") return 312, the value in the fixture?
Lint & types — ruff/mypy on Python, dotnet build warnings on .NET. The model has to read these signatures; sloppy ones cause real bugs.
Evaluation — run the eval set. Did the right tool get called? Did the answer contain the expected fact? You need a score, not a vibe.
Red-Team probes — adversarial inputs that try to drift the agent off topic or extract another customer's data. The Foundry red-team SDK ships a battery of these.

Evangelist takeaway. "We built an agent" is not a deliverable. "We built an agent and here is its pass rate on a versioned eval set, plus a red-team report" is a deliverable. Validation belongs at build time, not "we'll add it later."

Movement 6 — Publish & Deploy

When validation is green, the Coding Agent's outputs flow into Foundry and Azure:

Push to Microsoft Foundry — agent definitions, Skills, Toolbox tools, and custom evals register against your Foundry project. They are now governed, versioned, and observable.
Deploy to Azure — the runtime host (AG-UI server, workflow worker, Teams app, API surface) ships to your Azure target (App Service, Container Apps, AKS, Functions). Same env vars drive local dev and cloud.

The same artifact set deploys to dev, staging, and production. There is no "production-only" code in your agent.

LAYER 2 — The Runtime Agent (Runtime)

Now the agent is live. Every conversation, every action against your data, every memory it writes — that's Layer 2. Five concerns define it.

Concern 1 — Users & Channels

A Runtime Agent reaches users through the channels they already use:

Microsoft Teams — the agent shows up where work already happens.
Outlook — triage, reply, summarize, schedule.
Custom web / mobile / voice — built on AG-UI, which ships a React client covering streaming text, frontend tools, backend tools, shared state, generative UI, predictive updates, HITL prompts.

The channel is a deployment choice, not an architectural choice. The same agent definition can surface in Teams and on a React dashboard.

ZavaShop context. Mei's agent shows up in Teams. The CEO's control tower is a React app on top of AG-UI. The agent definition behind both is the same artifact set the Coding Agent produced.

Concern 2 — The Runtime Agent itself

The Runtime Agent is the loop you've heard about a thousand times — now it's a concrete piece of architecture:

AIAgent = model + instructions + tools + thread

Inside the loop:

The model plans & reasons about the next step.
It calls tools through MCP, Toolbox, or local functions.
It reads & writes memory.
It streams output back to the channel.

# Python — the runtime shape (exactly what the Coding Agent produced) agent = client.create_agent( name="Zara", instructions="You are Zara, the warehouse assistant for Seattle DC.", tools=[stock_lookup, po_lookup, warehouse_docs_mcp], )

Concern 3 — Tools & Integrations (the runtime capability surface)

At runtime, a Runtime Agent reaches the outside world through four kinds of capabilities — and which one to use is a real engineering decision:

Capability	Lives in	Use when
Function tool	The agent's own process	Local code: a calculation, a DB query, a fixture lookup
MCP tool	An external MCP server	The capability is owned by another system, exposed via MCP
Toolbox tool	The Foundry project (server-side, tenant-wide)	Capability is shared by multiple agents, must be governed
Agent Skill	The Foundry project	A combination of tools + policy as one named capability

Mental progression:

You don't have to start with Toolbox — but the moment a second agent touches the same domain, migrate.

ZavaShop context. Local fixtures → function tools. The warehouse handbook → MCP. Supplier-portal connectors shared by procurement, fulfillment, and finance → Toolbox tools. "Validate-PO-against-contract" → an Agent Skill.

Concern 4 — Memory & State

State at runtime comes in two flavors:

Thread = state inside one conversation

thread = agent.get_new_thread() await agent.run("Look up PO-1043.", thread=thread) await agent.run("And its supplier?", thread=thread) # knows which PO

Memory = state across conversations

Foundry Memory is durable, retrievable knowledge about a user — VIP status, packaging preferences, delivery windows. Memory holds stable preferences and facts, not chat transcripts.

ZavaShop context. Customer service agent Aria remembers across sessions that C-204 is VIP, prefers no cardboard, and wants 6–8pm delivery.

Concern 5 — Actions & Outcomes

Real systems take actions that change state and produce outcomes other systems observe:

Trigger events — kick off a workflow, page a human.
Generate outputs — write a PO, draft an email, push to a record.
Notify channels — send back to Teams, update a dashboard, hit a webhook.
Observability — every action streams to Application Insights / Azure Monitor.

This is also where Workflows live. WorkflowBuilder is Agent Framework's orchestration primitive:

Three workflow features matter most:

Reuse, don't rebuild — tools written at build time are workflow nodes at runtime.
Human-in-the-Loop (HITL) — pauses, asks a human, resumes from the exact step.
Checkpointing — workflows survive process restarts.

ZavaShop context. Fulfillment director Diego's team handles a $10K+ exception every day. Before: an email chain across 5 teams. After: a WorkflowBuilder graph with one HITL approval and full audit trail.

Cross-cutting: the shared services that make this safe

Both layers sit on top of platform services non-negotiable for enterprise deployment:

Service	What it does for your agents
Microsoft Entra ID	Who is the user? Who is the agent? Managed identity for tool calls
Microsoft Defender for Cloud	Threat detection across the agent's compute + data plane
Microsoft Sentinel	SIEM — correlate agent actions with security signals
Azure Key Vault	Secrets, keys, connection strings — never in code, never in .env checked to git
Azure Monitor / App Insights	Every agent turn, every tool call, every workflow step — observable and queryable
Azure Policy & governance	Guardrails on what can be deployed where, by whom

Skip this row and you have a demo that has not yet failed.

Mapping the ZavaShop workshop to the architecture

Layer 1 artifacts shipped in the repo:

.github/agents/zavashop-coding-agent.agent.md — the Coding Agent definition
.github/skills/agent-framework-{azure-ai,workflows,agui}-{py,csharp}/ — the six SKILLs
workshop/data/ — shared fixtures every artifact grounds in
Per-lab READMEs + eval_queries.jsonl — Layer 1 validation inputs

Layer 2 artifacts produced over the course of the workshop:

A single agent (Zara) — function tools + HostedMCPTool + Thread
A procurement agent (Pierre) — Toolbox + Agent Skills + approval policy
A customer-service agent (Aria) — Foundry Memory + Evaluation + Red-Team
A multi-agent fulfillment workflow (Diego) — WorkflowBuilder + HITL + Checkpoint
An AG-UI control tower for the CEO — covering all 7 AG-UI features

Same model across the stack — gpt-5.5 on Foundry + text-embedding-3-small. Change one env var, run the same artifact in the other language.

Three habits that separate strong agent engineers

Read the SKILL first. Make it ritual. The Coding Agent does it automatically; you should do it manually when reviewing the agent's output.
Treat tools as a public API. Names, signatures, docstrings, return shapes — they are how the model sees your system at runtime. Refactor them like any other API.
Measure before you tune. A prompt change without an eval delta is a vibe. With one, it's engineering.

Getting started in 60 seconds

git clone https://github.com/microsoft/Learn-Microsoft-Agent-Framework-with-Foundry-ZavaShop-Supply-Chain-Workshop cd Learn-Microsoft-Agent-Framework-with-Foundry-ZavaShop-Supply-Chain-Workshop # Foundry prereqs: gpt-5.5 + text-embedding-3-small deployed in your Foundry project az login --use-device-code # Python track python -m venv .venv && source .venv/bin/activate pip install agent-framework agent-framework-azure-ai agent-framework-ag-ui \ azure-identity python-dotenv fastapi "uvicorn[standard]" # .NET track dotnet --version # ≥ 10.0.100 # .env at repo root cat > .env <<EOF FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com/api/projects/<project-name> FOUNDRY_MODEL=gpt-5.5 AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-3-small AGUI_SERVER_URL=http://127.0.0.1:5100/ AG_UI_API_KEY=zava-control-tower-demo-key EOF # In VS Code → Copilot Chat → Agent Mode → pick zavashop-coding-agent # Then say: "I'm working on the inventory agent in Python — meet Mei."

The one mantra: "Read the SKILL first."

Closing thought

Modern agent development is not one job — it's two. The Coding Agent designs and builds; the Runtime Agent operates and delivers. Microsoft Agent Framework is the SDK that makes both layers feel like the same conceptual model. Microsoft Foundry is the platform both layers publish to and run on.

And the engine that turns a generic Copilot into a domain-aware engineer — that takes a sentence-long requirement and lands a runnable, validated, deployable artifact — is the SKILL. Write a good SKILL once, and every agent built afterwards inherits your team's taste, your fixtures, your patterns, your discipline.

The ZavaShop workshop is the smallest end-to-end example I can give you that actually exercises both layers, with six SKILLs ready to read. Walk it once, and the next time someone asks "how do we build agents in our org?", you won't be pointing at a tutorial — you'll be pointing at an architecture.

👉 Start with the workshop on GitHub

Power Pages & SPA: Deploying a Custom React SPA to Microsoft Power Pages

aakarshdhawan — Fri, 29 May 2026 07:00:00 GMT

Introduction

Modern business applications are no longer limited to traditional server-rendered pages or fixed portal layouts. Today’s users expect fast, responsive, highly interactive web experiences - the kind of experience developers commonly build using modern front-end frameworks such as React.

Microsoft Power Pages has evolved to support this shift. With Single Page Application support, developers can now build modern client-side applications and deploy them directly to Power Pages while still benefiting from the platform’s security, authentication, Dataverse integration, and governance capabilities.

In this blog, we’ll walk through how to build a simple React Single Page Application using Vite and deploy it to Microsoft Power Pages using the Power Platform CLI.

What We Will Build

In this walkthrough, we will:

Create a React + TypeScript application using Vite
Add simple client-side navigation
Build the app for production
Authenticate Power Platform CLI
Upload the compiled SPA to Power Pages
Activate and test the deployed site
Review key troubleshooting and governance considerations

Prerequisites

Before starting, ensure you have the following:

A Power Pages environment where you have admin privileges
Power Platform CLI installed and authenticated
A Power Pages site that supports SPA deployment
Node.js installed locally
A local React, Angular, or Vue project
Permission to allow JavaScript file uploads if blocked by the environment

Step 1: Create a React App Using Vite

Start by creating a new React + TypeScript app.

npm create vite@latest powerpages-spa -- --template react-ts cd powerpages-spa npm install npm run dev

This creates a lightweight React application using Vite. You can run it locally and validate that the app works before deploying it to Power Pages.

Step 2: Add Basic Pages and Navigation

Create a simple page structure.

mkdir src/pages

Create src/pages/Home.tsx.

export default function Home() { return ( <div> <h2>Welcome to Power Pages SPA</h2> <p>This React application is hosted on Microsoft Power Pages.</p> </div> ); }

Create src/pages/Products.tsx.

export default function Products() { return ( <div> <h2>Products</h2> <p>This is a sample product page rendered inside the SPA.</p> </div> ); }

Now update App.tsx.

import { HashRouter as Router, Routes, Route, NavLink } from "react-router-dom"; import Home from "./pages/Home"; import Products from "./pages/Products"; import "./App.css"; function App() { return ( <Router> <header> <h1>My Power Pages SPA</h1> <nav> <NavLink to="/">Home</NavLink> <NavLink to="/products">Products</NavLink> </nav> </header> <main> <Routes> <Route path="/" element={<Home />} /> <Route path="/products" element={<Products />} /> </Routes> </main> </Router> ); } export default App;

Power Pages handles routing on the server side. If you use BrowserRouter, your SPA will break on refresh or navigation, because Power Pages tries to resolve routes itself. To fix that, use HashRouter - it keeps navigation fully on the client side, avoiding any routing conflicts.

Install React Router if it is not already installed.

npm install react-router-dom

Step 3: Build the React App

Once the local app is ready, create the production build.

npm run build

For a Vite React app, this typically generates the compiled output inside the dist folder. This compiled folder is what we will upload to Power Pages.

Step 4: Authenticate Power Platform CLI

Authenticate to your Power Platform environment.

pac auth create --url https://yourorg.crm.dynamics.com

Replace the URL with your Dataverse environment URL. If you work with multiple environments, confirm the active environment before uploading.

pac auth list

Step 5: Upload the SPA to Power Pages

Use the following PAC CLI command to upload the React SPA.

pac pages upload-code-site ` --rootPath "./" ` --compiledPath "./dist" ` --siteName "My Power Pages SPA"

(Optional) You can also use a powerpages.config.json file in the root folder to define configuration such as site name, default landing page, and compiled path.

Example:

{ "siteName": "My Power Pages SPA", "defaultLandingPage": "index.html", "compiledPath": "./dist" }

Then upload using:

pac pages upload-code-site --rootPath "./"

Step 6: Activate the Site

After upload, the SPA site appears in Power Pages under inactive sites and must be activated to make it available to users.

To activate it:

Go to Power Pages.
Open Inactive sites.
Find your uploaded SPA site.
Select Reactivate.
Open the site URL and validate the deployment.

Step 7: Download and Re-upload an Existing SPA Site

If you need to download an existing SPA site for editing or backup, use:

pac pages download-code-site ` --path "./downloaded-site" ` --webSiteId "your-website-guid" ` --overwrite

Security and Dataverse Integration

A major advantage of deploying SPAs to Power Pages is that the app can still use Power Pages security and Dataverse integration patterns.

For Dataverse operations, developers can use Power Pages Web APIs to load content into the UI or create, update, and delete records, provided the required Web APIs are enabled and table permissions and web roles are configured properly.

const fetchAccounts = async () => { const response = await fetch("/_api/accounts"); const data = await response.json(); return data.value; };

Always configure table permissions and web roles carefully. The front end should never be treated as the security boundary. Power Pages Web API calls must rely on the platform’s permission model.

Troubleshooting: JavaScript Upload Restrictions

Some Dataverse environments block JavaScript file uploads by default. Microsoft Learn states that if you encounter an error such as “Import failed: The attachment is either not a valid type or is too large,” you may need to remove js from blocked attachments in the environment settings.

To check this:

Open Power Platform admin center.
Select the environment.
Go to Settings.
Expand Product.
Select Privacy + Security.
In Blocked Attachments, remove js.
Save the setting.
Re-run the upload command.

Key Differences from Traditional Power Pages Sites

SPA sites behave differently from traditional Power Pages implementations.

Microsoft Learn states that for SPA sites:

Server-side refresh returns the site root page and the client-side router renders sub-routes.
The pages workspace isn’t supported.
Styling workspace isn’t supported.
Liquid code and Liquid templates aren’t supported.
Adding out-of-the-box components such as lists and forms isn’t currently supported.
SEO support is limited because SPA sites use client-side rendering.

This means developers should treat Power Pages SPA development as a code-first model rather than a low-code page-authoring model.

Recommended Project Structure

A simple structure can look like this:

powerpages-spa │ ├── src │ ├── pages │ │ ├── Home.tsx │ │ └── Products.tsx │ ├── App.tsx │ └── main.tsx │ ├── dist │ ├── powerpages.config.json ├── package.json └── README.md

Conclusion

Power Pages SPA support opens up an exciting path for professional developers and fusion teams. You can now build modern React-based experiences and host them on Power Pages while continuing to use the platform’s authentication, Web API, Dataverse, and governance capabilities.

This approach is especially useful when your portal experience requires advanced UI customization, dynamic client-side behavior, or a component-driven front end that goes beyond traditional portal pages.

With React, Vite, and Power Platform CLI, the deployment flow becomes simple:

npm run build pac pages upload-code-site --rootPath "./" --compiledPath "./dist" --siteName "My Power Pages SPA"

Useful References

From Requirement to Production Code, How Engineering Squad Automates the Full Dev Lifecycle

paggarwal — Fri, 29 May 2026 07:00:00 GMT

I started wondering: what if instead of one AI assistant generating code snippets, you had an entire squad of specialized AI agents. Each owning a single stage of the delivery pipeline, they could collaborate, self-correct, and produce a complete, traceable output from a plain-text requirement?

That's Engineering Squad: an open-source, multi-agent framework built with LangGraph, Azure OpenAI, and Foundry Local. Nine agents. One pipeline. Zero manual handoffs.

You give it a requirement. It gives you back:

- User stories with acceptance criteria

- Technical design (API contracts, data models, architecture)

- Full implementation code (written into real files, not markdown)

- Unit tests and Playwright E2E tests

- Automated code review with a self-correcting feedback loop

When the Code Reviewer finds a bug, it doesn't just flag it, it routes the work back to the exact agent that needs to fix it.

When the Spec Agent hits ambiguity, it stops and asks you rather than guessing. The loop runs up to 5 iterations, and every run is versioned under a unique Run ID for full traceability.

It runs on Azure OpenAI for heavy reasoning, Foundry Local for lightweight tasks or entirely offline with --local-only mode. No cloud required.

How It Works

The squad is a directed graph of 9 specialized agents. Each agent has a single responsibility and a tuned system prompt. The orchestration is handled by LangGraph's StateGraph, which routes work through the pipeline and handles feedback loops.

The Agents

Agent	Model	Responsibility
Product Owner	Azure OpenAI gpt-4.1	Reads requirements, classifies impact scope
Story Agent	Foundry Local (qwen2.5-7b)	Converts requirements → structured user stories
Spec Agent	Azure OpenAI o3	Resolves ambiguity — asks the user interactively
Technical Design	Azure OpenAI gpt-4.1	Architecture, API contracts, data models, error handling
Developer	Azure OpenAI gpt-4.1	Writes code directly into the codebase
Unit Tester	Azure OpenAI gpt-4.1	Writes unit tests and evaluates them against implementation
Test Writer	Foundry Local (qwen2.5-7b)	Writes Playwright E2E tests using Page Object Model
Tester	Azure OpenAI o3	Final evaluation of code against all specs and tests
Code Reviewer	Azure OpenAI o3	Reviews everything, decides: approve or route back

The Self-Correcting Loop

This is where it gets interesting. The Code Reviewer doesn't just say "approved" or "rejected" — it makes a routing decision using structured output:

class ReviewDecision(BaseModel): decision: Literal[ "approved", # Ship it "requirement_confusion", # → Spec Agent (clarify ambiguity) "clarity_missing", # → Technical Design (refine design) "code_missing", # → Developer (fix implementation) "bug_found", # → Developer (fix bugs) "test_case_missing", # → Test Writer (add coverage) ] feedback: str # Actionable feedback for the target agent

LangGraph's conditional edges route the workflow back to the exact agent that needs to act. The loop runs up to 5 iterations with a hard stop to prevent infinite cycles.

workflow.add_conditional_edges( "code_reviewer", route_review, { END: END, "spec_agent": "spec_agent", "technical_design": "technical_design", "developer": "developer", "test_writer": "test_writer", }, )

Key Design Decisions

1. Impact Classification — Don't Run What You Don't Need

Not every change needs the full pipeline. The squad classifies scope first:

Scope	What Runs
config	Impact Analysis → Developer → Unit Tester → Reviewer
bugfix	Impact Analysis → Developer → Unit Tester → Tester → Reviewer
enhancement	Stories → Design (if needed) → Developer → All Tests → Reviewer
feature	Stories → Design → Developer → All Tests → Reviewer
refactor	Impact Analysis → Developer → Unit Tester → Reviewer

A config change doesn't need user stories. A bugfix doesn't need a full architectural design. This keeps runs fast and focused.

2. Code Goes Into Real Files, Not Markdown

This was a deliberate choice. The Developer Agent edits actual source files in your project — it doesn't dump code into a markdown artifact. The code_changes.md artifact is a change log that records what was modified and why, for traceability.

3. Existing Projects vs. Greenfield

Set PROJECT_TYPE: existing in requirements_input.txt, point it at your repos, and the squad will:

Scan your codebase for patterns, conventions, and architecture
Make targeted changes only — no rewriting from scratch
Preserve your existing coding style, error handling, and naming conventions

4. Two LLM Tiers — Cloud + Local

The framework uses a hybrid model strategy:

Azure OpenAI (gpt-4.1, o3) for complex reasoning: code generation, technical design, code review
Foundry Local (qwen2.5-7b, phi-3.5-mini) for lightweight tasks: user stories, test writing

This keeps costs down while maintaining quality where it matters. And with --local-only mode, you can run the entire squad on Foundry Local with zero cloud dependencies.

Running It Locally with Foundry Local

One of my favorite features: the entire squad can run 100% locally using Foundry Local. No Azure subscription, no API keys, no internet required.

Setup

# Install Foundry Local CLI (one-time) winget install Microsoft.FoundryLocal # Install Python dependencies pip install foundry-local-sdk openai langchain-openai langgraph python-dotenv # Run in local-only mode python main.py --local-only

When --local-only is set, every agent that would normally call Azure OpenAI gets redirected to Foundry Local:

def get_azure_llm(deployment: str, temperature: float = 0.1): # Local-only mode: redirect to Foundry Local if os.getenv("SQUAD_LOCAL_ONLY", "").lower() in ("true", "1", "yes"): from models.local_llm import get_local_llm return get_local_llm(temperature=temperature) # Otherwise: use Azure OpenAI with DefaultAzureCredential ...

The foundry-local-sdk (v1.1.0+) handles everything — initializing the runtime, downloading models, and loading them:

from foundry_local_sdk import FoundryLocalManager, Configuration # Initialize once (singleton) config = Configuration(app_name="my-app") manager = FoundryLocalManager(config) # Start OpenAI-compatible web service manager.start_web_service() print(manager.urls[0]) # SDK auto-discovers the endpoint # Download & load a model model = manager.catalog.get_model("qwen2.5-7b") model.download() model.load() # Chat directly — no web service needed chat = model.get_chat_client() response = chat.complete_chat([{"role": "user", "content": "Hello!"}])

Jupyter Notebook

The repo includes a Jupyter notebook (foundry_local.ipynb) that walks you through:

Installing Foundry Local
Loading a model
Sending chat completions (streaming and non-streaming)
Running the full Engineering Squad in local-only mode

Traceability — Every Run Is Versioned

Every squad execution gets a unique Run ID and produces a structured artifact set:

output/ runs/ 20260524_a3f9b1/ run_metadata.json ← run ID, timestamp, requirement hash, decision impact_classification.md user_stories.md technical_design.md code_changes.md ← change log (code is in real files) unit_test_results.md tests.md test_results.md review_feedback.md latest/ ← symlink to most recent approved run

The run_metadata.json is structured for future Azure DevOps integration — auto-creating work items, tasks, and test cases from squad output.

Two Ways to Run

Mode	Best For
GitHub Copilot Agent Mode	Existing codebases — Copilot has full workspace context via #codebase
Python CLI (python main.py)	New projects, CI pipelines, fully automated runs

Running with GitHub Copilot Agent Mode

This is the recommended way to run the squad on existing projects. Copilot has full access to your workspace — it can read files, write code, and run terminal commands — so it naturally understands your architecture, patterns, and conventions.

Prerequisites

VS Code with the GitHub Copilot and GitHub Copilot Chat extensions installed
A Copilot subscription that supports Agent Mode (Copilot Pro, Business, or Enterprise)

Setup

Clone the repo and open it in VS Code:

git clone https://github.com/prasunagga/engineeringSquad.git
code engineeringSquad

Switch to Agent Mode — In the Copilot Chat panel, click the mode dropdown (top of the chat input) and select "Agent". This is required — Ask and Edit modes don't have tool access.
Enable tools — Click the 🔧 tools icon (or gear/settings icon) at the bottom of the chat input area. Make sure the following tools are enabled:
- File operations (read, create, edit files)
- Terminal (run commands)
- Code search / workspace context
Without these enabled, the squad can't read your codebase or write code into files.
Edit your requirement — Open requirements_input.txt and write your requirement:PROJECT_TYPE: existing FRONTEND_PATH: plant-catalog BACKEND_PATH: Build a cart page where users can add plants, adjust quantities, and see totals.

Running the Squad

In Copilot Chat (Agent Mode), type:

/run-squad

This triggers the .github/prompts/run-squad.prompt.md file — a prompt file with mode: agent in its YAML frontmatter that orchestrates the full workflow:

--- mode: agent description: Run the full Engineering Squad workflow tools: - read_file - create_file - replace_in_file - insert_text - delete_file_range ---

Copilot will then execute the full pipeline: read requirements → classify impact → generate stories → design → write code → write tests → run tests → code review → approve or loop back.

How It Differs from Python CLI

	Copilot Agent Mode	Python CLI
Context	Full workspace awareness via #codebase	Reads files from paths in requirements_input.txt
Human-in-loop	Spec Agent asks you directly in chat	Spec Agent prints questions to stdout
Code editing	Uses VS Code's file editing tools	Writes files via Python open()
Test execution	Runs npm test / playwright test in VS Code terminal	Runs via subprocess
Model	Uses whichever model is selected in Copilot	Uses Azure OpenAI / Foundry Local

Individual Agent Prompts

The .github/prompts/ directory also contains standalone prompt files for running individual agents:

Prompt	Purpose
run-squad.prompt.md	Full orchestrated pipeline
developer.prompt.md	Developer agent only
code-reviewer.prompt.md	Code review only
story-agent.prompt.md	Generate user stories only
technical-design.prompt.md	Technical design only
test-writer.prompt.md	Write E2E tests only

Extending the Framework

The squad is designed to be modular. Here are the most common extension points:

Add a New Agent

Every agent follows the same pattern — a function that takes SquadState, calls an LLM, and returns updated fields:

# agents/my_agent.py from langchain_core.prompts import ChatPromptTemplate from graph.state import SquadState from models.azure_llm import get_azure_llm, DEPLOYMENT_DEVELOPER PROMPT = ChatPromptTemplate.from_messages([ ("system", "You are a security review specialist."), ("human", "Review this code for vulnerabilities:\n{code}"), ]) def my_agent_node(state: SquadState) -> dict: llm = get_azure_llm(deployment=DEPLOYMENT_DEVELOPER) result = (PROMPT | llm).invoke({"code": state["code"]}) return {"security_review": result.content}

Then wire it in:

Add state fields in graph/state.py
Register the node and edges in graph/workflow.py
Add artifact output in main.py

Swap the LLM for Any Agent

Each agent calls get_azure_llm(deployment=...) or get_local_llm(). You can:

Change the model — edit .env (e.g., AZURE_DEPLOYMENT_DEVELOPER=gpt-5.4)
Go fully local — python main.py --local-only
Use a different provider — replace get_azure_llm() with any LangChain-compatible LLM (Anthropic, Ollama, Groq, etc.)

Customize Agent Prompts

Each agent's system prompt is defined as a ChatPromptTemplate at the top of its file in agents/. Edit the prompt directly — no configuration layer to navigate.

Change the Review Loop

The routing logic lives in graph/workflow.py → route_review(). Add new decision strings, change the routing map, or adjust MAX_ITERATIONS (default: 5).

VS Code Copilot Agent Mode

The .github/prompts/ directory contains prompt files for running individual agents in VS Code Copilot Agent Mode. Edit these to customize agent behavior when running through Copilot.

What I Learned Building This

Structured output is essential for routing. Without Pydantic models for review decisions, the conditional edge routing would be fragile and string-matching-dependent.
Impact classification saves significant time. Running 9 agents for a one-line config change is wasteful. Classifying scope first makes the system practical.
The self-correcting loop works — but needs a hard stop. Left unchecked, agents can ping-pong feedback indefinitely. The 5-iteration cap is a pragmatic safety net.
Hybrid local + cloud models are the right balance. Not every task needs GPT-4.1. User story generation and test writing work well on smaller local models, cutting costs without sacrificing quality.
"Ask, don't guess" is the single most important principle. When the Spec Agent encounters ambiguous requirements, it stops and asks the user rather than hallucinating assumptions. This one rule prevents the most costly category of errors.

Try It Yourself

The framework is open source and designed to be extensible:

git clone https://github.com/prasunagga/engineeringSquad.git cd engineeringSquad pip install -r requirements.txt # Edit your requirement notepad requirements_input.txt # Run (local-only, no Azure needed) python main.py --local-only

Requirements:

Python 3.10+
Windows, macOS, or Linux
For local-only: Foundry Local (winget install Microsoft.FoundryLocal)
For cloud mode: Azure OpenAI endpoint + az login

What's Next

Azure DevOps MCP integration — Auto-sync stories, tasks, and test cases to ADO boards
CI/CD trigger — Auto-run the squad on PR creation or work item assignment
Multi-repo support — Frontend, backend, and infra in separate repositories
Cost estimation — Estimate effort and cloud costs from the technical design

Building Agentic Systems on Azure: Microsoft Foundry Agents SDK vs Microsoft Agent Framework

ChaitanyaThalloory — Thu, 28 May 2026 07:00:00 GMT

In my recent experience as a Senior Consultant at Microsoft, I’ve been actively involved in designing and delivering AI-driven solutions, with a strong focus on building intelligent agents using modern frameworks.

Along the way, I've built agents using both Microsoft Foundry Agents SDK (hereafter "Agents SDK") and Microsoft Agent Framework (MAF)

Both approaches are powerful and capable. However, once you move beyond simple proofs of concept, the developer experience and architectural patterns start to differ significantly.

This article provides a practical comparison based on real implementation experience and aims to help developers choose the right approach.

Approach 1: Agents SDK

Agents SDK provides a straightforward way to create agents with integrated tools and models.

Example: Creating an Agent

from azure.ai.projects import AIProjectClient

from azure.ai.agents.models import AzureAISearchTool, AzureAISearchQueryType

from azure.identity import DefaultAzureCredential

client = AIProjectClient(credential=DefaultAzureCredential(), endpoint=os.getenv("AZURE_AI_PROJECT_ENDPOINT"))

# Configure tools

ai_search = AzureAISearchTool(

index_connection_id=conn_id,

index_name="my-index",

query_type=AzureAISearchQueryType.SEMANTIC,

)

# Create agent (persisted in Foundry portal)

agent = client.agents.create_agent(

model=os.getenv("AZURE_AI_AGENT_DEPLOYMENT_NAME"),

name="MyAgent",

instructions="You are a helpful assistant.",

tool_resources=ai_search.resources,

tools=ai_search.definitions,

)

# Run conversation

thread = client.agents.threads.create()

client.agents.messages.create(thread_id=thread.id, role="user", content="Hello")

run = client.agents.runs.create(thread_id=thread.id, agent_id=agent.id)

What this approach provides

Native integration with Azure AI services (OpenAI, AI Search, MCP)
Managed execution environment
Simple and quick agent setup

Conceptually, this approach can be summarized as:

Model + Tools + Execution

Strengths

✅ Rapid development and onboarding
✅ Strong integration within the Azure ecosystem
✅ Well-suited for single-agent or tool-driven use cases
✅ Minimal infrastructure overhead

Challenges observed in practice

As the complexity of scenarios increases, certain limitations become more visible:

Multi-agent workflows require custom orchestration logic
Agent handoffs must be implemented manually
Context sharing across agents requires additional design effort

While this approach offers flexibility, it shifts orchestration complexity to the developer.

Approach 2: Microsoft Agent Framework (MAF)

Microsoft Agent Framework introduces a higher-level abstraction, focused on agent orchestration and system design.

Creating an Agent

from agent_framework import Agent, WorkflowBuilder, Message

from agent_framework.foundry import FoundryChatClient

from azure.identity import DefaultAzureCredential

client = FoundryChatClient(

project_endpoint=os.getenv("FOUNDRY_PROJECT_ENDPOINT"),

model=os.getenv("FOUNDRY_MODEL_DEPLOYMENT_NAME"),

credential=DefaultAzureCredential(),

)

# Create agents (in-process only, not persisted in portal)

researcher = Agent(client, name="ResearcherAgent", instructions="Research topics thoroughly.")

writer = Agent(client, name="WriterAgent", instructions="Write concise summaries.")

# Build and run multi-agent workflow

workflow = WorkflowBuilder(start_executor=researcher).add_edge(researcher, writer).build()

async for event in workflow.run(Message("user", "Summarize migration best practices"), stream=True):

print(event.content)

What this approach provides

Built-in orchestration capabilities
Native support for multi-agent workflows
Structured agent lifecycle management
Context and memory handling

Conceptually, this can be viewed as:

Agents + Orchestration + System Design

Observations from implementation

When implementing similar use cases using MAF:

Agent responsibilities became clearly defined
Routing and delegation patterns were significantly simplified
Overall system architecture became easier to maintain and scale

This approach encourages thinking in terms of agent ecosystems rather than isolated agents.

Architecture Comparison

Agents SDK

Microsoft Agent Framework (MAF)

Choosing the Right Approach

Use Agents SDK when:

You need rapid development for a single-agent use case
The workflow is relatively straightforward
You prefer flexibility and lower-level control

Use Microsoft Agent Framework when:

You are designing multi-agent systems
Your solution requires routing, delegation, or handoffs
Long-term scalability and maintainability are essential

Pros and Cons Summary

Agents SDK

Pros

Easy to get started
Strong Azure integration
Flexible design

Cons

Manual orchestration required
Limited native multi-agent support
Complexity increases as scenarios grow

Microsoft Agent Framework (MAF)

Pros

Built-in orchestration
Native multi-agent support
Scalable and structured architecture

Cons

Learning curve for new developers
More opinionated framework design
Reduced low-level control compared to SDK-based approach

References and Repositories

🔗 Microsoft Agent Framework (MAF)

📘 Documentation

🔗 Azure AI Projects / Agents SDK

📘 Documentation

Conclusion

Azure AI Projects and Microsoft Agent Framework both play important roles in the modern agent development landscape.

Agents SDK enables quick and flexible agent development
Microsoft Agent Framework enables structured, scalable agent systems

In practice, the choice depends on whether you are building a single agent feature or a multi-agent system.

Final Thought

Agents SDK helps you get started quickly.
Microsoft Agent Framework helps you scale with confidence

In a follow-up blog, I’ll dive into how the M365 Agents SDK compares with Microsoft Agent Framework, especially in the context of enterprise productivity and Copilot experiences.

Multi-Tenant Architecture: Real Challenges and an Azure Design Walkthrough

pranav_pratik — Wed, 27 May 2026 07:00:00 GMT

Azure Multi-Tenant Architecture (B2C Scenario)

Let’s start with a reference design commonly used in Azure-based systems.

A pretty standard setup looks something like this:

Microsoft Entra External ID (Azure AD B2C) for authentication
Azure API Management as the entry layer
App Service or Functions for the compute layer
Cosmos DB or SQL for storage
Redis for caching
Service Bus for async processing
Application Insights for monitoring

If you’ve worked on Azure systems, nothing here is surprising.
On paper, this architecture is clean, scalable, and “multi-tenant ready”.

But once traffic starts flowing and tenants behave differently, things start breaking in subtle ways.

1. Tenant Context Propagation Across Services

A request doesn’t stay in one place. It moves across:
- API layer
- queues/topics
- background workers

What I’ve seen happen multiple times:

tenant ID is present in the API, but missing in async flows
background jobs process data without knowing which tenant it belongs to
logs become useless because you can’t tie actions back to a tenant

The fix is simple in theory, but often missed in implementation:

Every message should carry tenant context. No exceptions.

If you rely on “it will be available somewhere”, it won’t be, especially in distributed systems.

Ensure tenant context is explicitly carried everywhere:

public class TenantMessage { public string TenantId { get; set; } public string Payload { get; set; } }

Every message, event, and async operation should include tenant scope.

2. Data Isolation in Shared Databases

Most teams start with a shared database model with tenant-based partitioning.
It works well initially.

Problems start creeping in later:

- someone forgets to add a tenant filter in a query
- a query suddenly scans across partitions
- one large tenant starts slowing down others

A simple query like this becomes critical:

var query = container.GetItemQueryIterator<Order>( new QueryDefinition("SELECT * FROM c WHERE c.tenantId = @tenantId") .WithParameter("@tenantId", tenantId) );

The tricky part is not writing it once, it’s making sure it’s applied everywhere, every time.

3. Authorization Beyond Tenant Boundaries

At the beginning, access control is simple:

“Users can access data from their own tenant.”

But then requirements grow:

admin access
cross-tenant visibility
reporting across firms or regions

And this is where things usually get messy.

Different services start implementing their own logic, and over time you end up with inconsistent behavior.

A simple check:

public bool CanAccess(string userTenant, string resourceTenant, bool isGlobalAdmin) { if (isGlobalAdmin) return true; return userTenant == resourceTenant; }

becomes much harder to manage when duplicated across multiple services.

One thing that helps a lot here is centralizing authorization logic early.

4. Caching as a Hidden Risk

Caching is usually added later for performance.

And that’s exactly why it becomes risky.

I’ve seen scenarios where:

cached data from one tenant is returned to another
because the cache key didn’t include tenant information

Fixing it is straightforward:

public string BuildCacheKey(string tenantId, string key) { return $"{tenantId}:{key}"; }

Cache keys must always include tenant boundaries

5. Resource Contention (Noisy Neighbor Problem)

All tenants share resources:

compute
database throughput
messaging

What happens in practice:

one high-load tenant impacts others
latency becomes unpredictable
system behavior differs per tenant

You start adding controls like:

if (RequestsPerTenant[tenantId] > 100) { return StatusCode(429); }

And gradually move towards:

throttling
workload isolation
prioritization

This is less of a design problem and more of an operational reality.

6. Observability in Multi-Tenant Systems

Logging works great, until you scale.

Then suddenly:

logs from all tenants are mixed
debugging becomes slow
it’s hard to answer basic questions like “which tenant failed?”

A small change makes a huge difference:

_logger.LogInformation( "Tenant={TenantId} Action=ProcessOrder OrderId={OrderId}", tenantId, orderId );

It sounds obvious, but it’s often inconsistent across services.

7. Backup and Restore Considerations

Taking backups is easy.

Restoring a single tenant isn’t.

In most shared database setups:

restore is done at database level
which affects all tenants

So if one tenant has a problem, recovery is not straightforward.

This is one of those areas where decisions made early in design matter a lot later.

Final Thoughts

Designing a multi-tenant system is not just about choosing Azure services.

The real challenges come from:

how tenant context flows
how isolation is enforced
how systems behave under uneven load

Most issues don’t show up on day one.
They appear gradually as tenants grow, scale, and behave differently.

References and Further Reading

If you want to explore these concepts in more depth, here are some useful official resources:

Building an On-Device Voice Assistant with Microsoft Foundry Local

Lee_Stott — Tue, 26 May 2026 07:00:00 GMT

Why on-device voice still matters

Most "voice AI" tutorials assume your audio leaves the machine. You ship a WAV to Whisper-API, your transcript to GPT-4, and a synthesized response back over the wire. That works — but it also means three round trips, three per-token bills, and three places your user's voice gets logged.

The new wave of small, hardware-optimised models changes the trade-off. NVIDIA's Nemotron Speech Streaming En 0.6B is a 600M-parameter streaming ASR model published into the Microsoft Foundry Local catalog. Paired with a small chat model like qwen2.5-0.5b or phi-4-mini, you can run the entire capture → transcribe → reason → respond loop in-process on a developer laptop, with no API keys and no network egress.

This post walks through how the fl-nemotron sample does it, the SDK pitfalls we hit on the way, and the design decisions that made the pipeline reliable.

What we're building

A browser-hosted assistant served by FastAPI at http://127.0.0.1:8000. The page captures microphone audio, posts it to /api/transcribe, then streams the chat reply back over Server-Sent Events from /api/chat. All inference runs locally through two Foundry Local models loaded into the same process.

The shape of the pipeline:

Microphone (browser MediaRecorder)
   │  WebM/Opus blob
   ▼
Client-side WAV encoder (16 kHz, mono, PCM-16)
   │  multipart/form-data
   ▼
FastAPI /api/transcribe
   │
   ▼
Nemotron Speech Streaming En 0.6B  (Foundry Local audio client)
   │  transcript text
   ▼
Chat LLM e.g. qwen2.5-0.5b         (Foundry Local chat client)
   │  streamed tokens
   ▼
FastAPI /api/chat → SSE → browser bubble

The version that bit us: `foundry-local-sdk >= 1.1.0`

Before any code, the single most important fact about this project:

The Nemotron Speech Streaming model only appears in the Foundry Local 1.1.x catalog. Older SDKs (0.5.x / 0.6.x) cannot resolve the alias nemotron-speech-streaming-en-0.6b and fail with model not found.

The module name also changed in 1.1.0 — it is now foundry_local_sdk (with the underscore-sdk suffix), not foundry_local. The pip wheel for foundry-local-core is bundled, so there is no separate MSI / winget install to worry about.

Pin it explicitly:

pip install --upgrade "foundry-local-sdk>=1.1.0,<2"

And verify before anything else:

python -c "import importlib.metadata as m; print('sdk', m.version('foundry-local-sdk'))"
# expect: sdk 1.1.0

Loading both models from one manager

The 1.1.x SDK exposes a single FoundryLocalManager that owns the runtime. Each loaded model gives you back a per-model OpenAI-compatible client — get_chat_client() for text models and get_audio_client() for ASR. There is no need to bring your own openai Python package; the SDK ships its own thin client.

The wrapper used in the repo (src/foundry_client.py) does this:

from foundry_local_sdk import Configuration, FoundryLocalManager

FoundryLocalManager.initialize(Configuration(app_name="fl-nemotron"))
manager = FoundryLocalManager.instance

chat_model = manager.load_model("qwen2.5-0.5b")
stt_model  = manager.load_model("nemotron-speech-streaming-en-0.6b")

chat_client  = chat_model.get_chat_client()
audio_client = stt_model.get_audio_client()

Both models are downloaded on first use into the Foundry Local cache and stay resident for the lifetime of the process. On a laptop with 16 GB RAM, the combined working set sits comfortably under 4 GB.

The transcription surprise

The first naive approach was the obvious one:

with open(wav_path, "rb") as f:
    result = audio_client.transcribe(file=f, model="nemotron-speech-streaming-en-0.6b")

That call fails on Nemotron. The bundled ONNX Runtime GenAI in foundry-local-core does not register the nemotron_speech multi-modal model type that the standard AudioClient.transcribe() path tries to instantiate. The error surfaces as a cryptic model-type registration failure deep inside the native runtime.

The fix is to use the streaming session API instead — a different native entry point (core_interop.start_audio_stream) that the streaming model does support. The repo isolates this in src/_nemotron_live.py:

def transcribe_wav_live(audio_client, wav_path, *, language="en"):
    with wave.open(str(wav_path), "rb") as w:
        sample_rate  = w.getframerate()
        channels     = w.getnchannels()
        sample_width = w.getsampwidth()
        pcm          = w.readframes(w.getnframes())

    session = audio_client.create_live_transcription_session()
    session.settings.sample_rate     = sample_rate
    session.settings.channels        = channels
    session.settings.bits_per_sample = sample_width * 8
    session.settings.language        = language
    session.start()

    # Feed PCM in ~100 ms chunks from a worker thread, then stop.
    bytes_per_sec = sample_rate * channels * sample_width
    chunk_bytes   = max(bytes_per_sec // 10, 1024)

    def _pusher():
        try:
            for offset in range(0, len(pcm), chunk_bytes):
                session.append(pcm[offset:offset + chunk_bytes])
        finally:
            session.stop()

    threading.Thread(target=_pusher, daemon=True).start()

    parts = []
    for resp in session.get_stream():
        for cp in getattr(resp, "content", []) or []:
            text = getattr(cp, "text", "") or getattr(cp, "transcript", "") or ""
            if text:
                parts.append(text)
    return " ".join(p.strip() for p in parts if p.strip()).strip()

Two things to notice:

Push from a thread, read from the main coroutine. session.append() is a blocking write into the native stream and session.get_stream() is a blocking generator. Run one in a worker thread so the other can drain in parallel — otherwise you deadlock the session.
Chunk to ~100 ms. Smaller chunks (e.g. 10 ms) spend more time crossing the FFI boundary than transcribing; larger chunks (e.g. 1 s) hold back partial results and hurt perceived latency.
Always session.stop(). Without it the generator never terminates and the request hangs.

The other transcription surprise: browsers don't send WAV

Inside the browser, MediaRecorder defaults to audio/webm; codecs=opus. That's great for size but bad for our STT model, which expects a 16-bit mono PCM WAV at a known sample rate. Decoding WebM/Opus server-side would require ffmpeg as a runtime dependency — which is exactly the kind of friction this project exists to remove.

The cleaner solution is to encode WAV on the client. AudioContext.decodeAudioData already understands WebM/Opus, so the page can decode the recording, resample to 16 kHz, mix to mono, and emit a PCM-16 WAV blob in 30 lines of JavaScript:

// Inside src/static/index.html
async function webmToWav(blob) {
  const ctx = new (window.AudioContext || window.webkitAudioContext)({ sampleRate: 16000 });
  const buf = await ctx.decodeAudioData(await blob.arrayBuffer());
  // Mix to mono
  const ch  = buf.numberOfChannels;
  const mono = new Float32Array(buf.length);
  for (let c = 0; c < ch; c++) {
    const data = buf.getChannelData(c);
    for (let i = 0; i < data.length; i++) mono[i] += data[i] / ch;
  }
  return encodeWav(mono, 16000);
}

function encodeWav(samples, sampleRate) {
  const buffer = new ArrayBuffer(44 + samples.length * 2);
  const view   = new DataView(buffer);
  // RIFF header
  writeStr(view, 0, "RIFF");
  view.setUint32(4, 36 + samples.length * 2, true);
  writeStr(view, 8, "WAVE");
  // fmt chunk
  writeStr(view, 12, "fmt ");
  view.setUint32(16, 16, true);              // PCM chunk size
  view.setUint16(20, 1, true);               // PCM format
  view.setUint16(22, 1, true);               // mono
  view.setUint32(24, sampleRate, true);
  view.setUint32(28, sampleRate * 2, true);  // byte rate
  view.setUint16(32, 2, true);               // block align
  view.setUint16(34, 16, true);              // bits per sample
  // data chunk
  writeStr(view, 36, "data");
  view.setUint32(40, samples.length * 2, true);
  // PCM-16 samples
  let o = 44;
  for (let i = 0; i < samples.length; i++, o += 2) {
    const s = Math.max(-1, Math.min(1, samples[i]));
    view.setInt16(o, s < 0 ? s * 0x8000 : s * 0x7FFF, true);
  }
  return new Blob([view], { type: "audio/wav" });
}

Now the server's /api/transcribe endpoint just writes the bytes to a temp file and hands them to transcribe_wav_live() — no audio decoding libraries on the Python side.

Wiring it into FastAPI

The server (src/app.py) is deliberately small. The notable detail is that the same process holds both Foundry Local model handles for its entire lifetime, so there is no warm-up cost per request:

@app.post("/api/transcribe")
async def transcribe(audio: UploadFile = File(...)):
    data = await audio.read()
    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
        f.write(data); path = f.name
    text = _ai_client.transcribe(path)
    return {"text": text}


@app.post("/api/chat")
async def chat(req: ChatRequest):
    if req.stream:
        return StreamingResponse(
            _sse(_ai_client.stream_completion(req.messages)),
            media_type="text/event-stream",
        )
    return {"text": _ai_client.chat_completion(req.messages)}

Streaming uses Server-Sent Events because they are trivially supported in both fetch() and the FastAPI runtime, and they don't require a WebSocket upgrade through any proxy a developer might have in front of localhost.

What it looks like

The repo includes screenshots of the running UI: a welcome screen with both models loaded, a streamed haiku reply, an inline code block with copy-to-clipboard, and the recording state for the microphone.

Performance, honestly

This is a small-model, CPU-friendly stack. On an Arm64 Surface running the x64 SDK under emulation:

First model load (cold cache): tens of seconds — downloads ~600 MB for Nemotron and ~400 MB for qwen2.5-0.5b.
Subsequent loads (warm cache): a few seconds per model.
End-to-end transcription of a 5-second utterance: well under a second after warm-up.
First chat token from qwen2.5-0.5b: typically 200–500 ms; full short reply within 1–2 s.

On x64 silicon with a recent CPU the numbers improve substantially, and the SDK will pick the best execution provider it finds (CPU / DirectML / CUDA) for each model.

Trade-offs to know about

Model quality. qwen2.5-0.5b is a 500M-parameter model. It is fast and small enough to ship on a laptop, but it is not GPT-4. Swap in phi-4-mini or mistral-nemo-12b-instruct if you have the RAM and want better reasoning — the wrapper accepts any chat alias in the Foundry Local catalog.
STT is English-only here. The current Nemotron streaming model in the catalog is ...-en-0.6b. Multilingual variants are likely to follow.
Browser microphone needs a real browser. Headless / automated browsers (Playwright, Puppeteer) deny getUserMedia by default. Open the page in Edge / Chrome / Firefox to grant the permission and capture audio for real.
No agent framework yet. This sample is deliberately a single-turn loop over a chat client — there is no tool calling, planning, or multi-agent orchestration. Adding the Microsoft Agent Framework on top would be a natural next step for richer behaviour.

Responsible AI considerations

Running locally removes the cloud-egress class of privacy concerns, but it does not remove responsibility:

Disclose recording. The browser prompts for mic permission; your UI should make it obvious when capture is active. The sample shows a red ⏹ button and a "Recording…" banner for that reason.
Don't log raw audio. The sample writes audio to a per-request NamedTemporaryFile and deletes it after transcription. Treat the WAV as sensitive data even when it never leaves the device.
Small models hallucinate. A 0.5B chat model is great for snappy local replies, but unsuitable for high-stakes answers. Pair it with retrieval, ground it on your own data, or escalate to a larger model when accuracy matters.

Try it

Clone github.com/leestott/fl-nemotron.
./setup.ps1 (or ./setup.sh) to create a virtualenv and install the pinned SDK.
python scripts/prefetch.py nemotron-speech-streaming-en-0.6b qwen2.5-0.5b to download both models.
.venv\Scripts\uvicorn.exe app:app --app-dir src --port 8000
Open http://127.0.0.1:8000 in a real browser and click the 🎤 button.

Where to go next

Foundry Local documentation — official docs for the runtime, catalog, and SDK.
microsoft/Foundry-Local — upstream samples and issue tracker.
NVIDIA Nemotron model family — background on the speech and language models being published into the catalog.
leestott/fl-nemotron — the full source for this post.

Key takeaways

Pin foundry-local-sdk >= 1.1.0. Earlier SDKs cannot see the Nemotron Speech Streaming model.
Use the LiveAudioTranscriptionSession API for Nemotron, not AudioClient.transcribe().
Encode WAV in the browser. It eliminates a heavy server-side ffmpeg dependency for a few lines of JS.
Push audio chunks on a worker thread and drain the response generator on the main one to avoid deadlocks.
A small Foundry Local chat model plus Nemotron STT gives you a credible local voice loop in a single Python process — no cloud, no keys, no data egress.

Building an End-to-End Azure RAG Strategy Agent with MS Foundry

SHAILESHDEVADIGA — Mon, 25 May 2026 07:00:00 GMT

High-Level Architecture

This architecture represents an end-to-end Retrieval-Augmented Generation (RAG) pipeline where raw documents are ingested from Azure Blob Storage, processed using Document Intelligence, transformed into embeddings via Azure OpenAI, and indexed in Azure AI Search for hybrid retrieval. A Foundry/MAF-based agent orchestrates query processing by combining user input with relevant search results and generates contextual responses, which are exposed through a FastAPI or CLI interface.

Azure-based RAG Pipeline with Agent-Orchestration

This solution is composed of two main layers:

1. Data Ingestion Layer (RAG Pipeline)

This layer transforms raw enterprise documents into searchable knowledge.

Flow:

Raw documents stored in Azure Blob Storage
- Supported formats: PDF, DOCX, PPTX, images, etc.
Document Intelligence extraction
- Extracts:
  - Text
  - Tables
  - Key-value pairs
  - Structure
- Writes output as structured JSON back to Blob (processed/)

Chunking + Embedding
- Documents are split into chunks
- Each chunk is embedded using Azure OpenAI (text-embedding-*)

Indexing into Azure AI Search
- Creates a hybrid index:
  - Keyword search
  - Semantic ranking
  - Vector search
- Enables flexible retrieval strategies

2. Query Layer (Strategy Agents)

This layer enables intelligent query answering.

Flow:

User sends a query via:
- FastAPI endpoint
- CLI interface

Query is handled by:
- Microsoft Agent Framework (MAF) agent
- Running on Azure AI Foundry

Agent:
- Queries Azure AI Search
- Retrieves top relevant chunks
- Injects them into LLM prompt

LLM generates grounded response
- This follows the standard RAG pattern:
  - Retrieval → Augmentation → Generation

End-to-End Flow

Key Azure Services Used

Service	Purpose
Azure Blob Storage	Raw + processed document storage
Azure AI Document Intelligence	Extract structured content
Azure OpenAI	Embeddings + LLM generation
Azure AI Search	Hybrid retrieval engine
Azure AI Foundry	Agent orchestration
Microsoft Agent Framework	Agent execution layer

Why this Architecture Matters

This solution goes beyond basic RAG and provides:

Hybrid Retrieval

Combines keyword + semantic + vector search
Improves recall and accuracy

Structured Document Parsing

Handles complex enterprise documents
Extracts tables and metadata

Agent-Based Orchestration

Enables reasoning over retrieval results
Extensible for multi-agent workflows

Scalable Data Pipeline

Supports continuous ingestion
Works with large document collections

Enterprise Considerations

Use Managed Identity for secure service access
Apply RBAC on Cosmos DB / Search / Storage
Enable Private Endpoints for network isolation
Use Guardrails + Evaluations in Foundry

Summary

This repository demonstrates a production-ready Azure RAG architecture:

Ingest → Extract → Chunk → Embed → Index
Retrieve → Reason → Generate
Powered by Azure AI Foundry + Agent Framework

By combining data engineering + AI orchestration, it enables enterprise AI systems that are:

Accurate
Grounded
Extensible

Repo: https://github.com/snd94/azure-rag-strategy-agent

Please refer to the Microsoft Learn Documentation for further information:

Microsoft Developer Community Blog articles

From AI Suggestions to Autonomous CRM Actions in Dynamics 365

🔷 Executive Summary

🔷 The Business Problem

🔷 Solution Overview: CRM Copilot Agent Accelerator

1. Trigger Layer

2. Orchestration Layer

3. AI Processing Layer

4. Data Layer

5. Experience Layer

🔷 The Real Innovation: Modular Add-On Packs

🔹 Key Add-On Categories

🔷 Deep Dive: Key Differentiators

1. From Reactive to Predictive AI

2. Visual AI Experience (PCF Controls)

3. Self-Improving Knowledge Base

4. From AI Suggestion → AI Action

🔷 Business Impact

🔷 Future Roadmap

🔷 Why This Matters

🔷 Call to Action

🔷 Closing Thought

🔷 References & Further Reading

Troubleshooting Azure Container Apps and Jobs for .NET and Django Workloads

Introduction

The Real-World Problem: "My Container App is stuck in a restart loop and I have no idea why"

Scenario 1: Your .NET Application Crashes at Startup

What You See

Why This Happens

Step-by-Step Fix

Scenario 2: Your Django Application Fails to Start

What You See

Why This Happens

Step-by-Step Fix

Scenario 3: Your Container App Job Fails Silently or Times Out

What You See

Step-by-Step Fix

Summary: Your Startup Troubleshooting Checklist

References and Sample Resources

Azure Container Apps docs (core)

.NET and Django references

Sample repositories

What's Next

Getting Secrets Out of YAML: Implementing Azure Key Vault CSI Driver on AKS with Workload Identity

Table of Contents

Why This Pattern Matters

The Problem with Secrets in YAML

What We Wanted to Achieve

Architecture Overview

How the Flow Works

Implementation Prerequisites

Implementation

Phase 1 — Platform Setup

1. Enable Required AKS Capabilities

2. Configure a Managed Identity

3. Configure Federated Identity Trust

4. Prepare Kubernetes Namespaces and ServiceAccounts

Phase 2 — Application Onboarding

5. Define Secret Retrieval Using SecretProviderClass

6. Update Workloads to Use Workload Identity

Enable Workload Identity

Attach the ServiceAccount

Mount the CSI Volume

Consume Secrets Inside the Application

7. Validate the Integration

Operational Recommendation

Understanding the YAML Components

Secret Rotation and Reloaders

Common Pitfalls and Troubleshooting

Federated Credential Subject Mismatch

Missing Workload Identity Label

Missing CSI Volume Mount

Key Vault Permission Issues

Environment Variables Do Not Auto-Refresh

Security and Operational Benefits

Key Takeaways

Microsoft Documentation References

Final Thoughts

Harness-Driven Agents: Secure Podcast Pipeline in Hyperlight MicroVM Sandbox

The moment the agent reached for rm -rf