Creating Autonomous Teams Agents Using OpenClaw, MCP, and Azure Container Apps
The one shift that changes everything For two years, "AI coding" meant autocomplete. A suggestion appears in your editor, you hit tab, you move on. The agent only existed while you were actively typing. That is no longer the only model. A new category of tools runs asynchronously and autonomously: you message the agent from a chat window — Teams, Slack, Telegram — describe what you want, and walk away. The agent plans, writes code, runs tests, deploys, and hands you back a result. Some of them never sleep: they hold a persistent memory, load their own skills, and act on a schedule without being prompted. This is the world of OpenClaw, Hermes Agent, and the other long-running autonomous agents that exploded across developer culture in 2026. OpenClaw alone crossed 377,000 GitHub stars and millions of active users, becoming — for a while — the most-starred project on GitHub. You install it with one line, connect a channel, and start delegating from your phone. The workflow moves from pair programming to delegation and review. The interactive copilot asks, "What should I write next?" The autonomous agent asks, "What do you need done?" And that reframing is exactly why three questions now keep architects awake: Is it safe? You are handing a self-driving process the ability to run shell commands, touch files, and call APIs. One community report memorably described these agents as a teammate in your group chat who happens to have root access to your codebase. That is not a compliment — it is a threat model. Can it fit into real multi-agent work? A single agent is a demo. Production is a fleet — specialists that hand off to each other with gates in between. Is it flexible and controllable? Autonomy is thrilling right up until the agent packages last week's stale files into this week's deliverable, or loops forever on a failing test. This post answers all three — not with hand-waving, but with a working reference implementation you can clone today: CustomCodingAgentApp in the Multi-AI-Agents-Cloud-Native repo, an "Agentic Prototype Factory" that turns a plain-language idea into a tested, live-on-Azure prototype without leaving the chat window. A product manager types "Build a BBC-style World Cup feature page" in Microsoft Teams. Minutes later they get back a running HTTPS URL and a downloadable source ZIP. Under the hood, five specialized OpenClaw agents powered by Microsoft Foundry gpt-5.5 collaborate in a shared sandbox, run real pytest/Jest suites, and ship the result to Azure Container Apps — all orchestrated behind a Model Context Protocol (MCP) service so any MCP client (GitHub Copilot, Claude, the Teams bot) can drive it. We'll build up to that architecture in the order you should learn it. Part 1 — Long-running autonomous agents, and their two hard problems What actually makes them different A traditional chatbot is text in, text out. It waits for you. An autonomous agent inverts that: Property Traditional chatbot Long-running autonomous agent Execution Responds to a prompt Acts proactively (a "heartbeat" wakes it on a schedule) Scope Words Files, shell, browser, APIs — the real machine Memory This session only Persistent across sessions Interface A web box Any chat channel + the terminal Autonomy None Plans and takes multi-step action on its own Architecturally, OpenClaw is not a library you import — it's a runtime. A single long-running process (the Gateway) bridges your messaging channels to an LLM backend, keeps sessions alive, queues work in ordered lanes, and drives the classic agent loop: call the model → execute the tool calls it asks for → feed results back → repeat until done. There is no rigid step-planner; the model itself steers. That is what makes it feel magical — and what makes it hard to contain. That containment problem has two faces. Hard problem #1 — Security The same properties that make an autonomous agent useful make it dangerous. Full system access + proactive execution + a 32,000-server tool ecosystem is a large, self-driving attack surface. OpenClaw's own short history is the cautionary tale: a critical one-click remote-code-execution CVE early in its life, hundreds of malicious community "skills" discovered on its marketplace, and tens of thousands of gateways found exposed on the open internet. None of this means "don't use autonomous agents." It means: never run one with ambient credentials on a machine you care about. The agent belongs in a box with a hard wall around it. Hard problem #2 — Persistence and continuity Real agent work is long. Refactoring a codebase, researching across dozens of pages, building-testing-deploying an app — these take minutes to hours, far past a single request/response. So the runtime needs durable sessions, a place to keep state, and a workspace that survives across steps. But a persistent workspace that is reused creates its own hazard: state leakage. Files from yesterday's task can contaminate — or get shipped inside — today's result. Continuity and cleanliness pull in opposite directions, and you have to engineer the tension out. One agent is a demo; production is a fleet A single monolithic agent asked to "gather requirements, write the code, test it, deploy it, and package it" will do all four mediocrely and blur the boundaries between them. The production pattern is orchestrator-worker: specialized agents, each with one job, handing off to the next through explicit gates. OpenClaw supports exactly this — it can spawn sub-agents and even dispatch external coding harnesses, acting as a meta-orchestrator rather than a single model. The open question is never whether to go multi-agent; it's where the seams and the guardrails go. The answer to "is it safe?": put the agent in a microVM If the agent needs root to be useful, then give it root — inside a disposable microVM, not on your host. In 2026 there are several credible ways to do this: Kata Containers on AKS — each pod gets its own lightweight VM boundary and guest kernel. Hyperlight Wasm — per-call, snapshot-restored Wasm microVMs for running LLM-generated code. Azure Container Apps dynamic sessions — prewarmed, Hyper-V-isolated sandboxes that start in milliseconds, scale to thousands, and are purpose-built for "secure execution of custom code" and "running LLM-generated scripts." That last one — the ACA sandbox — is the sweet spot for a chat-driven agent factory: strong isolation without you operating a Kubernetes cluster, and an exec API to run commands inside the box. It's what the reference implementation uses. Part 2 — Putting OpenClaw into the ACA sandbox Here is where the repo stops being a diagram and becomes running code. The Agentic Prototype Factory decomposes the "idea → live app" job into five specialized OpenClaw agents that run in sequence, all inside the sandbox: requirements → coding → testing → deployment → save Each is addressable as its own model target on the OpenClaw gateway's OpenAI-compatible API: model value Routes to openclaw / openclaw/default Default agent openclaw/requirements-agent Requirement Agent openclaw/coding-agent Coding Agent openclaw/testing-agent Testing Agent openclaw/deployment-agent Deployment Agent openclaw/save-agent Save & download Agent Control, not vibes: review gates with feedback loops Autonomy without gates is how you get an agent that confidently deploys a broken app. The orchestrator wires the five agents into a graph with hard, bounded gates: Every knob is explicit and lives in server.py: _MAX_TEST_ROUNDS = 3, _MAX_DEPLOY_REVIEW = 2, _DEPLOY_POLL_ATTEMPTS = 12, _DEPLOY_POLL_DELAY_S = 20. The Testing Agent must end each turn with a literal TESTS_PASSED / TESTS_FAILED verdict; the orchestrator won't declare success until it HTTP-checks the deployed URL and inspects the response body — because a ResourceNotFound can happily return an HTTP 200. That is what "flexible and controllable" looks like in practice: the LLM drives creatively inside a deterministic state machine. The deterministic pre-run wipe (solving state leakage) Because the sandbox is reused across runs (fast, cheap), the orchestrator does something disciplined before every run: it wipes all lingering agent workspaces. Stale files from a previous task can never leak into — or be packaged as — the new result. This is the engineered answer to Hard Problem #2. Working with the sandbox's limits, not against them The ACA sandbox exec API is hard-capped at ~120 seconds — shorter than a cold az acr build plus az containerapp create. A naive agent would time out and report failure. The clever bit: those commands finish server-side on Azure even after the client exec disconnects. So deployment is split in two: deploy-build <dir> <app> — installs the deploy helpers, writes a tight .dockerignore, and kicks off the ACR build tagged <app>:latest. If the client drops at ~120s, the image still lands in ACR. deploy-finish <app> — idempotent, polled up to 12×. It reports STILL_BUILDING until the image exists, then fires a --no-wait containerapp create, and finally returns DEPLOYED_URL=https://<fqdn>. This is the single most important lesson of the whole sample: an autonomous agent doesn't need a longer timeout — it needs to understand the durability semantics of the platform it runs on. Part 3 — MCP, and why its security is the whole ballgame The five-agent workflow is powerful, but it would be a silo if the only way to reach it were a bespoke API. Instead, the repo wraps the entire orchestration as a Model Context Protocol (MCP) service (acamcp_node) exposed over streamable HTTP at /mcp, with a tiny, legible tool surface: MCP tool What it does generate_prototype Run the full five-agent workflow end to end run_agent Invoke a single named agent check_gateway_health Liveness / readiness of the OpenClaw gateway The payoff is enormous: any MCP client can now drive the factory — GitHub Copilot, Claude, or the Teams bot we're about to meet. One protocol, many front-ends. But MCP is not just an integration convenience — it's a control plane, and every MCP tool is a privileged capability. In an ecosystem with 32,000+ community servers, "just add an MCP server" is a supply-chain decision. A tool call is code execution by another name. So the security posture has to be deliberate. Here is how the reference implementation hardens it — and the principles are portable to any MCP deployment: Auth in front of the protocol. The MCP ingress sits behind basic auth (MCP_BASIC_AUTH_PASSWORD); the gateway itself requires the gateway token as a bearer credential (Authorization: Bearer <token>). No anonymous tool calls. A tiny, named allowlist — not a blank check. The gateway routes only to six explicit model targets. There is no "run arbitrary agent" escape hatch; the routing table is the allowlist. No secrets in the workload. There are no model API keys anywhere in the running containers — model access is brokered entirely through Entra ID managed identities. The gateway token is stored as a Kubernetes secret and never baked into an image. Private by default. The gateway's OpenAI-compatible endpoint is operator-level access — it stays on private ingress, with TLS and authentication added before anything is ever exposed publicly. Least privilege at the identity layer. The gateway is granted exactly the Foundry roles it needs (Cognitive Services User / Cognitive Services OpenAI User) on the Foundry resource — nothing more. The takeaway for MCP is the same as for the agent itself: treat the protocol as a doorway, and put a guard on the door. Authentication, an explicit allowlist, private ingress, and brokered identity turn MCP from an open blast radius into a governed control plane. Part 4 — The complete solution: Teams + MCP on ACA + OpenClaw on the ACA sandbox Now assemble the three deployable components into one loop: The request lifecycle, end to end A PM sends one sentence in Teams. The teamsbot_app bot — acting as an MCP client via mcpClient.ts — opens an MCP handshake and calls generate_prototype. The MCP service on ACA (acamcp_node) runs the orchestrator: pre-run wipe, then requirements → coding → testing. The OpenClaw gateway in the ACA sandbox (acasbxapp_node) executes each agent, talking to Foundry gpt-5.5 through a managed identity — no keys in the box. Real pytest + Jest suites run inside the sandbox. Fail → loop back (bounded). Pass → deploy. Deployment uses the build + poll split to survive the ~120s exec cap; the app lands in Azure Container Apps and is health-checked body-aware at its live URL. The Save Agent produces an authenticated ZIP download URL. The bot streams each agent's progress back into the Teams thread and returns the running HTTPS URL + source ZIP — optionally auto-opening the project in VS Code Insiders. How the architecture answers the three questions The question How this solution answers it Is it safe? The autonomous agent runs in a Hyper-V-isolated ACA sandbox, not on anyone's laptop. No model keys in the workload — Entra ID managed identity brokers Foundry. MCP behind basic auth; gateway behind a bearer token on private ingress; token as a secret, never in an image. A deterministic pre-run wipe removes cross-run leakage. Does it fit multi-agent work? It is a multi-agent system — five specialist OpenClaw agents with A2A hand-offs and review gates — and because it's exposed via MCP, any client (Copilot, Claude, Teams) can orchestrate it. Is it flexible and controllable? Creativity lives inside a deterministic state machine: explicit TESTS_PASSED/FAILED verdicts, bounded retry loops (_MAX_TEST_ROUNDS, _MAX_DEPLOY_REVIEW), body-aware health checks, and a human approving in the Teams thread. Deploy it yourself The repo ships scripts for all three tiers (the gateway uses the platform's managed identity to reach Foundry — no key handling, no image rebuild): # 1) OpenClaw gateway + the 5 agents (acasbxapp_node) cd acasbxapp_node cp .env.example .env # gateway token, Foundry endpoint, sandbox ids ./scripts/build-openclaw-image.sh # build + push the OpenClaw image to ACR ./scripts/deploy-aks-gateway.sh # grant Foundry roles + deploy # 2) MCP service (acamcp_node) cd ../acamcp_node cp .env.example .env # ACR + cluster; gateway token read from ../acasbxapp_node/.env ./scripts/build-images.sh # build + push the MCP image ./scripts/deploy-aks.sh # secret + manifests to the openclaw namespace ./scripts/smoke-check.sh # verify the MCP handshake # 3) Teams bot (teamsbot_app) — Node.js/TypeScript MCP client cd ../teamsbot_app # configure + run per the folder README, then sideload the Teams app package The reference implementation targets Azure (ACA + AKS) — the OpenClaw gateway and MCP service run as containers, and the code-execution sandbox uses the ACA dynamic-sessions exec API. Keep the gateway on private ingress and add TLS before any public exposure. Final thought Strip away the World Cup demo and a reusable pattern remains — a blueprint for running any long-running autonomous agent in the enterprise: A message-driven agent (OpenClaw / Hermes) + a microVM sandbox (Azure Container Apps dynamic sessions) + an MCP control plane with auth + enterprise identity (Entra ID managed identity) + a human surface (Microsoft Teams). The autonomy that made these agents go viral is the same autonomy that makes security teams nervous. You don't resolve that tension by slowing the agent down — you resolve it by giving it a box with a hard wall, a control plane with a guard on the door, an identity instead of a secret, and a human in the loop. Do that, and "your PM types a sentence, Azure ships an app" stops being a scary demo and becomes something you can actually put in production. Clone it, break it, harden it further: kinfey/Multi-AI-Agents-Cloud-Native → code/CustomCodingAgentApp The chat window is the new terminal. Let's make it a safe one.Bringing Enterprise File Data to Users with Azure NetApp Files, Microsoft Foundry, and M365 Copilot
This is Part 3 of a 3-part series on extending AI to enterprise file data, showing how the knowledge pipeline is surfaced through enterprise AI agents and user experiences including Microsoft 365 Copilot.From Enterprise File Storage to an AI-Ready Data Foundation using Azure NetApp Files and OneLake
This 3-part series shows how to extend AI to enterprise file data – without migration – by combining Azure NetApp Files, OneLake, and a RAG-based architecture that surfaces grounded insights through enterprise AI agents. This is Part 1 of a 3-part series covering the data foundation, knowledge pipeline, and user experience layers.From File Data to AI‑Powered Knowledge Pipelines using Azure NetApp Files object REST API
This is Part 2 of a 3-part series on extending AI to enterprise file data hosted on Azure NetApp Files, building on the data foundation to create a knowledge pipeline that makes enterprise file data usable by AI systems.
From AI Suggestions to Autonomous CRM Actions in Dynamics 365
Modern CRM AI solutions often stop at case summarization—but real transformation requires more. This blog introduces a CRM Copilot Agent Accelerator built on Microsoft Power Platform, designed to evolve AI from simple insights to predictive intelligence and ultimately to autonomous actions. By combining Dynamics 365, Dataverse, Power Automate, and AI Builder, and extending capabilities through modular add-on packs, this approach enables organizations to reduce manual effort, improve decision-making, and scale service operations efficiently—without additional Copilot licensing.
File share migrations simplified with Azure Copilot Migration Agent
Building on our earlier announcement of discovery and assessment support for SMB and NFS file shares in Azure Migrate, we are extending the experience to support end-to-end file share migrations within the same workflow. With Azure Copilot Migration Agent, customers can move from discovery and assessment to migration through a single guided experience in Azure Migrate. By bringing planning and execution together, the agent helps organizations streamline migration activity, reduce handoffs, and maintain continuity across stages. Overview Since the release of file share discovery and assessment in Azure Migrate earlier this year, customers have indicated that while visibility into their file share estate improved, the transition to execution remained fragmented. In many cases, teams still had to work across separate workflows for inventory, readiness planning, and migration, increasing operational friction and the risk of losing context between stages. Azure Copilot Migration Agent helps address this gap by bringing discovery, assessment, planning, and execution into a single guided journey. Azure Migrate provides visibility and recommendations, while Azure Storage Mover supports execution in a connected, agentic experience. The result is a more consistent migration path that reduces complexity, preserves context, and helps teams move file shares to Azure with greater operational confidence. Customer Value This update streamlines the migration journey by connecting each stage of the process and reducing operational overhead. Natural language guidance helps teams start and manage migration activities much faster, often in hours or days instead of weeks. The experience supports the following scenarios: End-to-end discovery, assessment, and migration for on-premises Windows and Linux file shares (SMB) to Azure Files. Discovery and assessment for on-premises Windows and Linux file shares (NFS). Data transfers from one Azure Blob container to another container. Design principles The experience preserves continuity across inventory, readiness insights, and execution planning, enables direct movement of validated shares when heavyweight orchestration is unnecessary, maintains approval and sequencing controls, and supports the file and object movement patterns commonly required in production environments. Getting Started with Storage Migration in Azure Copilot Migration Agent (ACMA) Launch Azure Migrate: Sign-in to the Azure portal, open Azure Migrate. From the Getting Started page, open Azure Copilot Migration Agent, then select or create an Azure Migrate project. Describe the migration in natural language. The agent detects storage migration intent and assists with storage migration planning and routes execution requests seamlessly. Examples scenarios and prompts Migration of on-premises Windows Server data over SMB to Azure Files 2. Prompt: Help me transfer data from one Azure blob container to another blob container Call to action Storage integrated capability is launching in Limited Preview at Microsoft Build. Sign up for the Preview here. For questions, contact storagemigrationcopilotagent@microsoft.com. Learn More File share discovery and assessment in Azure Migrate Azure Copilot Migration Agent Azure Storage MoverToken economics–driven architecture: hybrid models, AI Runway, AKS Kata MicroVM, MCP
1. The moment the bill arrived For most of 2024 and 2025, "Agents" were a demo word. In 2026 they are a line item on the cloud invoice. Every major model provider — OpenAI, Anthropic, Google, Mistral, DeepSeek, and even the in-cluster open-weights serving stacks — now bills by the token. Input tokens, output tokens, cached tokens, reasoning tokens, tool-call tokens. The unit price has come down. The number of tokens an autonomous agent burns through has gone up by an order of magnitude. The slide deck I keep coming back to is module 02 of the Enterprise Agent Workshop — Token Economics and Cost Control. The short version: an agentic system is not a chat app. A chat app emits one model call per user turn. An agent emits a model call to plan, another to pick a tool, another to interpret the tool result, another to decide the next step, and another to summarize — and then it loops. Multiply by tools that themselves invoke models. Multiply again by retries and reflection. The bill is no longer "what does the model cost per million tokens." The bill is "what does my architecture cost per user request." This post is about an architecture that answers that question on purpose — and that does it without giving up the security properties an enterprise actually needs. The blueprint lives in this repo, BYOT_Dev: a four-agent SDLC tower (Requirements → Code → Test → Deploy) running on AKS, each agent boxed inside its own Kata MicroVM, each one exposing tools to GitHub Copilot Chat over the Model Context Protocol, and all of them sharing a single on-cluster small-language-model endpoint served by AI Runway 2. Why agentic workloads inflate the token bill Three forces compound: Autonomy multiplies call count. A user typing "build me a URL shortener" produces one prompt at the IDE. By the time a 4-agent pipeline has clarified requirements, generated code, written tests, and produced a Kubernetes manifest, you have spent 30–200 model calls — most of them invisible to the user. Reasoning eats output tokens. Modern reasoning models think before they speak. That hidden chain-of-thought is billed. A 5-line answer might charge you for 3,000 reasoning tokens. Context inflation. Every tool result is re-injected into the next call. A 50 KB code review answer becomes the context of the next refactor turn. Costs grow super-linearly with conversation depth. You can't out-prompt-engineer this. The only durable mitigation is architectural — and it has three levers: Lever What it means in practice Model tiering Use a small, cheap model for narrow tasks; reserve the frontier model for orchestration and judgement. Placement tiering Place each model where it's cheapest to run: on-cluster CPU for tiny SLMs, on-cluster GPU for mid-size models, cloud APIs for frontier reasoning. Protocol tiering Use a standard like MCP so the expensive orchestrator can hand off subtasks to the cheap workers without lock-in. The architecture this post describes pulls all three levers at once. 3. The mental model: frontier brain, small-model hands Look at this picture: spec: image: ghcr.io/kaito-project/aikit/llama3.2:1b model: { id: "kaito/llama3.2-1b", source: huggingface } engine: { type: llamacpp } provider: name: kaito overrides: resource: instanceType: Standard_D4s_v3 preferredNodes: ["aks-nodepool1-21523631-vmss000001"] nodeSelector: { agentpool: nodepool1 } resources: { cpu: "2", memory: "4Gi" } scaling: { replicas: 1 } AI Runway then takes care of: selecting the engine (llamacpp for CPU, vllm or dynamo for GPU); selecting the provider (kaito today, others coming); pulling the model image from the AIKit catalog; exposing an OpenAI-compatible Service at http://llama3-2-1b-cpu.airunway-models.svc:80/v1. A note on the CPU-only example. This repo deliberately uses CPU + Llama-3.2-1B to prove the architecture can run on the cheapest node SKU available. In production you should not assume CPU is always right. The right answer is scenario-driven: Scenario Suggested placement High-volume, narrow, latency-tolerant task (e.g. "expand a requirement into bullet points") On-cluster CPU SLM (1B–3B) — what this repo demonstrates Code generation, refactoring, multi-file reasoning On-cluster GPU mid-model (7B–14B) via KAITO vllm, on an AKS GPU pool that auto-scales from zero Privacy-sensitive enterprise data, must not leave the cluster On-cluster GPU, possibly with confidential compute Frontier reasoning, planning, judging tool output The Copilot seat's already-included frontier model, called sparingly via MCP — not a second pay-per-token endpoint you have to provision AI Runway makes that choice a YAML edit, not a refactor. The point of the abstraction is optionality — the right to change your mind about token economics quarter by quarter without rewriting agents. 5. Hybrid scaling: all inference on AKS, planning on the Copilot tokens you already pay for The single biggest token-economics mistake an enterprise can make right now is treating model placement as a binary — "all in the cluster" or "all on a pay-per-token cloud API." Real workloads are neither. The pattern that actually saves money has two ingredients, and both of them are already on your invoice: AKS that you already provisioned. A small CPU node pool for the steady-state workload, plus a GPU node pool that scales from zero when the small pool can't keep up. Same cluster, same Kata isolation, one invoice line. The Copilot seat the developer already pays for. Copilot Chat's frontier model has its own token allowance baked into the seat. Use that allowance — not a separately provisioned cloud inference endpoint — to do the planning that drives the cheap AKS workers via MCP. That is the whole "hybrid." No external Foundry endpoint, no second per-token meter for inference. Just AKS capacity that grows when you need it + a frontier brain you already pay for. The agent traffic split is roughly: ~85% of agent calls are short, narrow, predictable — "expand this requirement", "format this YAML", "summarize this diff". A 1B–3B model on a CPU node answers these in seconds; the bill is the node, not the token. ~15% are heavier — multi-file refactors, long-context reasoning, the 400-line FastAPI generation. They need a 7B–14B model on a GPU. Planning and judgement on top of all of it are done by the Copilot seat's frontier model, which the user is paying for whether you build BYOT or not. Lever A — a GPU node pool on the same AKS cluster, scaled 0 → N Keep the always-on tiny-cpu ModelDeployment for steady state. Add a second AI Runway ModelDeployment for the mid model on a GPU node pool that is created at size zero and managed by the AKS Cluster Autoscaler az aks nodepool add \ --cluster-name $CLUSTER --resource-group $RG \ --name gpupool \ --node-vm-size Standard_NC24ads_A100_v4 \ --node-count 0 --min-count 0 --max-count 4 \ --enable-cluster-autoscaler \ --node-taints sku=gpu:NoSchedule \ --workload-runtime KataVmIsolation # airunway/modeldeployment-mid-gpu.yaml (sketch) spec: image: ghcr.io/kaito-project/aikit/qwen2.5:7b engine: { type: vllm } provider: name: kaito overrides: resource: instanceType: Standard_NC24ads_A100_v4 nodeSelector: { agentpool: gpupool } tolerations: [{ key: sku, operator: Equal, value: gpu, effect: NoSchedule }] resources: { cpu: "4", memory: "32Gi", nvidia.com/gpu: "1" } scaling: { replicas: 0, maxReplicas: 4 } The key trick is replicas: 0 plus an autoscaler min-count 0. When nobody is asking the mid model anything, no GPU node is running and no GPU node is billed. The first request causes AI Runway to scale to 1, which triggers the Cluster Autoscaler to provision a GPU node, which gets scheduled with Kata Pod Sandboxing intact. When traffic dies down, both the replica and the node go back to zero. All of this is inside AKS — the agents never leave the cluster to find a GPU. Lever B — reuse the Copilot frontier tokens you already pay for This is the lever most token-cost writeups miss. Every developer using BYOT already has a Copilot seat. That seat carries a frontier-model token allowance which Copilot Chat consumes the moment the user types into the chat. The orchestration loop in docs/workflow.md — "plan which tool to call next, read the tool's output, summarize the result" — is paid out of that allowance, not out of a new inference endpoint you provision. This means: You do not stand up a separate cloud OpenAI / Foundry deployment for "the smart model." The smart model is already on the user's screen. You do not put a per-token meter on the agent-to-frontier path. The frontier is upstream of your agents — it calls them via MCP, not the other way around. The only per-token spend the architecture introduces is what Copilot itself charges against the seat, which is independent of how many BYOT agents you stand up. The net effect: the parts of the workload that are expensive per token (planning, judgement) run on tokens the company already buys; the parts that are cheap to compute (long-form generation) run on AKS compute you already pay for as node hours. How the agents pick between the two AKS backends The Agent Framework client in https://github.com/kinfey/Multi-AI-Agents-Cloud-Native/blob/main/code/BYOT_Dev/agents/app/airunway_client.py takes its base_url and model from the ConfigMap. Three strategies, in increasing sophistication: Per-role static binding. byot-requirements and byot-test (cheap, narrow) get AIRUNWAY_BASE_URL=tiny-cpu. byot-code and byot-deploy (heavier generation) get AIRUNWAY_BASE_URL=mid-gpu. One ConfigMap, one rollout. Try-then-scale-up inside the tool. Each tool tries tiny-cpu first; if the answer is too short, fails a quality check, or times out, it retries against mid-gpu. The small model handles the easy 85%; the GPU pool handles only the 15% that actually needed it. AI Gateway in front of both. Put Azure API Management as an AI Gateway in front of the two AI Runway services. The agent talks to one URL; the gateway does semantic caching, token budgeting, and load-aware routing between tiny-cpu and mid-gpu. Both backends remain in your AKS — the gateway only routes. A back-of-envelope token saving Assume one Copilot Chat session through the BYOT tower fires 30 model calls at the lower agents. If those 30 went to an external frontier API at, say, $5 / million output tokens with an average 2 K output per call, that is $0.30 / session in additional lower-tier model spend — stacked on top of what Copilot Chat already charges the seat for planning. With the hybrid AKS + seat-tokens pattern: 25–26 calls (~85%) → tiny-cpu on a CPU node that is already running for the always-on agents → ≈ $0 marginal 4–5 calls (~15%) → mid-gpu, billed as GPU node hours only while AI Runway has scaled up, and amortised across every concurrent BYOT user that lands on the same node → ≈ $0.02–0.05 Planning / judgement → already inside the Copilot seat allowance the developer is paying for → $0 additional A $0.30-per-session pay-per-token outcome collapses toward ≈ $0.02–0.05 of pure AKS compute, and the GPU bill returns to zero when nobody is asking hard questions. That is the lever. The reason it works is that AI Runway gives the agents a single in-cluster front door, AKS gives the cluster elastic GPU capacity it doesn't pay for while idle, and Copilot Chat brings its own pre-paid frontier brain. 6. Kata MicroVM: the hardware-level helmet for agentic code Cost is one half of the agentic-workload problem. The other half is what happens inside the box you put the agent in. Earlier this year I published Giving the Copilot SDK Agent a "hardware-level helmet" using Kata microVM on AKS. The argument, compressed: A traditional container is an apartment with shared roof — the host Linux kernel. For a hand-written service the tenant is predictable. For an agent, the tenant is the model, deciding at runtime which shell command to run, which file to read, which npx package to install. That's a new threat model. Container namespaces aren't sized for it. You want a dedicated guest kernel per Pod — a microVM. Kata Containers is the integration layer that gives Kubernetes microVMs. AKS ships it as Pod Sandboxing with the kata-vm-isolation RuntimeClass on top of Hyper-V — created automatically when the node pool is provisioned with --workload-runtime KataVmIsolation. In BYOT_Dev every agent Pod sets: spec: runtimeClassName: kata-vm-isolation containers: - name: agent securityContext: runAsNonRoot: true readOnlyRootFilesystem: true capabilities: { drop: ["ALL"] } seccompProfile: { type: RuntimeDefault } …and AKS does the rest. The Pod boots a real Hyper-V microVM, with its own guest kernel, before the container even starts. Verifying it is one command: kubectl -n agents exec deploy/byot-requirements -- uname -r # compare with the kernel on the node — they differ → microVM confirmed The repository also pins one agent per node via podAntiAffinity on kubernetes.io/hostname, so the four agents live on four physically distinct Kata hosts — a model escape in one cannot reach the others through a shared host kernel, because there is no shared host kernel. The connection to token economics is this: the moment you trust a cheap on-cluster model to run agent loops on real customer code, the security envelope has to be stronger than a normal container, not weaker. Kata is the thing that makes "cheap" and "safe" not a trade-off. And because AKS Pod Sandboxing applies the same way to the CPU pool, the GPU pool, and any future node pool you add for burst, the hybrid placement story above does not weaken the isolation story — every Pod, on every tier, still boots its own guest kernel. 7. MCP: how GitHub Copilot Chat actually drives this tower The final piece is the protocol. The agents inside the Kata MicroVMs are useless unless something can call them. The "something" the user already has open is GitHub Copilot Chat in VS Code. The Model Context Protocol is the standard Copilot Chat (and almost every other serious agentic IDE) speaks to remote tool servers. In this repo each role exposes its tools via FastMCP over Streamable HTTP — see agents/app/main.py and the per-role tool sets in agents/app/roles/. Service exposure is a small but important detail. The repo uses type: LoadBalancer for each role's Service — see k8s/services.yaml — because: kubectl port-forward does not work against Kata Pods (the listener lives inside the microVM, not in the host sandbox netns); kubectl proxy works but pins Copilot to localhost and requires a long-running local process; a LoadBalancer gives each agent a public Azure IP the IDE can hit directly. Once the four LoadBalancer IPs are in .vscode/mcp.json, Copilot Chat in agent mode sees four MCP servers — byot-requirements, byot-code, byot-test, byot-deploy — and the user can simply say: "Use the byot tower to take this idea — a URL shortener with click analytics — from requirements through deployment." What happens under the covers (docs/workflow.md): Copilot's frontier model plans the sequence. Frontier tokens spent: small, but smart. It calls byot-requirements.gather_requirements({"idea": "URL shortener…"}) over MCP. No frontier tokens; the cluster-side Llama-3.2-1B does the work. It calls byot-code.implement_from_requirements({...}). Same — cluster-side small model. It calls byot-test.generate_test_plan({...}). Same. It calls byot-deploy.generate_k8s_manifest({...}). Same. Copilot's frontier model reads the four results and presents a coherent summary to the user. Frontier tokens spent: small. The expensive model decided what to do five times. The cheap model did the actual long-form generation four times. That is the token-economics win, and the only reason it's possible without lock-in is that MCP is an open standard. 8. Reading the architecture as a budget statement Translate the picture into a unit-cost table — now with the hybrid tiers explicit: Layer Where the cost lives What controls it User input + IDE planning Copilot seat (per-user subscription) Already paid — flat rate Frontier orchestration tokens Copilot seat token allowance — already included, used for MCP planning, no separate endpoint Number of agent rounds Copilot does Tool-call traffic Azure LoadBalancer egress Negligible at this scale tiny-cpu inference (steady state, ~85%) AKS CPU node hours (1× D4s_v3 in this demo) Replicas, model size, batch size mid-gpu inference (autoscaled, ~15%) AKS GPU node hours on the same cluster, only while replicas > 0 Cluster Autoscaler / Karpenter min=0 max=N, scale-to-zero Hardware isolation AKS Pod Sandboxing (Kata) — same node hours Whether you turn it on (you should) Provider swap-out AI Runway YAML A kubectl apply Three things to notice. First, most of the per-request variable cost has moved from a token meter to a node meter. CPU hours are easier to forecast, easier to chargeback, and easier to cap than per-call token spend. You know how many D4s_v3 cores you're paying for; you do not know in advance how many tokens a frontier model will decide it needs. Second, GPU capacity is no longer a fixed bet, and it never leaves AKS. The GPU node pool sits at zero nodes until AI Runway needs it, and when it does, it scales up inside the same cluster under the same Kata RuntimeClass — no second region, no second tenancy, no second per-token bill. Third, the frontier brain is reused, not re-bought. The planning and judgement that drives the whole tower runs on the Copilot seat token allowance the developer already pays for. There is no separate "smart model" cloud endpoint provisioned by BYOT, so there is no second per-token meter to babysit. And because Kata Pod Sandboxing is included in AKS and applies the same way on the CPU pool and the GPU pool, the security cost on top of the compute cost is zero. That is what makes this architecture cost-aware and elastically-scalable and safety-aware at the same time. Those three used to be a trade-off. They no longer are. 9. Six commands, end-to-end For completeness, the repo's run order (README.md): # 0. one-time prereqs: az login, kubectl, helm, docker, aks-preview az login # 1. provision AKS with Kata + ACR + AzureLinux bash infra/01-create-aks-kata.sh # 2. install AI Runway controller + KAITO provider (pinned to v0.5.0) bash infra/02-install-airunway.sh # 3. deploy Llama-3.2-1B on CPU via AI Runway ModelDeployment bash infra/03-deploy-qwen.sh # 4. build & push the single agent image to ACR bash infra/04-build-push-agents.sh # 5. deploy the 4 Kata-isolated MCP agents bash infra/05-deploy-agents.sh # 6. print the public MCP endpoints for GitHub Copilot bash infra/06-show-mcp-endpoints.sh Drop the printed IPs into .vscode/mcp.json, open Copilot Chat, and you have a fully working, hardware-isolated, cost-aware agentic tower talking to a small model on a CPU node — driven by the frontier model the user is already paying a seat for. Add the mid-gpu ModelDeployment on a scale-to-zero GPU node pool alongside it whenever your traffic justifies the next tier; the agents and the Copilot integration don't change, and nothing leaves AKS. 10. Wrapping up: the through-line Let me trace it one more time: Token economics is the new SLO. Agentic workloads multiply model calls; every call has a price. Architecture, not prompts, is what bends the curve. Tier your models, tier your placement. Frontier reasoning at the top; small models for the bulk work; on-cluster CPU for steady state and on-cluster GPU for the heavy 15%. Mix AKS compute with the Copilot tokens you already pay for. Don't add a second pay-per-token cloud endpoint for inference. The heavy compute belongs on an AKS GPU pool that scales from zero; the planning belongs on the Copilot seat allowance the developer already has. That combination both saves tokens (no new per-token meter) and scales elastically (the cluster grows only when AI Runway asks). AI Runway makes placement a YAML edit. Today's CPU Llama is tomorrow's GPU Qwen on the same cluster. Same agent code. Kata MicroVM is non-negotiable for agentic code. The tenant is the model. The roof must be your own. AKS Pod Sandboxing makes it turnkey — and it applies the same way on the CPU pool and the GPU pool. MCP is the bridge. GitHub Copilot Chat is already an MCP client. Expose the cheap workers as MCP tools and the frontier brain calls them — burning the seat tokens, not new tokens. The reference build is in this repo. Six commands, four agents, one tiny CPU model, full microVM isolation, real Copilot Chat integration — and a hybrid scaling path you can layer on without changing the agents or leaving AKS. In the agentic era, a container is not just a box for your application — it is a box for uncertainty and for tokens. The microVM hardens the box; AI Runway lets you slide the model in and out of the box, between CPU and GPU nodes in the same cluster, without rewriting anything; MCP lets the user's expensive IDE drive the cheap box from the outside on tokens already on its tab. That is the through-line. Build the tower. Watch the bill. Further reading Sample Code Giving the Copilot SDK Agent a "hardware-level helmet" using Kata microVM on AKS. AI Runway and KAITO. Kata Containers · AKS Pod Sandboxing. Model Context Protocol · GitHub Copilot Chat MCP support.
Building a GitHub Copilot Agent Usage Dashboard
Introduction Working with organisations that are attempting to create GitHub Copilot custom agents, take-up of these agents by their community becomes important to know. Some questions quickly emerge are "how well are we actually using it?", "which agents are getting used and which have not had that much traction?". Native metrics provide high-level insights into adoption, but they lack the depth needed to answer more granular questions—such as which agent workflows are most used, or how behaviour evolves over time. In this post, I’ll walk through how to build an enterprise-grade GitHub Copilot usage dashboard that captures detailed telemetry from VS Code using OpenTelemetry, processes it in Azure Monitor, and visualises insights in Grafana—all using a reproducible, infrastructure-as-code approach. The dashboard can be made available to anyone that needs it. Architecture VS Code can be configured to emit metrics using Open Telemetry as a standard. This is a configuration item in VS Code and you essentially point it to an Open Telemetry Collector. The collector is an endpoint that can consume the telemetry. In this implementation, it is a container image that is hosted in Azure and I have chosen Azure Container Apps (ACA) for this purpose as it is an easy to use managed environment - but it could also run in Azure Kubernetes Service (AKS) with a little more effort. There is a prebuilt image opentelemetry collector for this and this has been adapted to inject configuration to send the telemetry to Azure Application Insights. For defining and hosting the dashboard, I have chosen another Azure managed service Azure Managed Grafana Sample Dashboard The sample dashboard is one that contains a collection of visualisations derived from the collected data in Application Insights. Azure Managed Grafana allows you to visually author these dashboards or they can be implemented as a JSON file and adapted from there. Note that the telemetry generated by VS Code gives the location of the users - city, region and country, but does not include any personally-identifying information (PII) and so cannot be used to track individuals. As I understand it, this is by design. Managed Grafana has its own permission structure, which may then be used to give users access to the dashboard. Implementation Details There is a GitHub repo Copilot Usage Dashboard that contains details of how to implement this together with instructions for either "click-ops" 🙂creation or via Terraform. So I suggest you follow the link to my repo to look at the details. In summary, there needs to be in Azure: Azure Container App (ACA) that hosts the collector - this needs to have public ingress Azure Container Registry (ACR) that hosts the docker image that is customised via the Dockerfile Key Vault that hosts the Application Insights connection string that ACA references Application Insights - this needs to be created with a flag to allow it to work with Grafana data Log Analytics Workspace that works with Application Insights Azure Managed Grafana to host the Grafana dashboard The main thing to bear in mind is that VS Code needs to be configured to emit OpenTelemetry { "github.copilot.nextEditSuggestions.enabled": true, "github.copilot.chat.otel.enabled": true, "github.copilot.chat.otel.exporterType": "otlp-http", "github.copilot.chat.otel.otlpEndpoint": "https://<fqdn>" } where the FQDN is the URL of the public ingress to the Azure Container App. There is a Dockerfile in this repo that just injects the correct configuration file into the OpenTelemetry collector image. It is this configuration file that tells the collector to emit to Application Insights. It is of the form below: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: batch: attributes: actions: - key: environment value: "prod" action: upsert exporters: azuremonitor: connection_string: "${APPLICATIONINSIGHTS_CONNECTION_STRING}" debug: verbosity: detailed service: pipelines: traces: receivers: [otlp] processors: [batch, attributes] exporters: [azuremonitor, debug] metrics: receivers: [otlp] processors: [batch, attributes] exporters: [azuremonitor] As can be seen above, there is a placeholder for the Application Insights connection string - in the ACA configuration this is an environment variable that then points to a secret which is in key vault. If all is well, VS Code will emit telemetry to the container image running in ACA and this will use its configuration to send to Application Insights. The Grafana dashboard then using this data. Troubleshooting The GitHub repo goes into the detail of troubleshooting, but the overall steps to troubleshoot are: If there is no data in Grafana, check that Grafana has access to Application Insights check whether there is telemetry being pushed into Application Insights by looking at the logs and looking for the contents of the table Dependencies. If there is telemetry there, then it is Grafana permissions. If not, look to ACA Look at ACA logs to see if it is healthy and look to see if there is any logs being received Use a curl request to send a fake log to ACA (a sample is in the repo) to see if the ACA is accepting logs Check the connection to Application Insights is correct and is being pulled from key vault or replace the environment variable value with the connection string directly If all good so far, then it may be that the configuration in VS Code is not correct or in the correct place. Hopefully the more detailed steps will resolve any issues quickly. Further thoughts and enhancements This implementation attempts to build a dashboard showing GitHub Copilot agent usage using a standard set of security controls, but more may be needed. Here is a list of possible enhancements: A more refined dashboard. This should be easy as there are samples for all sorts of visualisations and few of these may allow more focus on agent and model usage. the ACA-hosted OpenTelemetry collector has a public-facing ingress. This may need to be locked-down at the network level by address restriction or by a non-public ingress. Care would need to be taken to make sure that this is then visible/reachable to the intended VS Code user audience The ACA collector endpoint is not authenticated in of itself. This could be achieved at the container level by putting an authenticating proxy in the Dockerfile or at the ACA ingress level. Some investigation would be needed to see how the VS Code configuration could work with this and this may dictate largely what form this authentication can take. How the VS Code configuration changes can be automated for a user base has not been investigated as part of this work. It is assumed that an organisation may be able to roll-out these changes using their application deployment automation. Summary This approach provides a means by which an organisation can track the usage of GitHub Copilot agents (and their models), that is not provided by GitHub Enterprise dashboards. This will provide insights into the take up of custom agents and their underlying models - allowing an organisation to test whether their investments on custom agents are being used effectively. Additionally, the dashboards themselves can easily be rolled-out to a wider community than GitHub Enterprise one.Agents That Build Agents: A SKILL-first Blueprint with MS Agent Framework & Foundry
The single insight that changes everything Most "build an AI agent" tutorials collapse two completely different jobs into one tangled mess: the job of building an agent (writing the code, defining its tools, evaluating it, packaging it), and the job of running an agent (planning, reasoning, calling tools, remembering users, delivering outcomes). Once you separate them, modern agent development becomes a clean two-layer architecture: A Coding Agent sits on top — that's how you produce an agent. A Runtime Agent sits below — that's the agent your business operates. Microsoft Agent Framework is the SDK that ties them together; Microsoft Foundry is the platform both layers publish to and run on. But the secret ingredient — the thing that turns a generic Copilot into a domain-aware engineer — is the SKILL. SKILL is what the Coding Agent reads before writing a single line. It's how requirements become artifacts that actually match your framework, your conventions, and your fixtures. This post walks the entire two-layer architecture, in the order you should learn it — with SKILL as the star of Layer 1. We ground every concept in ZavaShop, a fictional global e-commerce company with 5 fulfillment centers, dozens of suppliers, and a CEO who wants one live dashboard for all of it. Both Python and .NET (C#) are first-class — pick the language your team will run in production. LAYER 1 — The Coding Agent (Build Time) The Coding Agent is not the agent your customer talks to. It's the agent that constructs the agent your customer talks to. Its output is a bundle of artifacts — code, agent definitions, workflows, skills, connectors, evals, tests, configs, docs — that flow through validation and into Foundry. Build time has five movements. Movement 1 — Requirements & Planning Before the Coding Agent writes a single line, you owe it three things: A real business pain. Not "let's build an agent." Rather: "Mei, the supervisor at Seattle DC, gets interrupted 60 times a day by stock-level questions." A list of acceptance criteria. What does "done" look like? "Agent answers stock questions for SKUs in our 10-SKU catalog. P95 latency under 4s. Wrong-tool rate under 5% on the eval set." The fixtures it'll run on. Real or realistic data — warehouses, SKUs, POs, customers — so the Coding Agent isn't reasoning about a vacuum. ZavaShop context. The workshop ships workshop/data/ — 5 warehouses, 10 SKUs, 6 POs, 8 suppliers, 5 contracts, 4 customers (3 VIP), 6 orders, 5 carriers, 4 open exceptions. Every artifact the Coding Agent generates is anchored to this shared fixture set, so numbers stay consistent across the entire system. Movement 2 — The Coding Agent + its SKILL (the star of build time) This is the movement most teams skip — and it's the one that decides whether your build-time output is professional code or "ChatGPT-shaped" code. What a Coding Agent actually is The Coding Agent is GitHub Copilot Chat in Agent Mode, configured with a domain-aware agent definition. In the ZavaShop workshop, it lives at .github/agents/zavashop-coding-agent.agent.md and is activated from the VS Code Agent picker. You start each session with one plain sentence: "I'm working on the inventory agent in Python — wire up stock and PO lookups against the fixtures, plus a HostedMCPTool for the warehouse handbook." Notice what's not in that sentence: no library names, no class names, no file paths. The Coding Agent has to fill all of that in. The mechanism it uses is the SKILL. What a SKILL is A SKILL is a structured contract that teaches the Coding Agent how to write code in your framework, your conventions, and your domain. It is the most important file in the entire build-time layer — without it, GitHub Copilot is a fluent generalist; with it, it becomes a domain-aware specialist that writes code your tech leads would have written. Conceptually, a SKILL contains: Section Purpose Scope & when to use "Use this SKILL for building agents on Foundry / Azure AI — tools, MCP, Toolbox, Skills, Memory, Threads" Framework idioms The exact way to construct AzureAIAgentClient, register function tools, wire HostedMCPTool, create a Thread Code patterns Reference snippets the Coding Agent imitates — naming, import order, error handling, type hints Fixture/data contract How to load workshop/data/, which loaders exist (find_stock, find_po, etc.), where to add sys.path Anti-patterns What not to do — don't hardcode the model name, don't write inline mock dicts, don't bypass the data loader Acceptance heuristics How to map a LAB's acceptance criteria to runnable checks (eval rows, smoke tests) A SKILL is versioned with the codebase. When the framework releases a new idiom, you update the SKILL once; every agent built afterwards picks it up automatically. This is the single biggest reason convention drift disappears. The six SKILLs in the ZavaShop workshop The workshop ships six SKILLs — three for each language track — and they cover three orthogonal capability surfaces: Track SKILL Use it for 🐍 Python agent-framework-azure-ai-py Single agent on Foundry: tools, MCP, Toolbox, Skills, Memory, Threads 🐍 Python agent-framework-workflows-py Multi-agent workflows: WorkflowBuilder, executors, HITL, Checkpoint 🐍 Python agent-framework-agui-py AG-UI server + client: SSE, frontend/backend tools, shared state, HITL 🟦 .NET agent-framework-azure-ai-csharp Same as the Python azure-ai SKILL, for C# 🟦 .NET agent-framework-workflows-csharp Same as the Python workflows SKILL, for C# 🟦 .NET agent-framework-agui-csharp AG-UI in ASP.NET Core: MapAGUI, AGUIChatClient, HITL How the Coding Agent uses SKILL The Coding Agent's loop is SKILL-first, code-second: The discipline is captured in the workshop's one mantra: "Read the SKILL first." It is not optional. Skip it and you're back to generic Copilot output. Movement 3 — A worked example: from a single sentence to a runnable agent Let's trace what happens when you sit down in front of the Coding Agent and say: "I'm working on the inventory agent in Python — wire up stock and PO lookups against the fixtures, plus a HostedMCPTool for the warehouse handbook." Step 1 — The Coding Agent routes The Coding Agent's definition (.github/agents/zavashop-coding-agent.agent.md) contains a routing table: It loads both files into context before doing anything else. Step 2 — The Coding Agent plans Using the SKILL's patterns and the LAB's acceptance criteria, it generates a plan: Every item in the plan traces back to either the SKILL (how) or the LAB README (what). No invention, no guessing. Step 3 — The Coding Agent generates Now — and only now — it writes code. Because the SKILL specified the patterns, what comes out is shaped like the rest of your codebase: # scripts/inventory_agent.py import os import sys from pathlib import Path # Pattern from SKILL: add data folder to sys.path, import named loaders sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "data")) from zava_data import find_stock, find_po # noqa: E402 from agent_framework import HostedMCPTool from agent_framework_azure_ai import AzureAIAgentClient from azure.identity import DefaultAzureCredential def stock_lookup(sku: str, warehouse: str) -> dict: """Return on-hand quantity for a SKU at a warehouse.""" return find_stock(sku, warehouse) def po_lookup(po_id: str) -> dict: """Return the status of a purchase order.""" return find_po(po_id) async def main() -> None: client = AzureAIAgentClient( project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], model_deployment_name=os.environ["FOUNDRY_MODEL"], # gpt-5.5 from env async_credential=DefaultAzureCredential(), ) agent = client.create_agent( name="Zara", instructions=( "You are Zara, the warehouse assistant for Seattle DC (SEA-01). " "Use the tools when asked about stock or POs. " "Answer briefly." ), tools=[ stock_lookup, po_lookup, HostedMCPTool( name="warehouse-docs", url=os.environ["WAREHOUSE_DOCS_MCP_URL"], ), ], ) thread = agent.get_new_thread() print(await agent.run("How many SKU-7421 are at SEA-01?", thread=thread)) Notice the things the SKILL enforced without you having to ask: env-var-driven config, named function tools with docstrings, sys.path data-loader pattern, HostedMCPTool placed alongside function tools, Thread for multi-turn. Step 4 — The Coding Agent validates The SKILL also told it how to validate. The Coding Agent runs: a smoke test against fixtures (SKU-7421 @ SEA-01 → 312), the eval set (eval_queries.jsonl) — was the right tool called? did the answer contain the expected fact? a red-team probe round. It reports back: "3/3 acceptance criteria pass. Eval score 5/5. Red-team: no successful prompt injections." Step 5 — Done What landed in your repo is not just a script. It's an artifact bundle — code + agent definition + tools + eval rows + a one-page README — that matches the way your team writes agents. That bundle is what flows into the next three movements. Movement 4 — Agent Artifacts (the outputs) A well-instructed Coding Agent produces eight kinds of artifact. Together they make up "an agent" in the deployable sense: Artifact What it is Why it matters Source code The Agent / Workflow program Versioned, reviewable, diffable Agent definitions Name, instructions, tool list The "personality" — independently editable Workflows WorkflowBuilder graphs Multi-agent orchestration as code Skills Named, packaged behaviors Reusable capabilities — one Skill, many agents Connectors MCP servers, Toolbox registrations Where the agent reaches into the world Evals eval_queries.jsonl and harness Regression target for every prompt change Tests & configs Unit tests, .env schema, deployment manifests Reproducibility Documentation READMEs, runbooks The agent your future self can operate Don't confuse two senses of "skill" here. A SKILL file (uppercase, in .github/skills/) instructs the Coding Agent at build time. An Agent Skill (a Foundry concept) is a named runtime capability the Runtime Agent calls. Both names are deliberate — Layer 1's SKILL produces, among other artifacts, Layer 2's Skills. Movement 5 — Validation Before any artifact reaches Foundry, four gates run: Tests — unit + integration. Did find_stock("SKU-7421", "SEA-01") return 312, the value in the fixture? Lint & types — ruff/mypy on Python, dotnet build warnings on .NET. The model has to read these signatures; sloppy ones cause real bugs. Evaluation — run the eval set. Did the right tool get called? Did the answer contain the expected fact? You need a score, not a vibe. Red-Team probes — adversarial inputs that try to drift the agent off topic or extract another customer's data. The Foundry red-team SDK ships a battery of these. Evangelist takeaway. "We built an agent" is not a deliverable. "We built an agent and here is its pass rate on a versioned eval set, plus a red-team report" is a deliverable. Validation belongs at build time, not "we'll add it later." Movement 6 — Publish & Deploy When validation is green, the Coding Agent's outputs flow into Foundry and Azure: Push to Microsoft Foundry — agent definitions, Skills, Toolbox tools, and custom evals register against your Foundry project. They are now governed, versioned, and observable. Deploy to Azure — the runtime host (AG-UI server, workflow worker, Teams app, API surface) ships to your Azure target (App Service, Container Apps, AKS, Functions). Same env vars drive local dev and cloud. The same artifact set deploys to dev, staging, and production. There is no "production-only" code in your agent. LAYER 2 — The Runtime Agent (Runtime) Now the agent is live. Every conversation, every action against your data, every memory it writes — that's Layer 2. Five concerns define it. Concern 1 — Users & Channels A Runtime Agent reaches users through the channels they already use: Microsoft Teams — the agent shows up where work already happens. Outlook — triage, reply, summarize, schedule. Custom web / mobile / voice — built on AG-UI, which ships a React client covering streaming text, frontend tools, backend tools, shared state, generative UI, predictive updates, HITL prompts. The channel is a deployment choice, not an architectural choice. The same agent definition can surface in Teams and on a React dashboard. ZavaShop context. Mei's agent shows up in Teams. The CEO's control tower is a React app on top of AG-UI. The agent definition behind both is the same artifact set the Coding Agent produced. Concern 2 — The Runtime Agent itself The Runtime Agent is the loop you've heard about a thousand times — now it's a concrete piece of architecture: AIAgent = model + instructions + tools + thread Inside the loop: The model plans & reasons about the next step. It calls tools through MCP, Toolbox, or local functions. It reads & writes memory. It streams output back to the channel. # Python — the runtime shape (exactly what the Coding Agent produced) agent = client.create_agent( name="Zara", instructions="You are Zara, the warehouse assistant for Seattle DC.", tools=[stock_lookup, po_lookup, warehouse_docs_mcp], ) Concern 3 — Tools & Integrations (the runtime capability surface) At runtime, a Runtime Agent reaches the outside world through four kinds of capabilities — and which one to use is a real engineering decision: Capability Lives in Use when Function tool The agent's own process Local code: a calculation, a DB query, a fixture lookup MCP tool An external MCP server The capability is owned by another system, exposed via MCP Toolbox tool The Foundry project (server-side, tenant-wide) Capability is shared by multiple agents, must be governed Agent Skill The Foundry project A combination of tools + policy as one named capability Mental progression: You don't have to start with Toolbox — but the moment a second agent touches the same domain, migrate. ZavaShop context. Local fixtures → function tools. The warehouse handbook → MCP. Supplier-portal connectors shared by procurement, fulfillment, and finance → Toolbox tools. "Validate-PO-against-contract" → an Agent Skill. Concern 4 — Memory & State State at runtime comes in two flavors: Thread = state inside one conversation thread = agent.get_new_thread() await agent.run("Look up PO-1043.", thread=thread) await agent.run("And its supplier?", thread=thread) # knows which PO Memory = state across conversations Foundry Memory is durable, retrievable knowledge about a user — VIP status, packaging preferences, delivery windows. Memory holds stable preferences and facts, not chat transcripts. ZavaShop context. Customer service agent Aria remembers across sessions that C-204 is VIP, prefers no cardboard, and wants 6–8pm delivery. Concern 5 — Actions & Outcomes Real systems take actions that change state and produce outcomes other systems observe: Trigger events — kick off a workflow, page a human. Generate outputs — write a PO, draft an email, push to a record. Notify channels — send back to Teams, update a dashboard, hit a webhook. Observability — every action streams to Application Insights / Azure Monitor. This is also where Workflows live. WorkflowBuilder is Agent Framework's orchestration primitive: Three workflow features matter most: Reuse, don't rebuild — tools written at build time are workflow nodes at runtime. Human-in-the-Loop (HITL) — pauses, asks a human, resumes from the exact step. Checkpointing — workflows survive process restarts. ZavaShop context. Fulfillment director Diego's team handles a $10K+ exception every day. Before: an email chain across 5 teams. After: a WorkflowBuilder graph with one HITL approval and full audit trail. Cross-cutting: the shared services that make this safe Both layers sit on top of platform services non-negotiable for enterprise deployment: Service What it does for your agents Microsoft Entra ID Who is the user? Who is the agent? Managed identity for tool calls Microsoft Defender for Cloud Threat detection across the agent's compute + data plane Microsoft Sentinel SIEM — correlate agent actions with security signals Azure Key Vault Secrets, keys, connection strings — never in code, never in .env checked to git Azure Monitor / App Insights Every agent turn, every tool call, every workflow step — observable and queryable Azure Policy & governance Guardrails on what can be deployed where, by whom Skip this row and you have a demo that has not yet failed. Mapping the ZavaShop workshop to the architecture Layer 1 artifacts shipped in the repo: .github/agents/zavashop-coding-agent.agent.md — the Coding Agent definition .github/skills/agent-framework-{azure-ai,workflows,agui}-{py,csharp}/ — the six SKILLs workshop/data/ — shared fixtures every artifact grounds in Per-lab READMEs + eval_queries.jsonl — Layer 1 validation inputs Layer 2 artifacts produced over the course of the workshop: A single agent (Zara) — function tools + HostedMCPTool + Thread A procurement agent (Pierre) — Toolbox + Agent Skills + approval policy A customer-service agent (Aria) — Foundry Memory + Evaluation + Red-Team A multi-agent fulfillment workflow (Diego) — WorkflowBuilder + HITL + Checkpoint An AG-UI control tower for the CEO — covering all 7 AG-UI features Same model across the stack — gpt-5.5 on Foundry + text-embedding-3-small. Change one env var, run the same artifact in the other language. Three habits that separate strong agent engineers Read the SKILL first. Make it ritual. The Coding Agent does it automatically; you should do it manually when reviewing the agent's output. Treat tools as a public API. Names, signatures, docstrings, return shapes — they are how the model sees your system at runtime. Refactor them like any other API. Measure before you tune. A prompt change without an eval delta is a vibe. With one, it's engineering. Getting started in 60 seconds git clone https://github.com/microsoft/Learn-Microsoft-Agent-Framework-with-Foundry-ZavaShop-Supply-Chain-Workshop cd Learn-Microsoft-Agent-Framework-with-Foundry-ZavaShop-Supply-Chain-Workshop # Foundry prereqs: gpt-5.5 + text-embedding-3-small deployed in your Foundry project az login --use-device-code # Python track python -m venv .venv && source .venv/bin/activate pip install agent-framework agent-framework-azure-ai agent-framework-ag-ui \ azure-identity python-dotenv fastapi "uvicorn[standard]" # .NET track dotnet --version # ≥ 10.0.100 # .env at repo root cat > .env <<EOF FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com/api/projects/<project-name> FOUNDRY_MODEL=gpt-5.5 AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-3-small AGUI_SERVER_URL=http://127.0.0.1:5100/ AG_UI_API_KEY=zava-control-tower-demo-key EOF # In VS Code → Copilot Chat → Agent Mode → pick zavashop-coding-agent # Then say: "I'm working on the inventory agent in Python — meet Mei." The one mantra: "Read the SKILL first." Closing thought Modern agent development is not one job — it's two. The Coding Agent designs and builds; the Runtime Agent operates and delivers. Microsoft Agent Framework is the SDK that makes both layers feel like the same conceptual model. Microsoft Foundry is the platform both layers publish to and run on. And the engine that turns a generic Copilot into a domain-aware engineer — that takes a sentence-long requirement and lands a runnable, validated, deployable artifact — is the SKILL. Write a good SKILL once, and every agent built afterwards inherits your team's taste, your fixtures, your patterns, your discipline. The ZavaShop workshop is the smallest end-to-end example I can give you that actually exercises both layers, with six SKILLs ready to read. Walk it once, and the next time someone asks "how do we build agents in our org?", you won't be pointing at a tutorial — you'll be pointing at an architecture. 👉 Start with the workshop on GitHubMoving Beyond Prompts: A Practical Introduction to Spec-Driven Development
In the last year, many of us have started writing code differently. We describe what we want, let AI generate an answer, review it, tweak the prompt, and try again. This loop—prompt, retry, adjust—has quietly become part of our daily workflow. At first, it feels incredibly productive. But as the complexity of the task increases, something changes. The iteration cycle becomes longer, outputs become inconsistent, and the effort shifts from solving the problem to refining the prompt. This is where a subtle but important shift in approach can help: moving from prompt-driven development to spec-driven development. The Problem: Prompt → Retry → Guess Most AI-assisted workflows today look something like this: Write a prompt describing the task Review the generated output Adjust the prompt Repeat until it looks acceptable In practice, this often simplifies to: Prompt → Retry → Guess Figure: Prompt-driven vs spec-driven workflow comparison For simple tasks, this works well. But for anything involving multiple inputs, constraints, or edge cases, the process can become unpredictable. In my experience, the challenge is not the model—it is the lack of structure in how we describe the problem. A Shift in Thinking: From Prompts to Specifications Instead of asking AI to “figure it out,” spec-driven development introduces a simple idea: Define the problem clearly before asking for a solution. A specification (spec) is not a long document—it is a structured way of describing: Inputs Outputs Constraints Edge cases When this structure is provided upfront, the interaction changes significantly. Rather than iterating on vague prompts, you are guiding the system with a clear contract. What This Looks Like in Practice Let’s take a simple example: an order summary API (for example, a backend service hosted on Azure App Service). Without a Spec (Typical Prompt) “Write an API that returns order details for a user.” A model can generate something reasonable, but in practice, the responses often vary: Field names may be inconsistent Pagination may be missing Edge cases (no orders, large datasets) may not be handled Structure may change across iterations Example response (typical output): { "userId": 123, "orders": [ { "id": 1, "amount": 250 } ] } With a Spec (Structured Input) Now consider providing a simple specification: Specification: Input: userId page pageSize Output: userId orders[] orderId totalAmount orderDate pagination page pageSize totalRecords Constraints: Default pageSize = 10 Return empty list if no orders Handle large datasets efficiently Example response (based on the spec): { "userId": 123, "orders": [ { "orderId": 1, "totalAmount": 250, "orderDate": "2024-01-10" } ], "pagination": { "page": 1, "pageSize": 10, "totalRecords": 50 } } Why This Tends to Work The difference here is not just stylistic—it is structural. An unstructured prompt leaves room for interpretation. A spec reduces ambiguity by defining expectations explicitly. In practice, I have observed that providing structured inputs like this often leads to the following: More consistent field naming Better handling of edge cases Reduced need for repeated prompt refinement Rather than relying on trial-and-error, the interaction becomes more predictable and aligned with expectations. Applying This to Existing Code (Refactor Scenario) This approach becomes even more useful when applied to existing code. Instead of asking: “Fix the bug in the Auth controller” You can define expected behavior: Input validation rules Response formats Error handling Authorization behavior The task then becomes aligning the implementation with the defined spec. This shifts the interaction from guesswork to validation—comparing current behavior with intended behavior. Example Comparison (Auth Scenario) Without Spec (Typical Prompt) “Fix the login issue in Auth controller” Possible outcomes include: Partial validation added Inconsistent error responses No clear handling of repeated failed attempts With Spec (Defined Behavior) Spec defines: Validate username and password Return consistent error responses Lock account after 5 failed attempts Do not expose internal errors Resulting behavior: Input validation is consistently applied Error responses follow a defined structure Edge cases like account lockout are handled explicitly This mirrors the same pattern seen in the API example—moving from ambiguity to clearly defined behavior. A Practical Way to Start You do not need new tools or frameworks to try this. A simple workflow that has worked well in practice: Ask – Describe the problem (prompt, discussion, or notes) Write a spec – Define inputs, outputs, constraints Refine – Remove ambiguity Generate – Use the spec as input Validate – Compare output with the spec This adds a small upfront step, but it often reduces back-and-forth iterations later. The Practical Challenge One important point to note: Writing a good spec requires understanding the problem. Spec-driven development does not eliminate complexity—it surfaces it earlier. In many cases, the hardest part is not writing code, but clearly defining: What the system should do What it should not do How it should behave under edge conditions This is also why specs evolve over time. They do not need to be perfect upfront. They improve as your understanding improves. Where This Approach Helps From what I have seen, this approach is most useful in scenarios where the problem involves multiple inputs, defined contracts, or structured outputs such as APIs, schema-driven systems, or refactoring existing code where consistency matters. Where It May Not Be Necessary For simpler tasks such as small scripts, minor UI changes, or quick experiments, a detailed specification may not add much value. In those cases, a straightforward prompt is often sufficient. A Note on Tools Tools like GitHub Copilot, Azure AI Studio, and AI-assisted workflows in Visual Studio Code tend to be more effective when given clear, structured inputs. Spec-driven development is not tied to any specific tool. It is a way of thinking about how we interact with these systems more effectively. References https://github.com/features/copilot https://platform.openai.com/docs/guides/prompt-engineering https://github.com/github/spec-kit Amplifier - Modular AI Agent Framework - Amplifier Final Thoughts Many discussions around AI-assisted development focus on what tools can do. This approach focuses on something slightly different: How developers can structure problems more effectively before implementation. In my experience, moving from prompts to specs does not eliminate iteration, but it makes that iteration more predictable and purposeful.
