Creating Autonomous Teams Agents Using OpenClaw, MCP, and Azure Container Apps
The one shift that changes everything For two years, "AI coding" meant autocomplete. A suggestion appears in your editor, you hit tab, you move on. The agent only existed while you were actively typing. That is no longer the only model. A new category of tools runs asynchronously and autonomously: you message the agent from a chat window — Teams, Slack, Telegram — describe what you want, and walk away. The agent plans, writes code, runs tests, deploys, and hands you back a result. Some of them never sleep: they hold a persistent memory, load their own skills, and act on a schedule without being prompted. This is the world of OpenClaw, Hermes Agent, and the other long-running autonomous agents that exploded across developer culture in 2026. OpenClaw alone crossed 377,000 GitHub stars and millions of active users, becoming — for a while — the most-starred project on GitHub. You install it with one line, connect a channel, and start delegating from your phone. The workflow moves from pair programming to delegation and review. The interactive copilot asks, "What should I write next?" The autonomous agent asks, "What do you need done?" And that reframing is exactly why three questions now keep architects awake: Is it safe? You are handing a self-driving process the ability to run shell commands, touch files, and call APIs. One community report memorably described these agents as a teammate in your group chat who happens to have root access to your codebase. That is not a compliment — it is a threat model. Can it fit into real multi-agent work? A single agent is a demo. Production is a fleet — specialists that hand off to each other with gates in between. Is it flexible and controllable? Autonomy is thrilling right up until the agent packages last week's stale files into this week's deliverable, or loops forever on a failing test. This post answers all three — not with hand-waving, but with a working reference implementation you can clone today: CustomCodingAgentApp in the Multi-AI-Agents-Cloud-Native repo, an "Agentic Prototype Factory" that turns a plain-language idea into a tested, live-on-Azure prototype without leaving the chat window. A product manager types "Build a BBC-style World Cup feature page" in Microsoft Teams. Minutes later they get back a running HTTPS URL and a downloadable source ZIP. Under the hood, five specialized OpenClaw agents powered by Microsoft Foundry gpt-5.5 collaborate in a shared sandbox, run real pytest/Jest suites, and ship the result to Azure Container Apps — all orchestrated behind a Model Context Protocol (MCP) service so any MCP client (GitHub Copilot, Claude, the Teams bot) can drive it. We'll build up to that architecture in the order you should learn it. Part 1 — Long-running autonomous agents, and their two hard problems What actually makes them different A traditional chatbot is text in, text out. It waits for you. An autonomous agent inverts that: Property Traditional chatbot Long-running autonomous agent Execution Responds to a prompt Acts proactively (a "heartbeat" wakes it on a schedule) Scope Words Files, shell, browser, APIs — the real machine Memory This session only Persistent across sessions Interface A web box Any chat channel + the terminal Autonomy None Plans and takes multi-step action on its own Architecturally, OpenClaw is not a library you import — it's a runtime. A single long-running process (the Gateway) bridges your messaging channels to an LLM backend, keeps sessions alive, queues work in ordered lanes, and drives the classic agent loop: call the model → execute the tool calls it asks for → feed results back → repeat until done. There is no rigid step-planner; the model itself steers. That is what makes it feel magical — and what makes it hard to contain. That containment problem has two faces. Hard problem #1 — Security The same properties that make an autonomous agent useful make it dangerous. Full system access + proactive execution + a 32,000-server tool ecosystem is a large, self-driving attack surface. OpenClaw's own short history is the cautionary tale: a critical one-click remote-code-execution CVE early in its life, hundreds of malicious community "skills" discovered on its marketplace, and tens of thousands of gateways found exposed on the open internet. None of this means "don't use autonomous agents." It means: never run one with ambient credentials on a machine you care about. The agent belongs in a box with a hard wall around it. Hard problem #2 — Persistence and continuity Real agent work is long. Refactoring a codebase, researching across dozens of pages, building-testing-deploying an app — these take minutes to hours, far past a single request/response. So the runtime needs durable sessions, a place to keep state, and a workspace that survives across steps. But a persistent workspace that is reused creates its own hazard: state leakage. Files from yesterday's task can contaminate — or get shipped inside — today's result. Continuity and cleanliness pull in opposite directions, and you have to engineer the tension out. One agent is a demo; production is a fleet A single monolithic agent asked to "gather requirements, write the code, test it, deploy it, and package it" will do all four mediocrely and blur the boundaries between them. The production pattern is orchestrator-worker: specialized agents, each with one job, handing off to the next through explicit gates. OpenClaw supports exactly this — it can spawn sub-agents and even dispatch external coding harnesses, acting as a meta-orchestrator rather than a single model. The open question is never whether to go multi-agent; it's where the seams and the guardrails go. The answer to "is it safe?": put the agent in a microVM If the agent needs root to be useful, then give it root — inside a disposable microVM, not on your host. In 2026 there are several credible ways to do this: Kata Containers on AKS — each pod gets its own lightweight VM boundary and guest kernel. Hyperlight Wasm — per-call, snapshot-restored Wasm microVMs for running LLM-generated code. Azure Container Apps dynamic sessions — prewarmed, Hyper-V-isolated sandboxes that start in milliseconds, scale to thousands, and are purpose-built for "secure execution of custom code" and "running LLM-generated scripts." That last one — the ACA sandbox — is the sweet spot for a chat-driven agent factory: strong isolation without you operating a Kubernetes cluster, and an exec API to run commands inside the box. It's what the reference implementation uses. Part 2 — Putting OpenClaw into the ACA sandbox Here is where the repo stops being a diagram and becomes running code. The Agentic Prototype Factory decomposes the "idea → live app" job into five specialized OpenClaw agents that run in sequence, all inside the sandbox: requirements → coding → testing → deployment → save Each is addressable as its own model target on the OpenClaw gateway's OpenAI-compatible API: model value Routes to openclaw / openclaw/default Default agent openclaw/requirements-agent Requirement Agent openclaw/coding-agent Coding Agent openclaw/testing-agent Testing Agent openclaw/deployment-agent Deployment Agent openclaw/save-agent Save & download Agent Control, not vibes: review gates with feedback loops Autonomy without gates is how you get an agent that confidently deploys a broken app. The orchestrator wires the five agents into a graph with hard, bounded gates: Every knob is explicit and lives in server.py: _MAX_TEST_ROUNDS = 3, _MAX_DEPLOY_REVIEW = 2, _DEPLOY_POLL_ATTEMPTS = 12, _DEPLOY_POLL_DELAY_S = 20. The Testing Agent must end each turn with a literal TESTS_PASSED / TESTS_FAILED verdict; the orchestrator won't declare success until it HTTP-checks the deployed URL and inspects the response body — because a ResourceNotFound can happily return an HTTP 200. That is what "flexible and controllable" looks like in practice: the LLM drives creatively inside a deterministic state machine. The deterministic pre-run wipe (solving state leakage) Because the sandbox is reused across runs (fast, cheap), the orchestrator does something disciplined before every run: it wipes all lingering agent workspaces. Stale files from a previous task can never leak into — or be packaged as — the new result. This is the engineered answer to Hard Problem #2. Working with the sandbox's limits, not against them The ACA sandbox exec API is hard-capped at ~120 seconds — shorter than a cold az acr build plus az containerapp create. A naive agent would time out and report failure. The clever bit: those commands finish server-side on Azure even after the client exec disconnects. So deployment is split in two: deploy-build <dir> <app> — installs the deploy helpers, writes a tight .dockerignore, and kicks off the ACR build tagged <app>:latest. If the client drops at ~120s, the image still lands in ACR. deploy-finish <app> — idempotent, polled up to 12×. It reports STILL_BUILDING until the image exists, then fires a --no-wait containerapp create, and finally returns DEPLOYED_URL=https://<fqdn>. This is the single most important lesson of the whole sample: an autonomous agent doesn't need a longer timeout — it needs to understand the durability semantics of the platform it runs on. Part 3 — MCP, and why its security is the whole ballgame The five-agent workflow is powerful, but it would be a silo if the only way to reach it were a bespoke API. Instead, the repo wraps the entire orchestration as a Model Context Protocol (MCP) service (acamcp_node) exposed over streamable HTTP at /mcp, with a tiny, legible tool surface: MCP tool What it does generate_prototype Run the full five-agent workflow end to end run_agent Invoke a single named agent check_gateway_health Liveness / readiness of the OpenClaw gateway The payoff is enormous: any MCP client can now drive the factory — GitHub Copilot, Claude, or the Teams bot we're about to meet. One protocol, many front-ends. But MCP is not just an integration convenience — it's a control plane, and every MCP tool is a privileged capability. In an ecosystem with 32,000+ community servers, "just add an MCP server" is a supply-chain decision. A tool call is code execution by another name. So the security posture has to be deliberate. Here is how the reference implementation hardens it — and the principles are portable to any MCP deployment: Auth in front of the protocol. The MCP ingress sits behind basic auth (MCP_BASIC_AUTH_PASSWORD); the gateway itself requires the gateway token as a bearer credential (Authorization: Bearer <token>). No anonymous tool calls. A tiny, named allowlist — not a blank check. The gateway routes only to six explicit model targets. There is no "run arbitrary agent" escape hatch; the routing table is the allowlist. No secrets in the workload. There are no model API keys anywhere in the running containers — model access is brokered entirely through Entra ID managed identities. The gateway token is stored as a Kubernetes secret and never baked into an image. Private by default. The gateway's OpenAI-compatible endpoint is operator-level access — it stays on private ingress, with TLS and authentication added before anything is ever exposed publicly. Least privilege at the identity layer. The gateway is granted exactly the Foundry roles it needs (Cognitive Services User / Cognitive Services OpenAI User) on the Foundry resource — nothing more. The takeaway for MCP is the same as for the agent itself: treat the protocol as a doorway, and put a guard on the door. Authentication, an explicit allowlist, private ingress, and brokered identity turn MCP from an open blast radius into a governed control plane. Part 4 — The complete solution: Teams + MCP on ACA + OpenClaw on the ACA sandbox Now assemble the three deployable components into one loop: The request lifecycle, end to end A PM sends one sentence in Teams. The teamsbot_app bot — acting as an MCP client via mcpClient.ts — opens an MCP handshake and calls generate_prototype. The MCP service on ACA (acamcp_node) runs the orchestrator: pre-run wipe, then requirements → coding → testing. The OpenClaw gateway in the ACA sandbox (acasbxapp_node) executes each agent, talking to Foundry gpt-5.5 through a managed identity — no keys in the box. Real pytest + Jest suites run inside the sandbox. Fail → loop back (bounded). Pass → deploy. Deployment uses the build + poll split to survive the ~120s exec cap; the app lands in Azure Container Apps and is health-checked body-aware at its live URL. The Save Agent produces an authenticated ZIP download URL. The bot streams each agent's progress back into the Teams thread and returns the running HTTPS URL + source ZIP — optionally auto-opening the project in VS Code Insiders. How the architecture answers the three questions The question How this solution answers it Is it safe? The autonomous agent runs in a Hyper-V-isolated ACA sandbox, not on anyone's laptop. No model keys in the workload — Entra ID managed identity brokers Foundry. MCP behind basic auth; gateway behind a bearer token on private ingress; token as a secret, never in an image. A deterministic pre-run wipe removes cross-run leakage. Does it fit multi-agent work? It is a multi-agent system — five specialist OpenClaw agents with A2A hand-offs and review gates — and because it's exposed via MCP, any client (Copilot, Claude, Teams) can orchestrate it. Is it flexible and controllable? Creativity lives inside a deterministic state machine: explicit TESTS_PASSED/FAILED verdicts, bounded retry loops (_MAX_TEST_ROUNDS, _MAX_DEPLOY_REVIEW), body-aware health checks, and a human approving in the Teams thread. Deploy it yourself The repo ships scripts for all three tiers (the gateway uses the platform's managed identity to reach Foundry — no key handling, no image rebuild): # 1) OpenClaw gateway + the 5 agents (acasbxapp_node) cd acasbxapp_node cp .env.example .env # gateway token, Foundry endpoint, sandbox ids ./scripts/build-openclaw-image.sh # build + push the OpenClaw image to ACR ./scripts/deploy-aks-gateway.sh # grant Foundry roles + deploy # 2) MCP service (acamcp_node) cd ../acamcp_node cp .env.example .env # ACR + cluster; gateway token read from ../acasbxapp_node/.env ./scripts/build-images.sh # build + push the MCP image ./scripts/deploy-aks.sh # secret + manifests to the openclaw namespace ./scripts/smoke-check.sh # verify the MCP handshake # 3) Teams bot (teamsbot_app) — Node.js/TypeScript MCP client cd ../teamsbot_app # configure + run per the folder README, then sideload the Teams app package The reference implementation targets Azure (ACA + AKS) — the OpenClaw gateway and MCP service run as containers, and the code-execution sandbox uses the ACA dynamic-sessions exec API. Keep the gateway on private ingress and add TLS before any public exposure. Final thought Strip away the World Cup demo and a reusable pattern remains — a blueprint for running any long-running autonomous agent in the enterprise: A message-driven agent (OpenClaw / Hermes) + a microVM sandbox (Azure Container Apps dynamic sessions) + an MCP control plane with auth + enterprise identity (Entra ID managed identity) + a human surface (Microsoft Teams). The autonomy that made these agents go viral is the same autonomy that makes security teams nervous. You don't resolve that tension by slowing the agent down — you resolve it by giving it a box with a hard wall, a control plane with a guard on the door, an identity instead of a secret, and a human in the loop. Do that, and "your PM types a sentence, Azure ships an app" stops being a scary demo and becomes something you can actually put in production. Clone it, break it, harden it further: kinfey/Multi-AI-Agents-Cloud-Native → code/CustomCodingAgentApp The chat window is the new terminal. Let's make it a safe one.Deploying Foundry Hosted Agents from Source Code
Introduction At Microsoft Build, it was announced that Foundry Hosted Agents now support source-code deployments. Previously, Hosted Agents required application code to be packaged in a container for deployment. This new functionality allows you to deploy the agent from a `.zip` file instead of from a container image. This post walks through the process of deploying a source-code Hosted Agent, briefly compares that approach to container-based Hosted Agent deployment, and provides a reusable GitHub Action for CI/CD deployments. It is part of a series of post whose source code is housed in simple-hosted-agent-responses repository. If Hosted Agents are new to you, read the previous posts, "Deploying Foundry Hosted Agents via REST API" and "GitHub Actions for Deploying Hosted Agents." Background A Foundry Hosted Agent helps abstract the management of the compute tier for your agent. It runs in a self-contained Micro-VM sandbox, meaning the Hosted Agent sandbox provides the CPU and memory allocation used to run your agent. Previously, this Micro-VM would download your code from an Azure Container Registry (ACR) and run it on the virtualized platform. Not all customers use container-based workloads today and, let's face it, not everything needs to be a container. So how do those customers and platforms take advantage of Foundry Hosted Agents? The answer is through source-code deployments of Foundry Hosted Agents. What is a Source Code Agent? Source Code Agents are like other Foundry Hosted Agents. The key deployment difference is that the code asset is a .zip file instead of a container image. This also changes the Agent Development Lifecycle compared with the containerized version of Foundry Hosted Agents. An important point of clarity: the way the agent is configured is a data plane operation. As such, taking advantage of Source Code Agent functionality does not require changes to the Foundry infrastructure itself when your Infrastructure as Code (IaC) is only provisioning the supporting resources in Bicep, Terraform, or PowerShell. The deployment change happens through the Foundry data plane. First, let's look at a container-based Foundry Hosted Agent: Now, let's compare it to the source-code version: Deployment Process Now that we've looked at the end result, let's talk through the steps required to deploy a Foundry Hosted Agent via source code. So in Foundry, what does the difference between a container-based and a source-code-based Foundry Hosted Agent look like? The Microsoft Learn docs outline this well: Every source-code deployment follows the same sequence: package -> create or update -> poll until active -> invoke. The source-code path uses `code_configuration` in the agent definition; the image-based path uses `container_configuration` instead--the two are mutually exclusive on a single version. If wanting to confirm and see in more detail one can refer to the Foundry Agent REST API documentation. The source layout can stay familiar, but the deployed artifact changes to a `.zip` file. Packaging the source code into a ZIP is the piece that differs from the container-image flow. The agent deployment to Foundry is also slightly different because it uses source-code configuration instead of container configuration. You can run this via `azd` with a command structured like the following: azd ai agent init --no-prompt --project-id "<project-resource-id>" --deploy-mode code --runtime python_3_13 --entry-point main.py This assumes `azd` is installed and authenticated, and that the authenticated identity has access to the Foundry project. The command initializes a code deployment for the project. However, we recognize that the majority of enterprise organizations will want to use other deployment methods. As such, REST API deployments are supported, as are the Python and C# SDKs for creating the agent. Taking this a step further, and similar to "GitHub Actions for Deploying Hosted Agents," let's create a reusable GitHub Action for deploying source-code-based Hosted Agents. GitHub Action If you are wanting to see the entire action it is part of the repository simple-hosted-agent-responses, which contains source code, IaC, and deployment options. Background First, we need to understand that we cannot reuse the GitHub Action from "GitHub Actions for Deploying Hosted Agents" because, as noted above, the REST API uses mutually exclusive options. In theory, we could add conditional logic across the parameters; however, it is cleaner to create a separate action. Before invoking this action, the workflow must authenticate to Azure because the action calls `az account get-access-token` to acquire a token for the Foundry data plane. Inputs inputs: project_endpoint: description: Foundry project endpoint URL required: true agent_name: description: Name of the hosted agent required: true source_code_zip: description: Path to the local source-code zip artifact required: true model_deployment_name: description: Name of the AI model deployment required: true cpu: description: CPU allocation for the hosted agent container required: false default: '0.25' memory: description: Memory allocation for the hosted agent container required: false default: '0.5Gi' runtime: description: Source-code runtime for the hosted agent required: false default: 'python_3_13' entry_point: description: Source-code entry point command for the hosted agent required: false default: '["python", "main.py"]' dependency_resolution: description: How Agent Service resolves dependencies for the source-code deployment required: false default: 'remote_build' max_polling_seconds: description: Maximum time to wait for the source-code deployment to reach active status required: false default: '600' For our inputs, `project_endpoint`, `agent_name`, `source_code_zip`, and `model_deployment_name` are required. The CPU, memory, runtime, entry point, dependency resolution, and max polling values are configurable properties with defaults set in the action. The source-code-specific inputs populate the `code_configuration` properties of the REST payload. These include `source_code_zip`, `runtime`, `entry_point`, and `dependency_resolution`. This information tells Foundry how to run the code from the `.zip` package. Outputs We should output values that make sense for downstream workflows. Every workflow may not use them, but it is useful to expose non-secret values when they can support later steps. In this case, we are creating a new version of the agent, so let's output that version ID. outputs: agent_version: description: Version ID returned by the Foundry data plane value: ${{ steps.post.outputs.agent_version }} Action The action maps the inputs to environment variables as the first step. After that, it gets an access token from Azure and calls the REST API endpoint. Once we have this, we prepare the body of the call. Verify against the API for all valid properties. For this example, I chose not to set `rai_config` and `tools` to keep things simple. runs: using: composite steps: - name: Create source-code metadata id: metadata shell: bash env: AGENT_NAME: ${{ inputs.agent_name }} MODEL_DEPLOYMENT_NAME: ${{ inputs.model_deployment_name }} CPU: ${{ inputs.cpu }} MEMORY: ${{ inputs.memory }} RUNTIME: ${{ inputs.runtime }} ENTRY_POINT: ${{ inputs.entry_point }} DEPENDENCY_RESOLUTION: ${{ inputs.dependency_resolution }} run: | METADATA_FILE=$(mktemp) ENTRY_POINT_JSON=$(python3 -c 'import json,sys; print(json.dumps(json.loads(sys.argv[1])))' "$ENTRY_POINT") jq -n \ --arg model "$MODEL_DEPLOYMENT_NAME" \ --arg cpu "$CPU" \ --arg memory "$MEMORY" \ --arg runtime "$RUNTIME" \ --arg dep_resolution "$DEPENDENCY_RESOLUTION" \ --argjson entry_point "$ENTRY_POINT_JSON" \ '{ description: "Hosted agent deployed from source code", definition: { kind: "hosted", protocol_versions: [{protocol: "responses", version: "1.0.0"}], cpu: $cpu, memory: $memory, code_configuration: { runtime: $runtime, entry_point: $entry_point, dependency_resolution: $dep_resolution }, environment_variables: {AZURE_AI_MODEL_DEPLOYMENT_NAME: $model} } }' > "$METADATA_FILE" echo "metadata_file=${METADATA_FILE}" >> "$GITHUB_OUTPUT" echo "Metadata file created at ${METADATA_FILE}" - name: Post source-code agent deployment to Foundry data plane id: post shell: bash env: PROJECT_ENDPOINT: ${{ inputs.project_endpoint }} AGENT_NAME: ${{ inputs.agent_name }} SOURCE_CODE_ZIP: ${{ inputs.source_code_zip }} METADATA_FILE: ${{ steps.metadata.outputs.metadata_file }} MAX_POLLING_SECONDS: ${{ inputs.max_polling_seconds }} run: | if [[ ! -f "$SOURCE_CODE_ZIP" ]]; then echo "Error: Source code zip not found at ${SOURCE_CODE_ZIP}" exit 1 fi CODE_ZIP_SHA256=$(sha256sum "$SOURCE_CODE_ZIP" | awk '{print $1}') echo "Source code SHA256: ${CODE_ZIP_SHA256}" FOUNDRY_TOKEN=$(az account get-access-token \ --resource "https://ai.azure.com/" \ --query accessToken -o tsv) # POST /agents/{name}/versions auto-creates the agent if it doesn't # exist and adds a new version if it does, so a single call covers # both first-deploy and update scenarios (matches update-agent). HTTP_STATUS=$(curl -s -o /tmp/source_code_response.json \ -w "%{http_code}" \ -X POST \ "${PROJECT_ENDPOINT}/agents/${AGENT_NAME}/versions?api-version=2025-11-15-preview" \ -H "Authorization: Bearer ${FOUNDRY_TOKEN}" \ -H "Accept: application/json" \ -H "Foundry-Features: CodeAgents=V1Preview,HostedAgents=V1Preview" \ -H "x-ms-agent-name: ${AGENT_NAME}" \ -H "x-ms-code-zip-sha256: ${CODE_ZIP_SHA256}" \ -F "metadata=@${METADATA_FILE};type=application/json" \ -F "code=@${SOURCE_CODE_ZIP};type=application/zip;filename=${AGENT_NAME}.zip") echo "HTTP ${HTTP_STATUS}: $(cat /tmp/source_code_response.json)" if [[ "$HTTP_STATUS" -lt 200 || "$HTTP_STATUS" -ge 300 ]]; then echo "Error: Foundry data plane returned HTTP ${HTTP_STATUS}" exit 1 fi RESPONSE=$(cat /tmp/source_code_response.json) AGENT_VERSION=$(echo "$RESPONSE" | python3 -c 'import sys,json; print(json.load(sys.stdin)["version"])') echo "agent_version=${AGENT_VERSION}" >> "$GITHUB_OUTPUT" echo "Agent version resolved as ${AGENT_VERSION}" START_TIME=$(date +%s) while true; do ELAPSED=$(($(date +%s) - START_TIME)) if [[ $ELAPSED -gt $MAX_POLLING_SECONDS ]]; then echo "Error: Agent version did not reach active state within ${MAX_POLLING_SECONDS} seconds" exit 1 fi VERSION_STATUS=$(curl -s \ -X GET \ "${PROJECT_ENDPOINT}/agents/${AGENT_NAME}/versions/${AGENT_VERSION}?api-version=2025-11-15-preview" \ -H "Authorization: Bearer ${FOUNDRY_TOKEN}" \ -H "Accept: application/json" \ -H "Foundry-Features: CodeAgents=V1Preview,HostedAgents=V1Preview" \ | python3 -c 'import sys,json; data=json.load(sys.stdin); print(data.get("status", "unknown"))' 2>/dev/null) echo "Current status: ${VERSION_STATUS} (elapsed ${ELAPSED}s)" if [[ "$VERSION_STATUS" == "active" ]]; then echo "Agent version ${AGENT_VERSION} is active" break fi if [[ "$VERSION_STATUS" == "failed" ]]; then echo "Error: Agent version reached failed status" exit 1 fi sleep 5 done Building the Source-Code Artifact Before calling the source-code Hosted Agent action, create the ZIP artifact that will be passed into `source_code_zip`. source-code: name: Build source-code artifact runs-on: ubuntu-latest permissions: contents: read steps: - name: Checkout uses: actions/checkout@v6 - name: Create source-code zip artifact run: | git archive --format=zip --output=source-code.zip HEAD:src/agent-framework/responses/basic - name: Upload source-code artifact uses: actions/upload-artifact@v7 with: name: source-code path: source-code.zip Calling the Action Now that we have the action, how can we scale this across multiple workflows? We pass in the required parameters and the ZIP artifact path. - name: Update agent with source code uses: ./.github/actions/update-agent-source-code with: project_endpoint: ${{ needs.deploy-iac.outputs.project_endpoint }} # Source-code agent shares the same Foundry project as the image-based # agent; the `-src` suffix keeps them as distinct agent versions. agent_name: ${{ inputs.agent_name }}-src source_code_zip: ./.artifacts/source-code/source-code.zip model_deployment_name: ${{ needs.deploy-iac.outputs.model_deployment_name }} And just to show we can call the same action multiple times, here are two examples that do just that: Deploy (Bicep) and Deploy (Terraform). Conclusion Source-code deployments give Foundry Hosted Agents another deployment path for teams that do not want, or do not need, to package every agent as a container image. By using a .zip artifact, teams can keep a familiar source-code packaging flow while still taking advantage of the managed compute abstraction that Hosted Agents provide. The reusable GitHub Action shown in this post turns that deployment process into a repeatable CI/CD step: package the source code, post the deployment to the Foundry data plane, poll until the new version is active, and expose the resulting agent version for downstream workflow steps. This keeps the deployment flexible while fitting into existing enterprise pipeline patterns. For organizations already using container-based Hosted Agents, source-code deployments do not replace that model; they expand the options available. Choose the deployment approach that best fits how your teams package, govern, and operate their agent workloads.Round Table: Building Browser-Capable Agents with the Browser Automation Tool
Some of the most valuable work still lives inside a browser: booking a class, pulling a figure off a dashboard, filling in a portal form, gathering research across a dozen tabs. These are exactly the tasks people wish an agent could just do. On 22 July 2026 at 2:30 PM BST (7:00 PM IST), the Microsoft Foundry community is running a 40‑minute Discord round table on the Browser Automation tool : how how it helps agents complete real browser workflows, where you see risk or friction, and what samples, docs, and product improvements would help you adopt it. This is a discussion, not a slideshow. Bring your real projects : the the web workflows you'd love to hand off, and the guardrails you'd want first. Join us in the Microsoft Foundry Discord community. Please arrive at the scheduled time for a quick tech check. Event at a glance What: Microsoft Foundry Discord Community Round Table : Building Browser-Capable Agents with the Browser Automation Tool When: 22 July 2026, 2:30 PM BST / 7:00 PM IST (40 minutes) Where: https://aka.ms/foundry/discord Event link https://discord.gg/Z8JZsrP5P5?event=1527676149264679013 Format: Interactive discussion : voice and chat, live polls, and a short prioritisation exercise voice and chat, live polls, and a short prioritisation exercise Who it's for: AI engineers and developers building agents that need to act on the web Opening question we'll start with: "What browser-based task would you love an AI agent to automate for you today?" The problem: the last mile of automation still runs in a browser Most real-world workflows eventually hit a website with no clean API , such as a supplier portal, an internal admin console, a booking page, or a legacy dashboard a supplier portal, an internal admin console, a booking page, a legacy dashboard. Traditional scripting can automate these, but selectors break, pages change, and every new site means another brittle script to maintain. What developers actually want is an agent that can look at a page, decide what to do, and do it : navigate, read, click, type, and hand back a structured result navigate, read, click, type, and hand back a structured result. That's the gap the Browser Automation tool in Microsoft Foundry is built to close , and doing it responsibly and doing it responsibly, with the right safeguards, is a big part of why we want your feedback. What is the Browser Automation tool? The Browser Automation Tool (BAT) gives Foundry agents the ability to drive a real browser to complete web workflows. It's available as an MCP tool, and it uses Playwright Workspaces , a generally available, cloud-scale service, a generally available, cloud-scale service : navigating, clicking at coordinates, typing, and applying filters as its headless browser infrastructure. When an agent gets a request, Foundry spins up an isolated, sandboxed browser session per interaction, so each run is private and segregated. How agents actually interact with a page BAT runs a perception–action loop. The model receives the current state of the page (including screenshots), decides the next action, and BAT executes it in the sandbox using Playwright and real oversight navigating, clicking at coordinates, typing, applying filters. After each action, BAT captures the updated state and sends it back to the model, repeating until the goal is met or the user stops. Because the model can parse HTML into a DOM, it can reason about the page rather than follow a fixed script. It also supports multi-turn conversations, so you can refine a request mid-flow to complete form-filling or scraping scenarios. Built for real use : watch the automation happen in real time for debugging. and real oversight Live View : a human-in-the-loop override for ambiguous or sensitive steps. watch the automation happen in real time for debugging. Take Control : each interaction gets its own sandboxed browser. a human-in-the-loop override for ambiguous or sensitive steps. Isolated sessions : for reliability, optimisation, and audit. each interaction gets its own sandboxed browser. Built-in observability : for internal systems (private preview). for reliability, optimisation, and audit. Private website browsing : Python, C#, JavaScript, Java, and the REST API. for internal systems (private preview). Broad SDK support , and the agent can make mistakes or be misled by malicious page content Python, C#, JavaScript, Java, and the REST API. A word on responsible use. BAT is powerful precisely because an AI can use credentials you share with it to reach email, financial, enterprise, or social accounts , watching for and the agent can make mistakes or be misled by malicious page content. You're responsible for reviewing your applications, scoping which credentials you provide, and adding your own mitigations. See the Foundry Agent Service transparency note. This is exactly the kind of trade-off we want to talk through together. Example scenario A user asks: "Report the year-to-date percent change of Microsoft's stock price." The agent navigates to a finance site, enters MSFT in the search bar, opens the stock page, clicks the YTD view on the chart, reads the value, and returns a clean, structured answer ; that's the no bespoke scraper, no hard-coded selectors, and a full trace of what it did. Discussion prompt: "Where would browser automation fit into your current projects or workflows?" How setup works (the short version) You'll want to understand the wiring before you scale, so it's worth a look ahead of the session. There are two moving parts: Create a Playwright Workspace in the Azure portal, enable the access token auth method, and grab the wss:// browser endpoint. Give your project identity a Contributor (or custom) role on the workspace. Connect the tool in Foundry under Build > Tools: create a toolbox, add Browser Automation, point it at your Playwright workspace and auth type, and publish. Copy the Project connection ID from the tool's details page : how agents interact with the web, example use cases, and responsible use. that's the BROWSER_CONNECTION_ID in your code. What we'll cover in the 40 minutes Welcome & opening question (0:00–0:03) : navigate, gather, interact, and return a structured result, end to end. the browser task you'd most love to automate. What is Browser Automation (0:03–0:07) : the workflows you're building, public vs. internal targets, and where you'd pick automation over scripting. how agents interact with the web, example use cases, and responsible use. Scenario walkthrough (0:07–0:12) : what agents may do autonomously, what needs approval, and the observability and enterprise safeguards you'd require. navigate, gather, interact, return a structured result : your biggest adoption blockers, missing docs, and the SDK samples and demos you'd prioritise. end to end. Use cases & opportunities (0:12–0:22) : vote live on top use cases, challenges, and feature requests. the workflows you're building, public vs. internal targets, and where you'd pick automation over scripting. Trust, security & governance (0:22–0:31) , and which would benefit most from a capable agent. what agents may do autonomously, what needs approval, and the observability and enterprise safeguards you'd require. Developer experience feedback (0:31–0:36) ; which should always require approval. your biggest adoption blockers, missing docs, and the SDK samples and demos you'd prioritise. Prioritisation & next steps (0:36–0:40) , sometimes with credentials, vote live on top use cases, challenges, and feature requests. Come prepared to talk about The browser-based workflows you're building today : navigation, data gathering, form filling, and research, via an MCP tool powered by Playwright Workspaces. and which would benefit most from a capable agent. Whether your scenarios target public websites, internal systems, or both. Why you'd choose browser automation over traditional scripting. Which actions you'd let an agent perform autonomously : a perception-action loop with screenshots and DOM parsing handles pages that break brittle scripts. and which should always require approval. The observability, audit, and enterprise safeguards you'd expect before running this in production. The examples, samples, and tutorials that would help you get started fastest. Responsible and secure by design Because BAT lets an agent take real actions on live websites : isolated sessions, Live View, Take Control, and observability for reliability and audit. sometimes with credentials : scope credentials carefully and add your own mitigations; the tool is powerful and in preview. governance is a first-class part of the conversation, not a footnote. Isolated per-session sandboxes, Live View, Take Control human-in-the-loop, and built-in observability are there so you can see, pause, and audit what an agent does. Bring your trust concerns, required guardrails, and governance requirements: they directly shape the roadmap. Note: the Browser Automation tool is in preview; APIs and capabilities may change, and it isn't recommended for production workloads yet. Key takeaways Browser Automation lets Foundry agents complete real web workflows : this round table feeds directly into the engineering and product teams. navigation, data gathering, form filling, research , and arrive on time for the tech check. via an MCP tool powered by Playwright Workspaces. Agents reason, not just replay , and browser-capable agents are how we cross it. a perception–action loop with screenshots and DOM parsing handles pages that break brittle scripts. Oversight is built in: isolated sessions, Live View, Take Control, and observability for reliability and audit. Responsibility is shared: scope credentials carefully and add your own mitigations; the tool is powerful and in preview. Your feedback shapes the product: this round table feeds directly into the engineering and product teams. Save your spot Add it to your calendar: 22 July 2026, 2:30 PM BST / 7:00 PM IST, and arrive on time for the tech check. Join the community: https://aka.ms/foundry/discord Prep with the sample: explore the browser automation sample in foundry-samples. Read the docs: Automate browser tasks with Foundry agents and the hosted-agent quickstart. Event registration link https://discord.gg/Z8JZsrP5P5?event=1527676149264679013 The last mile of automation still runs in a browser, and browser-capable agents are how we cross it. Come tell us what you'd automate, where you'd draw the line, and what you'd need to trust it in production. See you on 22 July.Beyond text: Returning images and interactive apps from MCP servers
The Model Context Protocol (MCP) is becoming a richer foundation for agent experiences. Though most servers return plain text from their tool calls, MCP servers can also return binary results and provide interactive apps in clients that support those features, like VS Code. In this post, I'll use both capabilities to build an MCP server that searches a collection of nature photos with natural language, lets the model inspect the matching images, and presents selected results in an interactive gallery. The same approach can be adapted to product catalogs, digital asset managers, photo archives, and other multimedia libraries. Searching the image library Let's start with the search experience from a user's perspective, then dive into the code behind it. After connecting VS Code to the deployed MCP server, I can ask a question in GitHub Copilot about the images: Find landscape photos that show dramatic terrain and water. Show me the strongest options for a nature gallery. The GitHub Copilot agent realizes that it can use the image search MCP tool to answer that question. Here's what it looks like in the chat interface: The tool results include rendered thumbnails. I can click a thumbnail to inspect it directly in VS Code, much like a file in the workspace, while the Copilot agent can review both the image binary data and their textual descriptions. Behind the scenes, the agent called the image_search tool with these arguments: { "query": "dramatic natural landscapes with mountains and water", "max_results": 5 } The tool call returned a mix of binary files and structured data: a thumbnail for each matching image, plus JSON containing its filename, display name, and generated description. The thumbnails let a multimodal model inspect the actual pixels, while the structured content gives the agent compact metadata it can reference in later tool calls. { "results": [ { "filename": "Picture1.jpg", "display_name": "Picture1.jpg", "description": "A clear mountain lake surrounded by pine forest and steep rocky peaks." }, ...] } Returning images from MCP tools Now let's look at the code powering that tool call. I built the server with FastMCP, a popular Python framework for writing MCP servers. I declare each tool by decorating a function with mcp.tool() and annotating its arguments with types and helpful descriptions. FastMCP converts the function signature into a JSON Schema that helps GitHub Copilot decide when and how to call image_search : @mcp.tool(annotations={"readOnlyHint": True}) async def image_search( query: Annotated[ str, "Text description of images to find (e.g., 'sunlit mountain lake')" ], max_results: Annotated[int, "Maximum number of images to return (1-20)"] = 5) -> ToolResult: """ Search for images matching a natural language query. Returns the image data and descriptions. """ Inside the function, I use Azure AI Search to perform hybrid retrieval, combining the text query with its vector embedding. The target index contains multimodal image embeddings and LLM-generated descriptions. Then I retrieve the image from Azure Blob Storage and resize it to a thumbnail. The tool returns both the binary image data for the thumbnails and structured metadata with image details. results = await search_client.search(search_text=query, top=max_results, vector_queries=[VectorizableTextQuery(k_nearest_neighbors=max_results, fields="embedding", text=query)], select=["metadata_storage_path", "verbalized_image"]) blob_service_client = get_blob_service_client() files: list[File] = [] image_results: list[dict[str, str]] = [] async for result in results: url = result["metadata_storage_path"] description = result.get("verbalized_image") container_name, blob_name = get_blob_reference_from_url(url) blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name) stream = await blob_client.download_blob() image_bytes = await stream.readall() image_format = get_image_format(url) display_name = os.path.basename(blob_name) file_basename = Path(display_name).stem thumbnail_bytes = resize_image_bytes(image_bytes, image_format) files.append(File(data=thumbnail_bytes, format=image_format, name=file_basename)) image_results.append({"filename": blob_name, "display_name": display_name, "description": description}) return ToolResult( content=files, structured_content={ "query": query, "results": image_results, }, ) Displaying selected images Finding the right images is only the first half of the experience. Once the agent has review the thumbnails and their generated descriptions, it needs a better way to present its favorite selected images to the user. That is where MCP apps come in. An MCP app renders an interactive webpage inside a sandboxed iframe in the MCP client. For this server, the app is a small, JavaScript-powered carousel for browsing the selected images. GitHub Copilot calls the display_image_files tool when it wants to render the carousel app: Returning apps from MCP tools Let's check out the code that powers that MCP carousel app. An app is associated with a tool, so I once again decorate a Python function with mcp.tool() . This time, I pass an AppConfig that points to the image viewer's HTML resource. @mcp.tool( app=AppConfig(resource_uri=IMAGE_VIEW_URI), annotations={"readOnlyHint": True}, ) async def display_image_files( filenames: Annotated[list[str], "List of image filenames to retrieve and display in a carousel."], descriptions: Annotated[list[str], "Image descriptions, in the same order as filenames."] ) -> ToolResult: """Fetch images by filename and render in carousel with filenames, descriptions, and file details.""" Inside the function, I fetch the selected images from Azure Blob Storage by filename, then return both the binary image data and structured content describing each image—its filename, generated description, MIME type, dimensions, format, and size. blob_service_client = get_blob_service_client() image_blocks: list[types.ImageContent] = [] image_results: list[dict[str, str | int]] = [] for image_index, filename in enumerate(filenames): blob_client = blob_service_client.get_blob_client(container=IMAGE_CONTAINER_NAME, blob=filename) stream = await blob_client.download_blob() image_bytes = await stream.readall() mime_type = get_image_mime_type(filename) with Image.open(io.BytesIO(image_bytes)) as image: width, height = image.size image_format = image.format image_blocks.append(types.ImageContent( type="image", data=base64.b64encode(image_bytes).decode("utf-8"), mimeType=mime_type)) image_results.append( { "filename": filename, "description": descriptions[image_index], "mimeType": mime_type, "width": width, "height": height, "format": image_format, "sizeBytes": len(image_bytes), } ) return ToolResult( content=image_blocks, structured_content={ "images": image_results, }, ) Next, I define the resource that serves the image viewer HTML page. I decorate a Python function with @mcp.resource , assign it a ui:// URL that is unique to the MCP server, and use its Content Security Policy (CSP) to declare which external domains the app may load resources from: @mcp.resource(IMAGE_VIEW_URI, app=AppConfig(csp=ResourceCSP(resource_domains=["https://unpkg.com"]))) def image_view() -> str: """Render images returned by display_image_files as an MCP App.""" return load_image_viewer_html() The final piece is the HTML that renders inside the app's iframe. This small page imports ext-apps, a JavaScript package that manages bidirectional communication with the MCP client. The JavaScript creates an App instance, defines the ontoolresult callback, and connects the app. That callback receives images from the tool result and renders them in the carousel. MCP apps can also send messages back to the host, although this read-only viewer does not need to. <!DOCTYPE html> <html lang="en"> <body> <div id="carousel"> <button id="prev" type="button" aria-label="Previous">‹</button> <div id="frame"></div> <button id="next" type="button" aria-label="Next">›</button> <span id="counter" aria-live="polite"></span> </div> <script type="module"> import { App } from "https://unpkg.com/@modelcontextprotocol/ext-apps@0.4.0/app-with-deps"; const app = new App({ name: "Image Viewer", version: "1.0.0", }); let images = []; let index = 0; const frame = document.getElementById("frame"); const prevBtn = document.getElementById("prev"); const nextBtn = document.getElementById("next"); const counter = document.getElementById("counter"); function show(i) { index = i; const img = images[index]; frame.innerHTML = ""; const el = document.createElement("img"); el.src = `data:${img.mimeType || "image/jpeg"};base64,${img.data}`; el.alt = "Blob image"; frame.appendChild(el); prevBtn.disabled = index === 0; nextBtn.disabled = index === images.length - 1; counter.textContent = images.length > 1 ? `${index + 1} / ${images.length}` : ""; } prevBtn.addEventListener("click", () => { if (index > 0) { show(index - 1); } }); nextBtn.addEventListener("click", () => { if (index < images.length - 1) { show(index + 1); } }); app.ontoolresult = ({ content }) => { images = (content || []).filter((block) => block.type === "image"); if (images.length > 0) { show(0); } }; await app.connect(); </script> </body> </html> Try it yourself! The full MCP server code is available in Azure-Samples/image-search-aisearch, along with a minimal image search website and an Azure AI Search indexing pipeline. The indexer uses an Azure OpenAI model to describe each image and Azure AI Vision to create multimodal embeddings. The repository includes a sample nature dataset, but you can replace it with any image collection. Here are more ways you could extend it it: Support more media types: add transcript search and a video or audio player app, while keeping the same search-then-display tool pattern. Enrich the metadata: index dates, locations, creators, accessibility text, or domain-specific tags alongside generated descriptions and embeddings. Optimize token consumption: images require many tokens, so returning too many thumbnails can quickly consume the model's context window. Experiment with smaller previews, higher compression, metadata-only search results, or a two-stage retrieval flow. Add authentication: many media libraries contain private or licensed assets. You can add key-based authentication or OAuth with the FastMCP auth providers, as I described in the MCP auth livestream. Once search results can carry both structured metadata and real media, an agent can do more than locate files: it can compare, curate, and present them in the same conversation. I hope you'll try the sample with a multimedia collection of your own!
