Give your Copilot agents real tools, without hand-wiring MCP
You’re running agents in the GitHub Copilot app - maybe three at once, each on its own branch, each working a task while you steer. It’s a good way to work, right up until an agent needs to reach outside your repo. It needs to read a SharePoint library, file a Salesforce record, or check an Outlook calendar, and suddenly it’s blind. An agent is only as capable as the tools you hand it. The usual way to hand it one of those tools is to wire up a Model Context Protocol (MCP) server by hand: find the endpoint, paste it into config, add an auth header, keep the token alive, and hope you got the casing right. Then your teammate does the same thing on their machine, and the next person after that. The MCP Connectors canvas removes the hand-wiring. It’s a plugin for the GitHub Copilot app: it lists the hosted MCP servers already published in your Azure Connector Namespace, and you connect one to Copilot by selecting it - no URL, no header, no local proxy. Where the servers come from Connector Namespace is the managed Azure service on the other end. It provides a list of curated MCP Servers that you can use to quickly create and manage MCPs connections. It also does the work you’d rather not: it stores and rotates the credentials, applies retry and throttling policies, and scales the server. Your machine never holds a raw secret related to the end systems that MCP touches, and access is governed by access policies, allowing fine grained control on who can use the MCP servers. So, when you connect a server from the canvas, you’re not standing up infrastructure. The server already exists, already authenticated, already managed. You’re pointing Copilot at it. Prerequisites GitHub Copilot app with canvas extension support. An Azure subscription with permission to cerate and view the namespace and to create connections and hosted managed MCP server configurations. Connector Namespace is in preview, and availability varies by region. Install From the GitHub Copilot app, add the following prompt. Install connector-namespaces v1.2.0 for my user account, reload it, verify it is running, then open the MCP Connectors canvas. Open the Connector Namespace canvas in a new session Once you install if you need to open the Connector Namespace in a new session, just request it in a prompt: Open the MCP Connectors canvas. Connect an MCP server Open the MCP Connectors canvas. Select Sign in to Azure, then choose a subscription and Connector Namespace. Browse or search the MCP servers grouped under Microsoft and Partners. Select Connect and complete the connector’s separate authentication or consent flow. Confirm the server appears under My MCPs. Restart GitHub Copilot so a new session loads the added tools. ℹ️Note That last step matters more than it look. Copilot loads an agent’s tools when a session starts, so a server you add mid-session shows up in the next one, not the one you’re in. Manage connected servers My MCPs shows the servers connected to Copilot. Sandbox opens a connected server in the Connector Namespace playground, so you can try a tool call yourself before you let an agent lean on it. Disconnect removes the MCP registration from your MCP configuration. The MCP is still available in Connector namespace to be added again later. Also if you are sharing the connector namespace with other developers, this only remove the your local configuration. Connect adds configured MCP connectors that are not registered locally yet to your local MCP configuration. Delete from namespace… removes the MCP registration from your MCP configuration and deletes the MCP connection from your connector namespace. If you are sharing the connector namespace with other developers this will affect any developer using this MCP connector – be careful when using this. Switch namespace switches the active subscription or namespace - useful when you keep dev and production connections apart. How it works and security The extension registers each server directly in Copilot’s user-scoped MCP configuration. There’s no local MCP proxy sitting in the tool path between the agent and the server. The canvas itself runs on loopback. Your Azure tokens stay in process memory, the selected namespace coordinates may be cached locally, and gateway credentials are sent only to the server’s configured HTTPS endpoints. The connector’s own secrets never reach your machine. That’s the namespace’s job, which is also why an admin can rotate or revoke a connection in one place instead of chasing down every developer’s config. Where can I use my connected MCP servers You manage the servers from the canvas, but they don’t only work there. The canvas writes to Copilot’s user-scoped MCP configuration at ~/.copilot/mcp-config.json, and the GitHub Copilot app is built on GitHub Copilot CLI. Connect a server once and it’s available to both. Agent sessions in the app. Any session on your machine picks the server up, whatever repository or branch it’s working on, because the config is user-scoped rather than per-project. Copilot CLI. The same server shows up in the terminal. Run copilot mcp list to confirm it, or /mcp show SERVER-NAME in an interactive session to see the tools it exposes. Two things to know: Tools load when a session starts, so connect first and start the session second. A project-level .mcp.json or .github/mcp.json takes precedence over your user config when the names collide, which is what you want if a repository pins its own version of a server. VS Code is the exception. Copilot Chat there reads .vscode/mcp.json, a separate file the CLI doesn’t use, so a server you connect in the canvas won’t appear in VS Code until you configure it there too. Where this all sits today The GitHub Copilot app went generally available in June 2026, and canvases shipped with it, so neither is a preview feature. The MCP Connectors canvas is a published extension you install at personal scope. You get the version we ship and tested, not one an agent generates on the fly from a prompt. Two caveats. GitHub doesn't publish a compatibility guarantee for the canvas extension format, so an app update may need a matching update from us. Re-run the install commands above to pick it up if you see any issues. On the Azure side, Connector Namespace is still in preview. Check the documentation for more details on this service outside the GitHub Copilot App support. If you're on Copilot Business or Enterprise, check with your admin that the GitHub Copilot app policy is enabled. It's on by default, and it's separate from the Copilot CLI policy. Try it Install the plugin, point it at a namespace, and connect one server your agents keep reaching for: the SharePoint library, the CRM, whatever it is. Then watch an agent actually use it in the next session. Tell me where the canvas saved you a config headache and where it got in your way. That’s the feedback we act on before general availability.Set Up Plaud Note Pro with Microsoft Foundry
Prerequisites Riffado, up and running: follow the setup guide in the official Riffado repository to get it going with Docker Compose. A Microsoft Foundry (formerly Azure AI Foundry) resource, with the models you want deployed; in my case, whisper for transcription and o3-mini for summaries. A Plaud device, or any audio recordings you can import into Riffado. Once Riffado is up, head to the Settings page > Providers > Add Provider, and select Custom. This is where the Azure details will go. Why "OpenAI-compatible" isn’t one thing on Microsoft Foundry Azure AI Foundry exposes two different API surfaces on the same resource, and which one serves your model depends on the model: Surface Path shape Serves OpenAI-compatible? v1 route /openai/v1/… gpt-4o-transcribe, gpt-4o-mini-transcribe, chat models, embeddings Yes: Bearer auth, model in the body, no api-version needed Classic route /openai/deployments/{name}/… Whisper (and other legacy audio) No: deployment name lives in the URL, and ?api-version= is mandatory A generic OpenAI client (Riffado's included) can only speak the first dialect. It has nowhere to put a deployment name in the path and no way to append a query parameter. That single fact drives everything below. Part 1 - Transcription Whisper and the DeploymentNotFound mystery Symptom My very first transcription attempt in Riffado failed with 404 Resource not found. Off to a flying start. Configured provider: base URL https://<resource>.services.ai.azure.com, model whisper. Dead end #1: the missing path The first bug was mine: the base URL had no path. Riffado's OpenAI client appends /audio/transcriptions to whatever you give it, so requests were hitting https://<resource>…/audio/transcriptions, a path that doesn't exist on the resource at all. Fixing the base URL to end in /openai/v1 got us to a more interesting error: POST /openai/v1/audio/transcriptions · model=whisper {"error":{"code":"DeploymentNotFound","message":"The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again."}} Dead end #2: catalog ≠ deployment Worth checking before anything else: selecting a model in the Foundry catalog is not deploying it. GET /openai/v1/models lists everything you could deploy; only Deployments → Deploy model creates an endpoint that answers. If you get DeploymentNotFound, first confirm a deployment actually exists (the listing below requires only the API key): enumerate real deployments (classic control-plane, key auth) curl -s -H "api-key: $KEY" \ "https://<resource>.openai.azure.com/openai/deployments?api-version=2023-03-15-preview" # → {"data":[{"id":"whisper","model":"whisper","status":"succeeded",…}]} The actual cause Here is the part that nearly drove me mad: the deployment existed and was succeeded, yet the v1 route still said DeploymentNotFound. Because Whisper deployments are not served on the v1 route at all. They only answer on the classic path. Verified side by side with the same tiny WAV file: Request Result POST /openai/v1/audio/transcriptions · model=whisper · Bearer 404 DeploymentNotFound POST /openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01 · Bearer 200 {"text":"you"} Same classic path, without ?api-version= 404 Resource not found Three constraints, then: Whisper needs the classic path; the classic path needs api-version; Riffado can send neither. One piece of good news hiding in the table: the classic route accepts Authorization: Bearer, not just Azure's api-key header, so the shim doesn't have to touch auth at all. The fix: a Caddy shim Drop a stock caddy:2-alpine container into the Compose network. Riffado points at it as if it were OpenAI; the shim rewrites the path, injects api-version, and proxies to Azure. The Bearer header passes through untouched. azure-shim.Caddyfile { admin off auto_https off } :80 { @transcribe path /v1/audio/transcriptions /audio/transcriptions handle @transcribe { rewrite * /openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01 reverse_proxy https://<resource>.services.ai.azure.com { header_up Host <resource>.services.ai.azure.com } } handle { respond "azure-shim ok" 200 } } docker-compose.yml (added service) azure-shim: image: caddy:2-alpine restart: unless-stopped volumes: - ./azure-shim.Caddyfile:/etc/caddy/Caddyfile:ro Riffado's provider settings become: Field Value Base URL http://azure-shim/v1 Model whisper (must equal the deployment name) API key the Azure resource key (forwarded as Bearer) Verified From inside the Riffado container: POST http://azure-shim/v1/audio/transcriptions → 200 {"text":"…"}. Transcription works end-to-end in the UI. Part 2 · Summaries & titles o3-mini and the empty answer Symptom The summary button showed "An unexpected error occurred." The container logs were more honest: riffado-app logs Error generating title: TypeError: undefined is not an object (evaluating 'C.choices[0]') Riffado calls chat/completions and reads choices[0] without checking whether the response was an error. So anything the API refuses becomes "an unexpected error." What was it refusing? Cause 1: reasoning models reject the classic knobs o3-mini belongs to Azure/OpenAI's o-series reasoning models, which hard-reject parameters every classic chat client sends. Riffado sends temperature: 0.7 and max_tokens: 50 for titles (0.5 / 2000 for summaries), and o3-mini answers: POST /openai/v1/chat/completions · model=o3-mini HTTP 400 {"error":{"message":"Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.", …}} # and with max_tokens fixed: HTTP 400 {"error":{"message":"Unsupported parameter: 'temperature' is not supported with this model.", …}} Cause 2: reasoning tokens starve the output Stripping the bad params gets you to 200, and then comes a subtler failure, my personal favourite of this whole saga. Reasoning models spend completion tokens on internal "thinking" before emitting a single visible character. Riffado's 50-token title budget is consumed entirely by reasoning, and the reply comes back syntactically valid and empty: max_completion_tokens reasoning_effort finish_reason content 50 not set length "" (all 50 spent reasoning) 2000 not set stop "Q3 Budget Planning Strategy Meeting" 2000 low stop same, less reasoning overhead The fix: a Node shim that rewrites the request body Caddy can rewrite paths but not JSON bodies, so this shim is ~60 lines of dependency-free Node on node:20-alpine. Per request it: converts max_tokens → max_completion_tokens, strips temperature / top_p / penalties, floors the token budget at 4000, sets reasoning_effort: "low", maps /v1/* → /openai/v1/*, and forwards to the Azure resource. o3-shim.js const http = require('http'); const https = require('https'); const UPSTREAM_HOST = '<resource>.services.ai.azure.com'; // Params o-series reasoning models reject on chat/completions. const STRIP = ['temperature','top_p','presence_penalty', 'frequency_penalty','logprobs','top_logprobs']; const server = http.createServer((req, res) => { const chunks = []; req.on('data', c => chunks.push(c)); req.on('end', () => { let body = Buffer.concat(chunks); // Riffado's base_url is http://o3-shim/v1 → map to Azure's /openai/v1 let path = req.url; if (path.startsWith('/v1/')) path = '/openai' + path; const ct = (req.headers['content-type'] || '').toLowerCase(); if (ct.includes('application/json') && body.length) { try { const j = JSON.parse(body.toString('utf8')); if (j && typeof j === 'object' && !Array.isArray(j)) { if ('max_tokens' in j) { if (!('max_completion_tokens' in j)) j.max_completion_tokens = j.max_tokens; delete j.max_tokens; } // Reasoning spends tokens before any visible output; small // budgets (Riffado sends 50 for titles) return empty strings. if (Array.isArray(j.messages)) { j.max_completion_tokens = Math.max(Number(j.max_completion_tokens) || 0, 4000); if (!('reasoning_effort' in j)) j.reasoning_effort = 'low'; } for (const k of STRIP) delete j[k]; body = Buffer.from(JSON.stringify(j)); } } catch (_) { /* not JSON - forward untouched */ } } const headers = { ...req.headers, host: UPSTREAM_HOST, 'content-length': Buffer.byteLength(body) }; const up = https.request( { host: UPSTREAM_HOST, port: 443, method: req.method, path, headers }, upRes => { res.writeHead(upRes.statusCode, upRes.headers); upRes.pipe(res); } ); up.on('error', e => { res.writeHead(502, {'content-type':'application/json'}); res.end(JSON.stringify({error:{message:'o3-shim upstream error: '+e.message}})); }); up.end(body); }); }); server.listen(80, () => console.log('o3-shim listening on :80')); docker-compose.yml (added service) o3-shim: image: node:20-alpine restart: unless-stopped working_dir: /app command: ["node", "/app/o3-shim.js"] volumes: - ./o3-shim.js:/app/o3-shim.js:ro Add a second provider in Riffado (base URL http://o3-shim/v1, model o3-mini, the resource's API key) and set it as the default enhancement provider (summaries/titles), keeping the Whisper one as default for transcription. Riffado's exact title request (temperature: 0.7, max_tokens: 50) through the shim → 200, finish_reason: stop, real title text. A full meeting-transcript summary returns structured key points and action items. The final shape Reading it left to right: Riffado never talks to Azure directly. Transcription requests pass through azure-shim, a stock Caddy container that rewrites each request onto Whisper's classic deployment path and injects the mandatory api-version parameter. Summary and title requests pass through o3-shim, a tiny Node server that rewrites the request body into the shape o3-mini accepts and floors the token budget so the model's internal reasoning cannot starve the actual answer. As far as Riffado is concerned, it is simply talking to two ordinary OpenAI providers. Both shims live on the Compose network only; nothing is exposed publicly. Riffado is unmodified. Verification checklist Each layer, testable in isolation. Run these before blaming the app: smoke tests # 1. Key + resource alive? (v1 models listing, Bearer auth) curl -s -H "Authorization: Bearer $KEY" \ https://<resource>.services.ai.azure.com/openai/v1/models | head -c 200 # 2. Whisper answers on the classic path? curl -s -H "Authorization: Bearer $KEY" -F file=@test.wav \ "https://<resource>.services.ai.azure.com/openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01" # 3. Shim translates correctly? (from inside the compose network) docker exec riffado-app node -e "fetch('http://azure-shim/') .then(r=>r.text()).then(console.log)" # 4. o3-mini via shim, sending the params Riffado sends? # (temperature + max_tokens:50; the shim must absorb both) If you'd rather not run shims Both shims exist because of the specific models chosen. Pick models that live natively on the v1 route and Riffado connects directly, with base URL https://<resource>.services.ai.azure.com/openai/v1 and zero extra containers: Transcription: deploy gpt-4o-mini-transcribe (or gpt-4o-transcribe) instead of Whisper. Summaries: deploy a non-reasoning chat model such as gpt-4o-mini, which happily accepts temperature and max_tokens. The shim approach earns its keep when you're standardized on specific models (Whisper's transcription quality, o3-mini's reasoning), or when you want a control point to add logging, retries, or budget caps later. For reference, this is what the finished setup looks like on Riffado's side. Each shim is registered as a plain Custom provider. Here is the whisper provider pointing at azure-shim, with Use for transcription ticked: And once both are saved, they sit side by side in the providers list, whisper tagged for transcription and o3-mini tagged for enhancement: A quick look at the Foundry portal In the Microsoft Foundry portal, head over to Models > AI Services and you will find a pleasant surprise: fifteen AI service models already deployed and ready to use, covering the Azure Speech family (including Voice Live and Speech to Text), Azure Translator, Azure Language, and Content Understanding: You can of course deploy another model for this, but the pre-deployed ones are a handy cost-saving option. Click on the Azure Speech – Voice Live radio button and you will be shown the Base URL and API Key, which you can then paste into the provider settings on Riffado's Settings page. A quick note on cost: these services are not free. They are billed pay-as-you-go based on usage. Azure Speech transcription is charged per audio hour, and Voice Live pricing is tiered by the model you choose. The free tier does include a monthly allowance, though. Check the Azure Speech pricing page before committing. And if you would rather deploy a dedicated transcription model such as whisper, Foundry gives you the flexibility to do just that. Open the model page in the catalogue, click Deploy, and go with Default settings unless you need custom quotas or guardrails: Let's test the setup On your Plaud device, just tap to start recording. The little LED bars light up to show it is listening: Or skip the device entirely and upload an audio file straight into Riffado using the Upload Audio button. Either way, the recording lands on the Recordings page; hit Transcribe and let the spinner do its thing: As you can see below, whisper, the transcription model we deployed earlier, even managed to transcribe a recording in Malay without a hitch. My 3:32 test clip came back as 186 words of clean Malay, with the language correctly detected and tagged: I have also set o3-mini as the enhancement provider, and it enhanced the transcription with a proper summary, key points, and title as well! The Meeting Notes-style summary came straight out of o3-mini through the shim, with zero manual prompting. Wrapping up What started as a TikTok-fuelled impulse buy nearly killed off by subscription pricing ended up as a fully self-hosted pipeline: Plaud for recording, Riffado as the interface, and Microsoft Foundry serving whisper and o3-mini behind two tiny shims. The total extra infrastructure came to two containers and roughly sixty lines of code, and not a single monthly subscription in sight. If you try this setup and run into a failure mode I have not covered here, do share it in the comments. Half the fun is in the debugging.Microsoft Agent Framework Multi-Agent Workflow Architecture for Automated Kubernetes Assessments
Why this system generates tests (design rationale) This project does not generate tests just to "check code." It generates tests because, in a Kubernetes learning game, the test suite is the grading contract. The design goals are: Scale content creation: instructors should not hand-author every task and checker. Keep grading objective: student success is measured against Kubernetes API state, not subjective review. Avoid fragile tasks: generated tasks must survive empty/wrong cluster states without crashing. Make failures repairable: when checks break, the system should patch and re-validate automatically. So the pipeline generates a full task package (setup, answer, check, cleanup) where tests define exactly what "correct" means. Lifecycle: from concept to production-ready grader Reader mental model: this is a content compiler with validation stages, not a single chat completion. Phase 1: Pedagogical intent -> structured concept The Idea Agent converts a topic into a constrained concept object (objective, progression, task IDs, difficulty variants). Memory rules block duplicates and previously failed concepts. Phase 2: Concept -> executable grading package The Generator Agent turns that concept into files: student instructions (instruction.md) learning material (concept.md) parameter source (session.json) setup and answer manifests (setup.template.yaml, answer.template.yaml) deterministic pytest flow (test_01 ... test_06) At this point, output is still untrusted draft content. Phase 3: Structural correctness gate Deterministic validation checks file presence, syntax, JSON/YAML shape, and template correctness. This catches basic integrity issues before cluster execution. Phase 4: Behavioral correctness gate (real cluster) Pytest executes against Kubernetes and verifies runtime behavior using real kubectl-derived state. This proves that generated checks actually evaluate cluster resources as intended. Phase 5: Anti-false-positive gate (skip-answer mode) The same suite runs with SKIP_ANSWER_TESTS=True to verify grader integrity: answer deployment is skipped test_05_check.py must fail If it still passes, the grader is invalid (it would accept wrong student submissions). Phase 6: Self-healing repair loop On any failure, deterministic error logs are fed into the Fixer Agent, which patches only broken files. The workflow then re-enters validation + test gates. Phase 7: Finalization If all gates pass -> task is kept as production-ready content. If retries are exhausted -> task is moved to unsuccessful/ with FAILURE_REPORT.txt for human triage. This lifecycle explains the core architecture decision: LLMs generate candidate graders, deterministic execution certifies them. Concrete generated sample (what the pipeline actually produces) Below is a representative generated task for topic: ConfigMap Environment Variable Injection. Generated directory layout tests/game01/050_configmap_env_injection/ ├── __init__.py ├── instruction.md ├── concept.md ├── session.json ├── setup.template.yaml ├── answer.template.yaml ├── test_01_setup.py ├── test_02_ready.py ├── test_03_answer.py ├── test_05_check.py └── test_06_cleanup.py session.json (runtime variables) { "namespace": "{{random_name()}}{{random_number(100,999)}}{{student_id()}}", "configmap_name": "{{random_name()}}", "deployment_name": "{{random_name()}}", "container_name": "app", "env_key": "APP_MODE", "env_value": "production" } Why this exists: task values are randomized per student/session, so tests verify behavior by variable contract instead of hardcoded names. setup.template.yaml (baseline state only) apiVersion: v1 kind: Namespace metadata: name: {{ namespace }} Why this exists: setup should create prerequisites only. It must not accidentally include the final answer. answer.template.yaml (expected correct solution) apiVersion: v1 kind: ConfigMap metadata: name: {{ configmap_name }} namespace: {{ namespace }} data: {{ env_key }}: "{{ env_value }}" --- apiVersion: apps/v1 kind: Deployment metadata: name: {{ deployment_name }} namespace: {{ namespace }} spec: replicas: 1 selector: matchLabels: app: env-demo template: metadata: labels: app: env-demo spec: containers: - name: {{ container_name }} image: nginx:latest env: - name: {{ env_key }} valueFrom: configMapKeyRef: name: {{ configmap_name }} key: {{ env_key }} Why this exists: defines canonical "correct cluster state" that graders must detect. test_02_ready.py (wait for setup resources) import json import time from tests.helper.kubectrl_helper import build_kube_config, run_kubectl_command class TestReady: def test_001_namespace_active(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) time.sleep(2) result = run_kubectl_command( kube_config, f"kubectl get namespace {json_input['namespace']} -o json", ) data = json.loads(result) assert data.get("status", {}).get("phase") == "Active" Why this matters: validates setup-stage readiness only. It should not check answer resources yet. test_05_check.py (student grading contract) import json from tests.helper.kubectrl_helper import build_kube_config, run_kubectl_command class TestCheck: def test_001_configmap_key_exists(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) result = run_kubectl_command( kube_config, f"kubectl get configmap {json_input['configmap_name']} -n {json_input['namespace']} -o json", ) data = json.loads(result) assert data["data"][json_input["env_key"]] == json_input["env_value"] def test_002_deployment_uses_configmap_env(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) result = run_kubectl_command( kube_config, f"kubectl get deployment {json_input['deployment_name']} -n {json_input['namespace']} -o json", ) data = json.loads(result) env = data["spec"]["template"]["spec"]["containers"][0].get("env", []) matched = [ e for e in env if e.get("name") == json_input["env_key"] and e.get("valueFrom", {}).get("configMapKeyRef", {}).get("name") == json_input["configmap_name"] and e.get("valueFrom", {}).get("configMapKeyRef", {}).get("key") == json_input["env_key"] ] assert matched, "Deployment container must consume env var from ConfigMap key" Why this matters: this is the real grading logic. If a student deploys wrong resource wiring, this test fails with explicit reason. Why skip-answer validation is essential for this sample When SKIP_ANSWER_TESTS=True, answer deployment is skipped. In this mode: test_03_answer.py should be skipped test_05_check.py must fail (ConfigMap/Deployment wiring is absent) If test_05_check.py still passes, the grader is broken (false positive), and the workflow routes to Fixer. Rule Builder Workflow Flowchart Multi-graph architecture views 1) Control-plane graph (orchestration DAG) 2) Runtime sequence (who calls what) 3) State machine (task lifecycle) 4) Prompt lifecycle graph (how prompts evolve) 5) MCP + Kubernetes execution boundary graph Core Agent Framework Primitives Used This repository is a practical example of Agent Framework as a graph orchestrator, not just an agent wrapper. WorkflowBuilder builds a typed DAG with explicit edges. @executor functions implement deterministic nodes (validation, pytest, decisions, routing prep). AgentExecutor wraps LLM agents so they behave as graph nodes. WorkflowContext shared state carries typed data and retry metadata between nodes. add_multi_selection_edge_group(...) + selector functions enforce conditional routing. MCPStdioTool connects filesystem MCP tools into agents for controlled file I/O. Production graph construction (from workflow/builder.py) looks like this: workflow = ( WorkflowBuilder(start_executor=initialize_retry) .add_edge(initialize_retry, generator_executor) .add_edge(generator_executor, parse_generated_task) .add_edge(parse_generated_task, run_validation) .add_edge(run_validation, run_pytest) .add_edge(run_pytest, make_decision) .add_multi_selection_edge_group( make_decision, [keep_task, remove_task], selection_func=select_action, ) .add_edge(keep_task, run_pytest_skip_answer) .add_multi_selection_edge_group( run_pytest_skip_answer, [check_loop, complete_workflow], selection_func=select_skip_answer_action, ) .add_multi_selection_edge_group( check_loop, [fix_task, complete_workflow], selection_func=select_loop_action, ) .add_edge(fix_task, fixer_executor) .add_edge(fixer_executor, parse_generated_task) .build() ) This is the architectural heart of the system: agents and deterministic executors are first-class nodes in the same graph. Detailed Node-by-Node Mechanics 1. Idea Agent (🧠): Concept Synthesis with Memory Constraints The Idea Agent (agents/k8s_task_idea_agent.py) generates a structured concept with three difficulty variations (BEGINNER/INTERMEDIATE/ADVANCED). It is memory-aware: task_ideas_memory.json tracks successful concepts. task_ideas_failure_memory.json tracks concepts that failed downstream. Memory constraints are injected using AgentMiddleware (system-level prompt injection) to avoid duplicate or previously failed concepts. For Responses-only models, the agent switches to a tool-call contract (save_k8s_task_concept) instead of structured response formatting. 2. Generator Agent (⚙️): MCP-Backed File Authoring The Generator Agent receives a strict prompt and writes task files through MCP filesystem tools. Key framework details: Built through chat_client.as_agent(...). MCP tool attached via tools=mcp_tool. Function-call execution observability added with LoggingFunctionMiddleware. Uses absolute-path-only policy in instructions to prevent path drift. 3. Deterministic Validation + Test (✅): Non-LLM Gates After generation, the graph moves through deterministic executors: run_validation calls pure Python checks (k8s_task_validator). run_pytest executes pytest --import-mode=importlib --rootdir=. .... Raw pytest output is persisted in workflow state for later fixing. This is critical: no LLM is asked whether code is correct. 4. Skip-Answer Test (🧪): Grader Correctness Gate Even if standard tests pass, the workflow enforces a second tier: SKIP_ANSWER_TESTS=True pytest --import-mode=importlib --rootdir=. Implementation detail: the executor writes JUnit XML, parses it, and asserts that: test_03_answer.py is skipped test_05_check.py fails as expected If test_05_check.py does not fail, the task is treated as invalid and sent back to retry/fix. 5. Fixer Agent + Retry Loop (🔧): Bounded Self-Healing On failure, fix_task builds a targeted prompt containing: failure reasons from deterministic nodes full captured pytest output explicit rule to patch only broken files in place The Fixer Agent runs through AgentExecutor, writes patches via MCP, and the graph loops back to parse_generated_task. Retries are stateful (retry_count, max_retries) and hard-bounded. On exhaustion, complete_workflow moves the task to unsuccessful/<game>/ and writes FAILURE_REPORT.txt. Agent Prompt Design (The Part That Makes It Work) If you want to understand why this pipeline works, you need to inspect prompts as operational contracts, not generic instructions. Real Idea Agent Prompt (from code) IDEA_AGENT_INSTRUCTIONS = ( "You are a Kubernetes task idea generator that creates detailed task concepts with three difficulty variations. " "Read official K8s documentation and propose comprehensive learning concepts for a Kubernetes game. " "\n\nYour task:\n" "1. Choose ONE Kubernetes concept not yet covered (check context for existing concepts)\n" "2. Generate exactly 3 variations: BEGINNER, INTERMEDIATE, and ADVANCED\n" "3. Use 3-digit task IDs (001-999) in format: XXX_concept_name_level (e.g., 041_secrets_basic)\n" "4. Each variation should build on the previous one with increasing complexity\n" "5. Include practical, hands-on scenarios covering: Workloads, Services, Storage, Configuration, Security, Scheduling, Policies\n" "\nProvide the concept, tags, description, and 3 variations with task_id, difficulty, title, objective, key_skills, and estimated_time." ) Responses-only models use a stricter tool-call version: IDEA_AGENT_INSTRUCTIONS_TOOL_CALL = ( IDEA_AGENT_INSTRUCTIONS + "\n\n" "**CRITICAL**: You MUST call the save_k8s_task_concept tool to save your generated concept.\n" "...\n" "Always call save_k8s_task_concept with your generated concept." ) Idea Agent Prompt Contract The Idea Agent prompt enforces: one concept per run exactly three difficulty variations strict task ID format (XXX_concept_name_level) practical skill progression Core pattern: You are a Kubernetes task idea generator... 1. Choose ONE Kubernetes concept not yet covered 2. Generate exactly 3 variations: BEGINNER, INTERMEDIATE, ADVANCED 3. Use 3-digit task IDs in format XXX_concept_name_level ... It is strengthened by runtime memory injection: previously generated concepts are blocked previously failed concepts are blocked For Responses-only models, the contract becomes tool-driven: CRITICAL: You MUST call the save_k8s_task_concept tool... This reduces ambiguity in output structure and makes downstream parsing deterministic. Real Generator Agent Prompt (from code) def _get_generator_instructions(): return ( "You are a Kubernetes task generator with filesystem tools.\n" f"The MCP filesystem is rooted at: {PATHS.tests_root.parent}\n" f"You MUST use ABSOLUTE paths for ALL file operations.\n" f"Task directory: {PATHS.game_root}/XXX_task_name/\n" "...\n" "CRITICAL: test_02_ready.py checks resources from setup.template.yaml, NOT answer.template.yaml.\n" "...\n" "MUST use polling loops (60s timeout, 15s interval)\n" "MUST use try/except and safe .get() JSON access\n" ) The generator prompt is long on purpose: it encodes path correctness, file schema, YAML/Jinja structure, and testing strategy in a single deterministic contract. Generator Agent Prompt Contract The Generator prompt is intentionally long and prescriptive because it defines filesystem safety and grading correctness requirements. Key constraints encoded in the prompt: Absolute path writes only (prevents writing to wrong workspace paths) No directory creation (directory is pre-created by executor) Required file set (instruction.md, concept.md, session.json, templates, tests) test-flow invariants: test_01_setup.py deploys setup test_02_ready.py checks setup resources only test_03_answer.py deploys answer test_05_check.py validates final solution robust test coding style: polling loops, try/except, .get()-based JSON parsing, explicit debug output Example contract fragment: CRITICAL PATH RULES: ✅ CORRECT: /abs/path/tests/gameXX/050_task/file.py ❌ WRONG: tests/gameXX/050_task/file.py (relative) CRITICAL: test_02_ready.py checks resources from setup.template.yaml, NOT answer.template.yaml. This is why generation quality is high before the Fixer loop even starts. Real Runtime Retry Prompt Builder (from code) def _build_retry_generation_prompt(combined: CombinedValidationResult) -> str: task_id = combined.test.task_id failure_reasons = _build_failure_reasons(combined) return ( f"Generate a complete Kubernetes learning task with ID '{task_id}' about '{combined.target_topic}'. " f"This is retry attempt {combined.retry_count + 1} of {combined.max_retries}. " f"\n\n⚠️ PREVIOUS ATTEMPT FAILED:" f"\n{chr(10).join([f' - {reason}' for reason in failure_reasons])}" f"\n\nIMPORTANT: You MUST use the exact task ID '{task_id}' - do not generate a new ID." f"\n\n✅ Directory already exists: {PATHS.game_root}/{task_id}/" f"\nWrite all files directly into this directory. Do NOT call create_directory." "..." ) This means retries are not generic retries; they are failure-conditioned retries with precise constraints. Fixer Agent Prompt Contract The Fixer prompt is a repair protocol, not a regeneration prompt. It includes: exact failure reasons from deterministic validators raw pytest output instruction to read current task files first strict directive to patch only broken files Core behavior constraints: DO NOT rewrite all files. Make TARGETED FIXES to ONLY the broken files. Use ABSOLUTE paths for all file operations. This keeps retries cheap, preserves working artifacts, and improves convergence speed. Real Runtime Fix Prompt Builder (from code) def _build_fix_prompt(combined: CombinedValidationResult, raw_test_output: str) -> str: task_id = combined.test.task_id failure_reasons = _build_failure_reasons(combined) prompt = ( f"Fix the failed Kubernetes task '{task_id}' located in '{PATHS.game_root}/{task_id}/'." f"\n\nThis is fix attempt {combined.retry_count + 1} of {combined.max_retries}." f"\n\n⚠️ TASK FAILED WITH THESE ERRORS:" f"\n{chr(10).join([f' - {reason}' for reason in failure_reasons])}" ) if raw_test_output: prompt += f"\n\n📋 FULL TEST OUTPUT:\n```\n{raw_test_output}\n```" prompt += ( f"\n\n🔍 YOUR TASK:" f"\n1. READ all files from '{PATHS.game_root}/{task_id}/'" f"\n6. Make TARGETED FIXES to ONLY the broken files" f"\n7. WRITE ONLY the fixed files back" f"\n\n⚠️ CRITICAL: DO NOT rewrite all files! Only fix the broken ones!" ) return prompt How Prompt Output Enters the Agent Framework Graph The prompt builders above are used by deterministic executors and sent to agent nodes through AgentExecutorRequest: await ctx.send_message( AgentExecutorRequest( messages=[Message(role="user", contents=[fix_prompt])], should_respond=True ) ) So prompt generation and graph routing are tightly coupled: each route transition emits a specific prompt payload into the next LLM node. Runtime-Constructed Prompts in Executors The most important prompts are built dynamically in workflow executors: _build_retry_generation_prompt(...) _build_fix_prompt(...) These functions inject live context: retry_count / max_retries concept + objective metadata validation/test failure reasons full captured test logs So each retry is context-rich and specific, not another blind generation attempt. Prompt + Middleware + Deterministic Gates = Reliability In this repository, reliability does not come from prompt text alone. It comes from three layers working together: Prompt contracts constrain agent behavior. Middleware injects memory and logs tool invocations. Deterministic executors enforce objective pass/fail gates. That combination is why the workflow remains auditable and predictable even when LLM outputs vary. Agent Framework Execution Model in This Repo Strongly-Typed Message Passing workflow/models.py defines transport models used between nodes: ValidationResult and TestResult (Pydantic) CombinedValidationResult (dataclass with should_keep and should_retry) InitialWorkflowState (seed payload for each run) This keeps node contracts explicit and simplifies selector logic. Fail-Fast Shared State Management Executors use ctx.get_state(...) with a sentinel (_MISSING) and raise explicit exceptions if required state is absent. This prevents hidden fallback behavior and catches graph/data wiring errors early. Conditional Routing with Selectors Selectors (workflow/selectors.py) encode graph decisions: select_action → keep vs remove select_skip_answer_action → complete vs loop select_loop_action → fix vs complete This separates decision policy from executor implementation. Streaming Workflow Runtime workflow.run(initial_state, stream=True) emits output events incrementally. The runner (workflow/runner.py) consumes these events to detect successful completions and update concept memory accordingly. Agent Construction and API Selection Strategy The repository uses Azure CLI auth (AzureCliCredential) and dynamically selects API mode by deployment name (agents/config.py): Chat Completions path: OpenAIChatCompletionClient Responses-only model path: OpenAIChatClient or custom ResponsesAgent Why this matters: some codex-class deployments are Responses-only, so the architecture supports both without changing workflow logic. How MCP Actually Controls Kubernetes (Important Distinction) In this repo, MCP is used for filesystem control; Kubernetes control is done through kubectl tools. 1) MCP server role: controlled file I/O The workflow starts MCP stdio servers (official filesystem server) and mounts them into agents: docs_mcp_tool = MCPStdioTool( name="filesystem_docs", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", str(PATHS.k8s_docs_root)], load_prompts=False, ) tests_mcp_tool = MCPStdioTool( name="filesystem_tests", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", str(PATHS.tests_root.parent)], load_prompts=False, ) Those MCP tools are passed into Generator/Fixer agents, which then call MCP file functions (read/write/list) inside allowed roots only. 2) Kubernetes cluster control role: kubectl execution tool Cluster actions are not performed by MCP filesystem server; they are performed by a dedicated function tool: def run_kubectl_command(command: str) -> str: kubeconfig_path = os.environ.get("KUBECONFIG", "/home/developer/.kube/config") cmd_list = ["kubectl"] + command.split() result = subprocess.run( cmd_list, capture_output=True, text=True, check=True, env={**os.environ, "KUBECONFIG": kubeconfig_path}, ) return result.stdout And the Kubernetes agent forces tool usage: agent = responses_client.as_agent( name="KubernetesAgent", instructions="...You MUST use the run_kubectl_command tool...", tools=[run_kubectl_command], default_options={"tool_choice": "required"}, ) So the control plane is: MCP filesystem → manipulate generated task files. kubectl tool → query/mutate real cluster state. deterministic pytest/validator executors → accept or reject results. 3) End-to-end command flow in practice When generated tests run, they execute real kubectl get ... -o json checks in test code, and the deterministic runner captures raw output: pytest_command = f"pytest --import-mode=importlib --rootdir=. {task_with_val.task_directory}/" result = run_pytest_command(pytest_command) ctx.set_state(f"raw_output_{task_with_val.task_id}", raw_output) This means Kubernetes state verification is always grounded in live command output, not model speculation. Should MCP run Kubernetes tests? Short answer: not in this design. Current architecture keeps test execution deterministic and local: pytest is run by run_pytest_command(...) (pure Python subprocess runner) test results are parsed and stored in workflow state retry/fix routing uses those deterministic outputs This is intentional. If test execution were delegated to an LLM-facing MCP command tool, you would lose strict control over execution semantics and error handling. Recommended pattern: Use MCP for file/document access and controlled editing. Use deterministic executors for pytest and validation. Use LLM agents only for generation and repair. If you still want MCP-driven test execution, add a separate locked-down command MCP server (only whitelisted pytest/kubectl commands), but keep pass/fail decision logic in deterministic executors. How tests are run in this workflow (with code) The workflow executes tests in deterministic executors, not inside LLM agents. 1) Workflow node calls pytest runner run_pytest executor builds the command and calls the pure Python runner: @executor(id="run_pytest") async def run_pytest(task_with_val: TaskWithValidation, ctx: WorkflowContext[TestResult]) -> None: from agents.pytest_runner import run_pytest_command pytest_command = f"pytest --import-mode=importlib --rootdir=. {task_with_val.task_directory}/" result = run_pytest_command(pytest_command) raw_output = result["details"][0] if result.get("details") else "" ctx.set_state(f"raw_output_{task_with_val.task_id}", raw_output) ... 2) Deterministic subprocess execution The runner normalizes command flags and executes pytest via subprocess: def run_pytest_command(command: str) -> dict[str, Any]: normalized_command = _normalize_pytest_command(command) # adds -s if needed cmd_list = shlex.split(normalized_command) result = subprocess.run( cmd_list, capture_output=True, text=True, check=False, cwd=str(PATHS.pytest_rootdir), ) combined_output = result.stdout + "\n" + result.stderr _save_test_output(normalized_command, combined_output, skip_answer) ... Exit codes are interpreted deterministically: 0 → pass 5 → no tests collected (fail) others → fail with exit code reason 3) Skip-answer validation tier After normal pass, the workflow runs pytest again with SKIP_ANSWER_TESTS=True and parses JUnit XML: os.environ["SKIP_ANSWER_TESTS"] = "True" pytest_command = f"pytest --import-mode=importlib --rootdir=. --junitxml={junit_path} {task_dir}/" result = run_pytest_command(pytest_command) test_05_failed, test_03_skipped = _parse_skip_answer_junit(junit_path) The parser checks per-testcase outcomes: if "test_05_check.py" in context and has_failure_or_error: test_05_failed = True if "test_03_answer.py" in context and has_skipped: test_03_skipped = True 4) How failures trigger fix loop If pytest fails (or skip-answer logic fails), failure reasons and raw output are pushed into state, then the Fixer Agent receives a generated fix prompt containing that output: ctx.set_state(f"failure_reasons_{task_id}", reasons) ctx.set_state(f"raw_output_{task_id}", raw_output) fix_prompt = _build_fix_prompt(combined, raw_test_output) await ctx.send_message( AgentExecutorRequest( messages=[Message(role="user", contents=[fix_prompt])], should_respond=True ) ) That is the key loop: deterministic test output drives LLM repair, then deterministic tests re-run. ResponsesAgent Internals (Advanced Agent Framework Pattern) The custom ResponsesAgent (agents/responses_agent.py) demonstrates a lower-level integration pattern: Connect MCP tool lazily. Call Responses API. Parse ResponseFunctionToolCall items. Execute tools (MCP + custom callables). Feed function_call_output back to model. Repeat until final text response. It also runs a middleware chain around tool invocations, preserving observability and consistency with standard agent paths. Why This Architecture Is Robust This design works because Agent Framework is used as a deterministic orchestration layer around probabilistic generation: LLM creativity is constrained by typed state and strict prompts. deterministic executors act as objective quality gates. retries are targeted, bounded, and auditable. failures produce durable forensic artifacts (FAILURE_REPORT.txt + test logs). For Kubernetes education pipelines, this yields high throughput without sacrificing grader reliability. GitHub Repo - https://github.com/wongcyrus/k8s-game-rule-builder About the Author Cyrus Wong is the senior lecturer of Hong Kong Institute of Information Technology (HKIIT) @ IVE(Lee Wai Lee).and he focuses on teaching public Cloud technologies. He is a passionate advocate for the adoption of cloud technology across various media and events. With his extensive knowledge and expertise, he has earned prestigious recognitions such as AWS AI Hero, Microsoft MVP- Microsoft Foundry, and Google Developer Expert - Cloud(AI).
Productize, observe, version, and automate MCP servers in Azure API Management
Introduction As organizations move from AI-assisted applications to agentic workflows, MCP servers are becoming a critical integration layer between agents, tools, APIs, data sources, and enterprise systems. Azure API Management already helps teams bring MCP servers under enterprise governance. But as MCP adoption scales, platform teams need more than basic exposure. They need a way to package MCP servers for the right consumers, understand tool usage in detail, manage changes safely, and automate configuration across environments. These are familiar API management challenges — and the same patterns that organizations already use for APIs can now be applied more deeply to MCP servers. We are excited to announce new generally available capabilities for MCP server management in Azure API Management: Add MCP servers to products to package and govern MCP capabilities for specific consumers MCP tool observability to trace tool usage, logs, errors, and payload context MCP server versioning to run multiple versions side by side and manage change safely Management API and Bicep support to automate MCP server configuration as part of CI/CD workflows Together, these capabilities extend MCP server management in Azure API Management and help make MCP servers first-class managed resources — productized, observable, versionable, and automatable. Why MCP server management matters MCP gives agents a standard way to connect with tools and external capabilities. That standardization is powerful, but it also introduces a new operational surface for enterprises. Without a management layer, teams can quickly run into questions such as: Which MCP servers are approved for use? Who can access each server? How do we expose MCP servers to different developer or agent audiences? How do we monitor tool calls, latency, errors, and cost? How do we run preview and production versions side by side? How do we automate MCP server configuration across environments? These are not just developer experience questions. They are enterprise governance questions. With Azure API Management, MCP servers can now be managed using the same core patterns organizations already use for APIs: products, subscriptions, policies, observability, versioning, and automation. What’s new 1. Add MCP servers to products Azure API Management products are a proven way to package APIs for consumption. With this release, you can now add one or more MCP servers to APIM products as well. This makes it easier to expose MCP capabilities to specific consumers, teams, applications, or agent experiences using familiar product-based governance. For example, a platform team can create a product for internal agents that includes approved MCP servers such as: Customer profile lookup Order status retrieval Knowledge base search Ticket creation Workflow automation tools By adding MCP servers to products, teams can use familiar controls such as subscriptions, quotas, approval workflows, and access management to govern how MCP capabilities are consumed. Why it matters: MCP servers are no longer isolated endpoints. They can be bundled, governed, and delivered as secure, consumable products. 2. MCP tool observability As agents use MCP servers to discover and invoke tools, teams need more than basic traffic visibility. They need end-to-end trace context for each agent-to-tool interaction. With MCP observability in Azure API Management, teams can inspect key MCP-specific details, including: Operation context: whether the request was a tools/list or tools/call operation Session context: the MCP session ID through gen_ai.conversation.id Client context: MCP client name and version Protocol context: MCP protocol name and version Server context: MCP server name and version Access context: authentication type and API type Tool context: tool name and tool type for tool invocation traces Error context: error type and error message when a call fails Payload context: tool invocation arguments and results when payload logging is enabled This is especially important for agentic workflows, where a single user request may trigger multiple tool calls across different systems. With APIM, MCP traffic can be traced, inspected, and monitored using the same operational practices teams already use across their API estate. Why it matters: MCP servers are not just accessible through APIM — they are observable. Platform teams can trace tool calls, inspect errors, and understand MCP usage with the same operational discipline they expect from managed APIs. 3. Expose multiple MCP versions Enterprise teams need safe ways to evolve MCP servers over time. With MCP server versioning in Azure API Management, you can expose multiple versions of the same MCP server side by side. This allows teams to run a stable GA version while introducing a preview or next version for early adopters. For example: v1 can serve the majority of production traffic. v2 can be exposed to a subset of consumers for testing. Teams can monitor adoption, errors, latency, and behavior. Once the new version is validated, v2 can be promoted with confidence. This pattern is especially useful when MCP tools evolve, schemas change, new capabilities are added, or teams want to validate agent behavior before rolling changes out broadly. Why it matters: MCP servers can now follow a safer lifecycle model: preview, validate, route, promote, and retire. 4. Management API and Infrastructure as Code MCP server management also needs to work at enterprise scale. With Management API and Infrastructure as Code support, teams can provision and configure MCP servers programmatically through Azure API Management APIs and automation pipelines. This allows platform teams to define MCP server resources as part of repeatable deployment workflows using tools such as Bicep, Terraform, ARM, REST APIs, and CI/CD pipelines. Teams can automate configuration for: MCP server endpoints Runtime and transport settings Authentication configuration Metadata and ownership Versioning Product association Policies Environment promotion This is critical for organizations that need consistent MCP governance across development, test, staging, and production environments. Why it matters: MCP server management can now be automated, reviewed, deployed, and governed like the rest of your API platform. How these capabilities work together Individually, each capability solves an important operational need. Together, they create a complete management model for MCP servers in Azure API Management. A platform team can: Register or expose MCP servers through Azure API Management. Package them into products for specific consumers. Apply access controls, subscriptions, quotas, and policies. Observe tool-level usage, latency, errors, traces, and cost. Run multiple versions side by side. Promote changes safely. Automate deployment through APIs and Infrastructure as Code. This brings the full API management playbook to MCP. Instead of treating MCP servers as unmanaged agent extensions, organizations can operate them as governed enterprise resources. Example scenario Imagine a company building internal copilots for customer support, sales, and operations. Each copilot needs access to different tools: Customer lookup Order history Case management Knowledge search Refund workflows Escalation workflows With MCP and Azure API Management, the platform team can expose these capabilities as MCP servers and organize them into products. The customer support copilot can subscribe to the support product. The sales copilot can subscribe to the sales product. Early adopters can be routed to a preview version of a tool. Operations teams can monitor usage, errors, latency, traces, and cost. Platform teams can automate the entire setup across environments. The result is a more governed and scalable way to bring MCP-based tools into enterprise agent workflows. Getting started To get started with MCP server management in Azure API Management: Create or identify an MCP server you want to expose through Azure API Management. Add the MCP server as a managed resource in APIM. Add the MCP server to an APIM product. Configure access, subscriptions, quotas, and approval workflows. Enable observability to monitor tool-level usage and traces. Use versioning to manage preview and production versions. Use the Management API or Infrastructure as Code to automate configuration. Conclusion MCP is quickly becoming an important standard for connecting agents to tools and enterprise capabilities. But for MCP to succeed in production, organizations need more than connectivity. They need governance, lifecycle management, observability, and automation. With these new MCP server management capabilities in Azure API Management, platform teams can manage MCP servers using the same trusted patterns they already use for APIs. MCP servers are now first-class APIM resources — productized, observable, versionable, and automatable. We are excited to see how customers use these capabilities to build the next generation of governed, enterprise-ready agentic applications.New AI gateway capabilities in Azure API Management
Multi-model, multi-protocol AI applications are quickly becoming the norm. Teams are mixing OpenAI, Anthropic, and Vertex AI models, exposing tools through MCP, and wiring agents together with A2A. As that surface grows, so does the work of keeping it secure, observable, and consistent. Our ongoing strategy for the AI gateway capabilities in Azure API Management centers on that problem: providing one place to manage models, MCP tools, and agents, no matter which provider or protocol is behind them. The updates below are the latest steps in that direction. Unified Model API (preview) The headline change in this release: the Unified Model API lets clients speak one API format — OpenAI Chat Completions — while API Management transforms requests to the backend provider, whether that's a model using OpenAI Chat Completions or Anthropic Messages API. By centralizing model access behind a single API layer, you can: Standardize on a single API format for clients, independently from the formats used by backend models. Unify observability, security, and governance with policies that apply across model providers. Configure failover across model providers. Decouple client-facing model names from backend model names using aliases. Learn more about the unified model API. Model aliases Model aliases give clients a stable, provider-neutral name to use when calling a model. By assigning an alias like gpt or claude-sonnet, you decouple the client-facing model name from the actual backend deployment. That makes a few common operations a lot easier: Upgrading a model. Update the alias target to point at a new version — no client code changes required. A/B tests. Shift traffic between backends behind the same alias using API Management's load balancing capabilities. Vendor swaps. Replace one provider with another without touching application code. Model discovery Developers can discover available models by calling the /models endpoint of the Unified Model API. API Management returns the list of model aliases, so apps and tools can adapt to what the platform team has published — without out-of-band documentation. Anthropic and Vertex AI models (GA) AI gateway policies and observability now work with Anthropic and Google Vertex AI models, alongside the providers we already support. You can: Apply runtime policies such as content safety, token limits, and semantic caching to Anthropic and Vertex AI traffic. Collect logs, traces, and metrics for these models in the same place as the rest of your AI traffic. If you're running a multi-provider setup, you no longer need a separate governance story for each vendor. Learn more about AI gateway capabilities in API Management. Anthropic API operations in Microsoft Foundry import When you import a Microsoft Foundry resource as an API in Azure API Management, the import now creates operations for Anthropic APIs alongside the existing model APIs. In a few clicks, you can stand up an API that mediates traffic to Foundry models using either the OpenAI or Anthropic API format — no manual operation definitions needed — and then apply the same policies, security, and observability you use for the rest of your AI traffic. Learn more about Microsoft Foundry import. Token metrics for additional token types (preview) Token tracking used to stop at prompt, completion, and total tokens. Modern models add cached, reasoning, and thinking tokens, which can make up a significant share of token consumption, cost, and latency. API Management now logs metrics for these additional token types into Application Insights, across API formats (OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages API) and providers (Microsoft Foundry, OpenAI, Amazon Bedrock, Google Vertex AI, and others). With richer signals, your cost dashboards, budget alerts, and capacity planning can actually reflect how today's models behave. Learn more about token metrics. Content safety for MCP and A2A (GA) The llm-content-safety policy now covers MCP and A2A traffic in addition to LLM traffic. That includes MCP tool-call arguments, MCP response text, and A2A payloads. A couple of related improvements: llm-content-safety can now be configured directly as an outbound policy. Two new attributes — window-size and window-overlap-size — let you tune how messages exceeding the Azure Content Safety limit of 10,000 characters are chunked and forwarded for validation, balancing detection sensitivity with Azure Content Safety call volume. The result is one consistent safety policy across LLM, MCP, and A2A flows instead of stitching together custom filters per protocol. Learn more about the content safety policy. A2A APIs (GA) Support for Agent-to-Agent (A2A) APIs in API Management is now generally available. Agent APIs can now be governed with the same policies, identity, and observability you use for the rest of your APIs. What you can do with A2A APIs in API Management: Mediate JSON-RPC runtime operations to your agent backend with full policy support — including the content safety improvements above. Expose and manage agent cards, automatically transformed by API Management to represent the managed agent API. Log traces to Application Insights using OpenTelemetry GenAI semantic conventions for deep correlation between API and agent execution traces. What's new in GA, on top of the preview: Available in classic tiers, in addition to v2 tiers — bring A2A governance to existing API Management resources without migrating tiers. Richer diagnostic logging for A2A APIs, giving more actionable telemetry for monitoring and troubleshooting agent traffic. Learn more about A2A support in API Management. Related: Bring Your Own Model in Foundry Agent Service (GA) Last month, Bring Your Own Model (BYOM) in Foundry Agent Service went GA. BYOM lets enterprise teams route Foundry agent model calls through their own infrastructure — typically for compliance, governance, or to reuse an existing model gateway. This pairs naturally with the AI gateway capabilities in Azure API Management. Put API Management in front of your models, apply the policies and observability described above, and have Foundry agents call through it — getting consistent governance for both your direct AI traffic and your agent workloads. Get started Together, these updates make Azure API Management a more complete AI gateway: consistent governance, security, and observability across models from various providers, MCP tools, and agent interactions. Some of these features are still rolling out. They will first become available in v2 tiers of API Management and in the AI release channel for classic tiers, then continue rolling out to the rest of classic tier resources over the following weeks. Get started with the unified model API or explore the AI gateway capabilities in API Management.MCP Test Console and Git Repository synch in Azure API Center
Why This Matters As organizations race to build AI-powered applications, the Model Context Protocol (MCP) has emerged as the standard way to connect AI agents with external tools and data sources. Managing these MCP servers at enterprise scale, however, has been a growing challenge — until now. AI agents are only as useful as the tools they can access. MCP servers expose those tools — from databases and internal APIs to third-party services — in a standardized way that any AI agent or model can consume. As your MCP ecosystem grows, so does the challenge of keeping track of what's available, what's working, and what your teams are actually using. Azure API Center already serves as a centralized registry for APIs across your organization. Now it extends that same governance model to MCP servers, complete with developer-friendly discovery, live testing, and automated synchronization from your source repositories. New Feature: MCP Test Console in the API Center Portal Developers can now test MCP server tools interactively without leaving the Azure portal. Once an MCP server is registered in your API Center inventory, the API Center portal — your organization's customizable developer portal — surfaces a dedicated test console on the server's Documentation tab. Developers simply select a tool, click Run tool, and immediately see the response. This means your teams can: Validate tools before connecting them to agents — no more building a test harness from scratch. Explore tool schemas interactively — the portal surfaces endpoint details and input/output schemas alongside the live console. Onboard faster — developers browsing your internal MCP registry can go from discovery to verified integration in minutes. The MCP server tiles in the portal provide a clear, browsable view of all registered servers. Each tile surfaces the server's endpoint URL, available tools, and installation instructions for Visual Studio Code — giving developers everything they need to get started in one place. Getting started: Set up your API Center portal, then navigate to any registered MCP server. On the Documentation tab, select a tool and click Run tool to open the test console. New Feature: Synch MCP Servers from a Git Repository Managing API assets shouldn't require manual registration every time something changes. With Git repository integration, Azure API Center can automatically sync assets — including MCP server definitions — directly from your source repository. How It Works When you connect a Git repository to your API Center: An environment is created in your API Center representing the repository as an asset source. API Center regularly synchronizes MCP servers from the repository into your inventory — no manual intervention required. Assets appear in your inventory on the Inventory > Assets page with a visual link indicator, making it easy to identify which assets are source-controlled. This is especially valuable for teams that maintain MCP server definitions, skill files, or OpenAPI specs in version control. As your repository evolves, your API Center inventory stays current automatically. Setting It Up Step 1: Secure your access credentials (for private repos) If your repository is private, store a personal access token (PAT) as a secret in Azure Key Vault. Your API Center instance uses a managed identity to retrieve this secret securely — you can configure the managed identity manually or let API Center handle it automatically during the integration setup. Step 2: Connect the repository In the Azure portal, go to your API Center and navigate to Platforms > Integrations > + New integration > From Git repository. You'll configure: Repository URL — including an optional branch and subfolder path (e.g., https://github.com/<org>/<repo>/tree/main/skills). Git provider — such as GitHub. Asset type configuration — API Center defaults to a skill asset type with the file pattern **/skill.md, but you can add additional asset types to match your repository structure. PAT reference — select the Key Vault secret containing your PAT, if applicable. Environment details — give the repository environment a friendly name, resource ID, type (e.g., Production), and lifecycle stage for synced assets. Step 3: Let the sync run Once created, the integration runs automatically. Your assets will appear in the Inventory > Assets view, linked to their source in the repository. Access Control for Private Repositories The integration uses Azure's managed identity framework to authenticate to Key Vault. Assign your API Center's managed identity the Key Vault Secrets User role on your Key Vault to grant the necessary read access. If you prefer, API Center can configure this automatically — just enable the Automatically configure managed identity and assign permissions option during integration setup. Bringing It Together: A Complete MCP Governance Story Together, these two features complete an end-to-end workflow for enterprise MCP governance: Register → Connect your Git repository and let API Center automatically synch your MCP servers and skills as they evolve. Discover → Developers and AI engineers browse the API Center portal to find the right MCP server for their agent, with full schema visibility and endpoint details. Test → The built-in test console lets developers validate tools interactively before committing to an integration. Govern → Use API Center's access management capabilities to control who can view and consume specific MCP servers across your organization. And if you're building MCP servers on Azure services, the registry integrates directly with Azure API Management, Azure Logic Apps, and Azure Functions — so your MCP ecosystem and your API ecosystem share a single source of truth. Get Started Register and discover MCP servers in Azure API Center Synchronize API assets from a Git repository Set up the API Center portal Explore MCP Center — Azure API Center's public MCP registryMore Control, Less Overhead: Custom Domain Upgrades in Azure API Management v2
Multiple custom domains in Premium v2 Large organizations rarely expose APIs under a single domain. A global enterprise might need api.contoso.com for external partners, apis.hrportal.contoso.com for internal teams, and dev.europe.contoso.com for a regional developer portal — all at once. Until now, achieving this required spinning up separate API Management instances, adding cost and operational complexity. Azure API Management Premium v2 now supports multiple custom domains within a single instance — across gateway, developer portal, and management endpoints. This allows organizations to: Configure distinct hostnames for different endpoints and target audiences Align API experiences with business units, products, or regional brands Simplify domain-scoped networking and security policies Reduce the need for separate APIM instances created solely for domain separation For enterprises managing large, distributed API estates, this provides greater flexibility in how APIs and developer experiences are exposed — while maintaining centralized governance. Wildcard custom hostnames in Premium v2 and Standard v2 As API estates grow, managing individual certificates for every subdomain becomes a scaling problem fast. Each new surface — payments.api.contoso.com, inventory.api.contoso.com, orders.api.contoso.com — previously required its own hostname registration and certificate. Ten new API surfaces meant ten separate management tasks. Azure API Management Premium v2 and Standard v2 now support wildcard entries in custom hostnames. A single *.api.contoso.com entry paired with a single wildcard certificate covers all subdomains automatically — no per-subdomain configuration required. This helps teams: Simplify certificate and domain management at scale Accelerate onboarding of new API surfaces without repeated hostname setup Maintain consistent branded endpoints across dynamic subdomains Reduce operational overhead for rapidly growing API environments By extending this capability to both Premium v2 and Standard v2, Azure API Management makes flexible, scalable domain management accessible to more organizations without requiring higher-tier deployments. Both updates are generally available now. Learn more about Azure API Management v2 tiers and how they help organizations build scalable, enterprise-grade API platforms. Further reading: Configure a custom domain name for Azure API ManagementAzure API Center Introduces a Data Plane MCP Server for Enterprise-Wide API and AI Asset Discovery
As organizations scale their adoption of MCP-based tooling and AI agents, one challenge keeps surfacing: developers spend too much time figuring out what APIs, tools, and AI assets exist — and then manually wiring up connections to each one. Today, we're excited to announce general availability of a new capability that changes that. What's new Azure API Center now provides a data plane MCP server — a unified enterprise discovery endpoint that gives agents and developer tools a single connection point to your organization's full catalog of registered MCP servers, tools, APIs, and AI assets. Instead of hunting across systems or hand-configuring integrations one by one, developers and agents can now connect once and immediately access everything that's been registered in your API Center. Why this matters The MCP ecosystem is growing fast. So is the number of enterprise APIs and AI assets that teams need to manage and consume. Without a central discovery mechanism, that growth creates friction — more manual configuration, more drift between what's available and what's actually reachable, and more integration complexity for every new agentic application. The Azure API Center data plane MCP server addresses this directly. With it, teams can: Give agents centralized access to enterprise APIs and AI assets without custom routing logic Eliminate manual configuration of connections to individual MCP servers Automatically surface newly registered MCP servers and tools without reconfiguration Simplify discovery and consumption across a rapidly growing enterprise catalog Built for how organizations actually operate Agentic applications don't just need APIs — they need to find the right APIs, trust that the catalog is current, and connect reliably at scale. By acting as a unified discovery endpoint, Azure API Center helps teams operationalize AI ecosystems with stronger discoverability, governance, and developer productivity, while meaningfully reducing integration complexity. This is especially valuable as enterprises move from experimenting with AI agents to deploying them in production workflows, where manual integration approaches don't scale. How to enable the data plane MCP server Turning on the MCP server takes just a few clicks in the Azure portal. Navigate to your API Center instance and open Data API settings under the Consumption section in the left-hand menu. From there, under MCP endpoint, toggle Enable API Center MCP endpoint to on. Once enabled, your MCP endpoint URL (in the form https://<your-instance>.data.<region>.azure-apicenter.ms/mcp) will appear and can be copied directly for use in agent configurations or developer tools. Note: When enabled, the MCP endpoint is also surfaced on the developer portal homepage, so developers can connect via CLI without needing to look up the URL separately. You can also enable the Plugin marketplace endpoint from the same settings page to let developers browse and install approved plugins and skills from your organization's marketplace. The Visibility section lets you control which APIs are exposed through the data plane — use Add condition to filter the catalog based on your governance requirements. Get started Learn more about Azure API Center and how organizations are building unified catalogs for APIs, MCP tools, agents, and AI assets.Find what you need, faster: Azure API Center now supports custom metadata filtering
Enterprise API and AI catalogs have expanded dramatically. Where teams once managed dozens of APIs, they now govern hundreds — spanning business units, environments, compliance domains, and an ever-growing roster of AI assets. The catalog itself has become a discovery challenge. What's new Developers can now filter catalog assets using organization-defined metadata attributes. These aren't generic tags — they're the classifications your organization already uses: environments, business units, domains, compliance tiers, ownership groups, and more. Custom metadata filtering works across all major asset types in Azure API Center: APIs Skills Agents MCP tools Why it matters Discovery friction is a hidden tax on developer productivity. When a developer needs to find the right API for a project, every minute spent navigating inconsistent lists or applying manual filters is a minute not spent building. At scale, this compounds quickly. Custom metadata filtering addresses this directly by aligning the catalog's search experience with how your organization already thinks about its assets: Surface the right assets faster — filter by internal classifications and governance models instead of browsing overwhelming lists Improve discoverability at scale — no need to retag or reorganize existing assets to make them findable Align with your organizational taxonomy — filter by domain, environment, business unit, compliance requirement, or any custom attribute your teams already use Built for governed, AI-ready teams This update reinforces Azure API Center's role as the foundation for scalable, AI-ready discovery experiences — where governance and developer velocity move together, not against each other. By making enterprise catalogs easier to navigate, Azure API Center helps developers spend less time searching and more time building with governed APIs and AI assets. Get started Learn more about Azure API Center custom metadata filtering and how organizations are building scalable, AI-ready discovery experiences.GA: Azure API Center Now Supports Plugin Registration
As organizations scale their AI and integration ecosystems, one challenge keeps surfacing: developers don't have a reliable, governed way to discover and reuse the plugins their teams have already built. Plugins end up siloed in individual repos, shared over Slack, or duplicated across teams — slowing down development and creating governance blind spots. What's new With this update, developers can register plugins directly into Azure API Center's enterprise catalog. Once registered, plugins are discoverable, governable, and consumable alongside the rest of an organization's API and AI portfolio — no more hunting across repositories or relying on word-of-mouth to find what's already been built. Why it matters Plugins are increasingly central to how AI-powered applications are built. Agents depend on them. Integrations are built on top of them. But without a governed home, even well-built plugins go undiscovered and get rebuilt from scratch. Plugin registration in Azure API Center helps organizations: Centralize plugin discovery within a governed catalog, so developers always know where to look Surface vetted plugins that teams can confidently find and reuse — reducing duplication and accelerating development Align plugins with source-controlled workflows, keeping development practices consistent across the catalog Reduce friction between building a plugin and enabling it for real-world integration One catalog for your entire AI ecosystem This update reflects a broader vision for Azure API Center: a single, unified catalog for everything an organization builds and consumes — APIs, plugins, agents, and AI assets. By bringing plugins into this experience, teams can operationalize reusable integration and AI capabilities at scale, with the governance and discoverability that enterprise development requires. Whether your teams are building copilot extensions, orchestration layers, or custom integrations, Azure API Center gives them a governed foundation to build on — and a shared place to discover what's already there. Learn more about Azure API Center and how organizations are building centralized catalogs for APIs, plugins, agents, and AI assets.
