Educator Developer Blog articles

Building a hands-free voice concierge with Microsoft Foundry Voice Live and a Hosted Agent

Lee_Stott — Fri, 29 May 2026 10:16:09 GMT

This post walks through a small, working sample that wires the browser microphone to Azure AI Speech Voice Live, binds the realtime session to a Foundry hosted agent, and lets the agent answer travel questions using tool calls. The full source, infrastructure, and labs live in the repository linked at the end.

Why this combination matters

Voice user interfaces have historically been hard to build well. Streaming audio, partial transcripts, barge-in, voice activity detection, tool dispatch, and audio playback have traditionally meant stitching together five or six services. The combination of Voice Live and a Foundry hosted agent collapses that into one realtime WebSocket session with a single binding field.

Voice Live owns the audio loop: speech to text, neural text to speech, semantic turn detection, noise suppression, and echo cancellation.
The Foundry hosted agent owns the brain: instructions, memory, model selection, evaluators, and tool calling.
The link between them is one query parameter on the WebSocket URL.

What this means in practice: the browser never sees a model API key, never instantiates a tool, and never owns the agent prompt. The browser does microphone capture and audio playback. Everything else lives server-side.

The scenario

The sample is called Contoso Travel Concierge. The user is mid-journey, hands and eyes busy, and wants to ask things like:

What is the weather in Tokyo this weekend?
Is BA005 from Heathrow on time?
What time is check-in at the Marriott Marquis?

Each question triggers a tool call on the hosted agent. The reply is short, speakable, and synthesised back to the user in under a second on a warm connection.

Architecture

There are four moving parts. Three of them are managed Azure services. Only the broker is your code.

Browser client – captures PCM16 audio at 24 kHz and streams it over a WebSocket to the broker. Plays back audio chunks the broker forwards from Voice Live.
Session broker (FastAPI) – authenticates to Azure with DefaultAzureCredential, builds the Voice Live WebSocket URL with a short-lived bearer token, and relays frames in both directions.
Voice Live – the Azure AI Speech realtime endpoint. Transcribes the user, hands the text to the bound agent, and synthesises the agent’s reply.
Foundry hosted agent – a prompt-kind agent in Azure AI Foundry with instructions, tool definitions, and the microsoft.voice-live.enabled metadata flag set to true.

Two design choices are worth calling out.

The broker is small on purpose. It does authentication, URL construction, and WebSocket relay. It does not transcode audio, run business logic, or hold conversation state. Voice Live and the agent already do those things well.

The agent binding is a URL query parameter, not an SDK call. There is no per-turn HTTP request to the agent runtime. Voice Live opens a session against the agent once and streams turns through it for the lifetime of the WebSocket. That is what keeps latency low.

The Voice Live URL contract

This is the single most important thing to get right. The public Microsoft sample that ships under liupeirong/ai-foundry-voice-agent targets a different URL shape (services.ai.azure.com host, agent-id + agent-access-token parameters, an Authorization header). That shape is rejected by Foundry resources that expose voice-live-enabled agents. The shape below is the one the portal itself uses, and the one this sample dials.

Three details cause most failures:

The host must be <resource>.cognitiveservices.azure.com, not services.ai.azure.com. The broker rewrites this automatically from VOICE_LIVE_ENDPOINT.
The bearer token travels in the authorization query parameter, URL-encoded, with a literal Bearer prefix and a + (or %20) before the token. No Authorization header is sent.
agent-name and model are both the agent’s display name. agent-version is empty when you want the latest published version.

Walkthrough: from clone to spoken reply

Prerequisites

Python 3.11 or later (the sample is developed on 3.13).
The Azure CLI, signed in with az login --tenant <your-tenant-id>.
An Azure AI Foundry project in a Voice Live region (eastus2, swedencentral, or westus2).
A deployed prompt-kind agent in that project with Enable Voice Live turned on.
The Cognitive Services User role on the Foundry resource for the identity the broker will use.

Configure the broker

Copy .env.sample to .env and fill in four values:

AZURE_AI_PROJECT_ENDPOINT=https://<your-resource>.services.ai.azure.com
AZURE_AI_PROJECT_NAME=<your-foundry-project-name>
VOICE_LIVE_ENDPOINT=wss://<your-resource>.services.ai.azure.com/voice-live/realtime
VOICE_LIVE_API_VERSION=2025-10-01
FOUNDRY_AGENT_ID=<your-agent-name>

The agent name is what the Foundry portal shows on the agent card. The broker uses it for both the agent-name and model query parameters.

Install and run

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
.\scripts\start-local.ps1

The broker exposes three endpoints:

GET /healthz – liveness probe.
GET /config – returns the session.update the browser sends as its first frame.
WS /ws – the bi-directional relay to Voice Live.

Smoke test

.\scripts\test-session.ps1

A successful run prints:

[OK] /ws upgraded
   -> sent session.update
   <- {"type":"session.created",…}
   <- {"type":"session.updated",…}
[OK] session.updated received -- E2E works

This confirms the entire chain: local broker, DefaultAzureCredential token, Foundry Portal URL shape, Voice Live handshake, and the bound agent acknowledging the session.

Open the browser UI

Browse to http://localhost:8000/, click Start talking, and ask one of the sample questions. Transcripts appear in real time and the spoken reply plays back through the audio context.

Inside the broker

The relay logic is tiny – the heavy lifting is the URL construction. The function below is the canonical reference; copy it if you are porting the pattern to another language.

def build_voice_live_ws_url(agent_access_token: str) -> str:
    """
    Build the Foundry Portal style Voice Live WebSocket URL.

    Auth lives in the query string only. No Authorization header is sent.
    """
    host = _ws_host_from_endpoint(VOICE_LIVE_ENDPOINT)
    qs = urlencode(
        {
            "trafficType": "FoundryPortal",
            "agent-name": FOUNDRY_AGENT_ID,
            "agent-version": "",
            "agent-project-name": AZURE_AI_PROJECT_NAME,
            "api-version": VOICE_LIVE_API_VERSION,
            "model": FOUNDRY_AGENT_ID,
            "client-request-id": str(uuid.uuid4()),
            "authorization": f"Bearer {agent_access_token}",
        },
        quote_via=quote,
    )
    return f"wss://{host}/voice-live/realtime?{qs}"

The relay itself is a pair of asyncio tasks: one forwarding browser frames upstream, one forwarding Voice Live frames back. Audio bytes are passed straight through – the broker never decodes them.

Deploying the hosted agent

The most reliable way to create a voice-live-enabled agent is the Foundry portal. Agents created via the Assistants v2 SDK do not carry the required metadata by default and will be rejected by the Voice Live URL shape above.

The portal steps are:

Open the Foundry project, go to Agents, and click New agent.
Choose Prompt agent as the kind, name it (for example travel-concierge), and pick a model deployment.
Paste the contents of agent/src/prompts/system.txt into the instructions box.
On the Voice tab, switch Enable Voice Live on. This is what sets the microsoft.voice-live.enabled = true metadata.
Add the three tools (get_weather, get_flight_status, get_hotel_info) from agent/agent.yaml on the Tools tab.
Publish the version and write the agent name back to .env as FOUNDRY_AGENT_ID.

The full deployment guide, including how to host the broker on Azure Container Apps with a managed identity, is in docs/deployment.md in the repository.

Three lessons from getting this to production

1. Voice output must be written for speech, not for screens

Foundry agents tend to format answers in markdown with citations like ([data.jma.go.jp](https://…)). When Voice Live synthesises that text, the user hears the URL read aloud, character by character. The fix is to write the agent instructions so the spoken text never contains URLs, markdown, or symbols. A short block at the end of the agent instructions does the job:

Voice output rules
- This output is read aloud by TTS. Never include URLs, domain names, or
  citation markers like "(source.com)" in your reply. Cite by speakable
  source name only.
- Never use markdown for formatting. No asterisks, brackets, backticks,
  bullets, or hashes. Write in plain spoken sentences.
- Keep numbers speakable: say "thirty degrees Celsius", not "30C / 86F".
- Keep replies under about 40 words unless the user asks for detail.

The browser transcript can still render markdown for the eyes. The sample does so with a small, escaping markdown renderer that whitelists bold, italic, code, and http(s) links only, so the same agent reply looks polished on screen even though the spoken version contains none of it.

2. Identity is simpler than it looks

The broker uses DefaultAzureCredential and requests the https://ai.azure.com/.default scope. Locally that resolves to your az login credentials. In Azure Container Apps it resolves to the user-assigned managed identity. In both cases the only role assignment you need on the Foundry account is Cognitive Services User. There is no API key path on the working URL shape – it is bearer tokens all the way down.

3. The wrong sample wastes a day

If you start from the public liupeirong/ai-foundry-voice-agent repository against a portal-provisioned voice-live agent, the WebSocket either returns HTTP 400 or closes silently with code 1006. The cause is the URL shape, not your code. The reference probe in scripts/probe_portal_shape.py is the single source of truth for the working contract – keep it as a regression test.

Responsible AI and security notes

Credentials never reach the browser. Tokens are minted server-side and travel only on the upstream Voice Live URL.
No secrets in source. The .env file is gitignored. The .env.sample contains only placeholders.
Markdown rendering is escape-first. The browser HTML-escapes the agent reply before applying its small markdown whitelist, and links are restricted to http(s) URLs so the rule cannot emit javascript: hrefs.
Tool calls are auditable. Every turn shows up as a run in the Foundry portal under the agent, with the prompt, model output, and tool inputs and outputs visible for review.
Voice biometric considerations. If you plan to handle account verification by voice, plug in dedicated speaker recognition rather than relying on the conversational model.

Key takeaways

Voice Live plus a Foundry hosted agent is a session-level integration, not an API integration. One URL, one binding field, one WebSocket.
The browser is a thin client. Authentication, URL construction, and relay all live in a small FastAPI broker.
Get the URL shape right (cognitiveservices.azure.com, token in the query string, agent-name equals model equals the agent display name) and the rest is plumbing.
Use the Foundry portal to create the agent so the voice-live metadata is set correctly.
Write agent instructions for the ear, not the eye, then layer screen formatting on top in the browser.

Get the code and try it

Repository: github.com/microsoft/foundry-agent-voice-mode-sample
Deployment guide: docs/deployment.md in the repository.
Labs: three progressive workshops under labs/ – basic voice, adding tools, and binding a hosted agent.
Reference docs: Voice Live in Azure AI Speech and Agents in Microsoft Foundry.

If you build something on top of this pattern, open an issue or pull request on the repository. The sample is intentionally small so it stays easy to fork.

Building Reliable AI Coding Workflows Using Modular AI Agent Optimization

ShardaKaur — Thu, 28 May 2026 19:53:45 GMT

Artificial Intelligence is rapidly transforming the modern software development industry. AI-powered coding assistants such as GitHub Copilot, Claude Code, and other Large Language Model (LLM)-based systems are helping developers automate repetitive coding tasks, improve productivity, and accelerate software development processes. These tools can generate code, assist with debugging, provide recommendations, and support developers during implementation. However, despite their growing capabilities, many AI coding assistants still face challenges related to reliability, maintainability, project-specific conventions, and structured software engineering workflows.

Most coding assistants perform well for generic programming tasks but often struggle when working with domain-specific development requirements, API integrations, project architectures, validation workflows, and coding standards. In real-world software engineering environments, developers require systems that not only generate code but also follow project conventions, maintain readability, support modular development, and improve long-term maintainability.

The project “AI Agents Optimization” focuses on improving the reliability and effectiveness of AI coding agents by designing structured workflows, modular configurations, validation mechanisms, and optimized task execution strategies. The objective of the project is to investigate how AI agents can become dependable collaborators in practical software engineering tasks instead of functioning only as autocomplete systems.

The project explores different approaches for organizing AI agent workflows using structured instruction handling, modular task division, context management, validation systems, and integration of external tools and documentation sources. Different agent configurations are analyzed and evaluated to understand how workflow optimization affects software development quality and performance.

Why Existing AI Coding Workflows Often Fail

Most AI coding assistants perform well for isolated coding tasks but struggle in real-world engineering environments where projects involve multiple files, coding standards, APIs, validation requirements, and contextual dependencies.

For example, a generic prompt such as:

“Build authentication middleware”

may generate functional code, but the output often lacks:

Project-specific architecture
Error handling consistency
Validation logic
Security best practices
Dependency awareness

This project approaches the problem differently by introducing a structured workflow pipeline where AI agents operate in defined stages rather than generating outputs in a single step.

The workflow separates planning, generation, validation, and refinement into independent modules. This improves maintainability, reduces inconsistent outputs, and supports iterative refinement similar to real software engineering workflows.

Project Objectives

The primary objective of this project is to optimize AI coding agents for real-world software engineering workflows. The project aims to improve how AI systems handle development tasks such as code generation, debugging, testing, validation, feature implementation, and workflow management.

Another major objective is to design modular AI workflows where different stages of software development are managed systematically. The workflow focuses on task planning, instruction processing, validation, refinement, and output evaluation. This structured approach improves transparency, maintainability, and consistency in AI-generated outputs.

The project also aims to evaluate how AI coding agents perform under different configurations and development scenarios. By testing multiple workflows and structured instruction methods, the project analyzes how optimization techniques improve development reliability and coding quality.

Technologies and Tools Used

The project utilizes multiple modern technologies and development tools for experimentation and workflow optimization.

Technology / Tool	Purpose
Python	Automation and scripting
GitHub Copilot	AI-assisted coding
Claude / LLM APIs	AI workflow experimentation
Visual Studio Code	Development environment
Git & GitHub	Version control and repository management
Structured Prompting	Workflow optimization
MCP Concepts	Tool and context integration

These tools collectively support the implementation and testing of optimized AI coding workflows.

Implementation Workflow

The system was implemented using a modular AI workflow pipeline where each stage performs a dedicated engineering task.

Step 1 — Task Parsing

The user submits a development task or coding requirement. The Instruction Processing Module extracts:

Objective
Constraints
Project context
Expected output format

Example structured prompt:

Task: Create JWT authentication middleware

Language: Node.js

Constraints:

- Use Express.js

- Add token validation

- Follow modular architecture

- Include error handling

Step 2 — Planning & Reasoning

The Planning Module divides the task into subtasks such as:

Route handling
Token verification
Error management
Security validation

This improves reasoning consistency before generation begins.

Step 3 — Code Generation

The Code Generation Module produces outputs using structured prompts and contextual references instead of generic instructions.

Step 4 — Validation

Generated outputs are validated using:

Syntax checks
Logical consistency checks
Formatting standards
Dependency validation

Step 5 — Refinement

If validation fails, the workflow loops back into refinement where issues are corrected before final delivery.

System Workflow

The workflow of the AI Agents Optimization system is based on modular task execution and structured development processes. The workflow begins with task planning and requirement analysis. The AI agent receives structured instructions along with coding constraints, project context, and validation requirements.

The system processes the provided instructions and generates outputs according to defined workflows and development standards. Different configurations are tested to evaluate how instruction structures and modular task handling influence the quality of generated code

The workflow also includes validation and refinement stages where generated outputs are analyzed for correctness, maintainability, and consistency. The project focuses not only on code generation but also on improving readability, workflow transparency, debugging support, and adherence to project conventions.

Key Features of the Project

Structured AI workflow design
Modular task execution
AI-assisted software development
Workflow optimization strategies
Validation and refinement mechanisms
Integration of development tools and documentation
Improved maintainability and readability
Support for practical software engineering workflows

Challenges Faced During Development

One of the major challenges encountered during the project was maintaining consistency and reliability in AI-generated outputs. Different AI models often produce different responses depending on prompts, context, and task structure. Designing workflows that improve output stability and maintain coding standards required careful experimentation and optimization.

Another challenge involved integrating structured workflows while ensuring flexibility in task execution. AI systems often require clear instructions and contextual information to produce accurate outputs. Balancing automation with maintainability and project-specific requirements was an important aspect of the project.

Managing validation and refinement processes was also challenging because generated outputs needed to be evaluated not only for correctness but also for readability, maintainability, and software engineering best practices.

Observations and Outcomes

During experimentation, structured workflows produced more reliable and maintainable outputs compared to single-prompt generation approaches.

Some important observations included:

Reduced repetitive corrections during code refinement
Improved consistency in generated outputs
Better adherence to coding structure and formatting
More stable workflow behavior for multi-step tasks
Improved readability and maintainability of generated code

The validation and refinement stages were particularly effective in reducing incomplete outputs and improving response quality.

Although the project focuses primarily on workflow architecture and qualitative analysis rather than benchmark testing, the results demonstrate that modular AI pipelines can significantly improve practical software engineering workflows.

Future Enhancements

The project can be further enhanced by implementing advanced multi-agent collaboration systems where multiple AI agents work together on complex software development tasks. Future versions may also include real-time documentation integration, automated testing frameworks, cloud-based workflow management, and improved reasoning models.

Additional enhancements may include IDE extensions, intelligent debugging systems, automated code review mechanisms, and adaptive workflow optimization based on project requirements.

Conclusion

The AI Agents Optimization project demonstrates how structured workflows and modular configurations can improve the effectiveness of AI-powered coding assistants in modern software engineering environments.

By focusing on workflow optimization, validation mechanisms, modular task execution, and structured instruction handling, the project highlights the future potential of AI agents as reliable development collaborators capable of supporting real-world software engineering processes.

The project represents an important step toward building dependable AI-assisted development systems that improve productivity, maintainability, and software quality while supporting modern engineering practices.

How to Try This Workflow

Define a structured development task
Provide project constraints and context
Break the task into subtasks
Generate output using structured prompts
Validate output quality
Refine based on validation feedback

Hybrid AI Agents in Python: Routing Between Foundry Local and Microsoft Foundry

Lee_Stott — Wed, 27 May 2026 07:00:00 GMT

Why hybrid, and why now

If you build AI features today, you are caught between three forces. Users want low latency and strong privacy. Product teams want frontier reasoning capability. Finance teams want predictable cost. No single model satisfies all three. Run everything on a small on-device model and you bottleneck on complex questions. Send everything to a frontier cloud model and you pay for trivial requests, leak sensitive data across a network boundary, and add hundreds of milliseconds of latency to greetings.

The pragmatic answer is hybrid inference: a lightweight local model classifies every request first, simple or sensitive ones stay on the device, and only the genuinely hard or frontier-capability requests escalate to the cloud. Microsoft now ships both halves of that pattern as supported Python SDKs — foundry-local-sdk for on-device inference and azure-ai-projects for Microsoft Foundry cloud models. This post walks through a working reference implementation that combines them behind a single ask() call.

The full source is at github.com/leestott/fl-mixedmodel. It is Python-only, secretless by design, and ships with a Gradio diagnostics UI, a CLI demo mode, and a full pytest suite.

The contract: one schema, two paths

The most important architectural decision is that callers never know which path served a request. Every response, local or cloud, returns the same dataclass:

class InferencePath(str, Enum):
    LOCAL = "local"
    CLOUD = "cloud"
    LOCAL_FALLBACK = "local_fallback"   # cloud attempted, fell back to local
    CLOUD_FALLBACK = "cloud_fallback"   # local attempted, fell back to cloud

@dataclass
class AgentResponse:
    answer: str
    path: InferencePath
    model: str
    reason: str
    confidence: float
    latency_ms: float
    correlation_id: str
    prompt_tokens: Optional[int] = None
    completion_tokens: Optional[int] = None
    fallback: bool = False
    fallback_reason: Optional[str] = None
    metadata: dict = field(default_factory=dict)

This is what makes the design honest. The router can change, the cloud model can be swapped from gpt-4o to gpt-5.4, fallback policies can flip — and the calling code never breaks. The four InferencePath values give you full observability without leaking implementation details into the API surface.

Architecture in one diagram

┌─────────────┐   prompt    ┌──────────────────────────┐
│   caller    │ ──────────► │   HybridAgentService     │
└─────────────┘             │      .ask(prompt)        │
                            └────────────┬─────────────┘
                                         │
                            ┌────────────▼─────────────┐
                            │     RoutingPolicy        │
                            │  1. Heuristic gate       │
                            │  2. Local router LLM     │
                            │  3. Hard policy gates    │
                            └─────┬─────────────┬──────┘
                                  │             │
                          LOCAL  ◄┘             └► CLOUD
                                  │             │
                       ┌──────────▼──┐   ┌──────▼───────┐
                       │ Foundry     │   │ Microsoft    │
                       │ Local SDK   │   │ Foundry      │
                       │ (phi-4-mini)│   │ (gpt-5.4)    │
                       └─────────────┘   └──────────────┘

Best practice: the two-stage router pattern

Before walking through the implementation, it is worth stating the design pattern explicitly, because it is the part that generalises beyond this specific repo. The cleanest design for hybrid inference is a two-stage router.

Stage 1 — local router. A small local model performs intent and complexity classification first. It does not answer the question; it decides where the question should go.
Stage 2 — route the answer.
- If the prompt is simple, private, latency-sensitive, or clearly within local capability, route to a local task model on the device.
- If the prompt is complex, needs deeper reasoning, a larger context window, or a capability unavailable locally, escalate to a cloud frontier model in Microsoft Foundry.

Microsoft's current guidance for the cloud side is to use the Responses API and choose one of two control modes:

Pass a specific deployment name (for example gpt-5.4) when you want deterministic control over which model serves the request, which is the right choice for regulated workloads, repeatable evaluations, or cost ceilings.
Pass model-router as the deployment when you want Microsoft Foundry to automatically select the best available cloud model for each request. This is a sensible default for general-purpose agents where you would rather let the platform optimise the model choice as new ones are released.

The reference repo exposes both as environment variables so you can switch without code changes:

# .env.example
FOUNDRY_CLOUD_MODEL_DEPLOYMENT=gpt-5.4        # deterministic
FOUNDRY_CLOUD_ROUTER_DEPLOYMENT=model-router  # auto-select

Best practice: pin the right SDK versions

Two SDKs do the heavy lifting and both have had recent breaking changes, so version discipline matters.

Local development — foundry-local-sdk. The current public guidance is to use the Foundry Local SDK package foundry-local-sdk, which provides model discovery, download, cache, load, unload, chat completions, embeddings, audio transcription, and an optional built-in web service. Use version 1.1.0, released on 5 May 2026. Earlier versions used an OpenAI-compatible client surface that has since been replaced by the FoundryLocalManager → load_model → get_chat_client → complete_chat chain shown above. Pin it explicitly:
```
# requirements.txt
foundry-local-sdk>=1.1.0
```
Cloud orchestration and agents — azure-ai-projects. For cloud-side orchestration, Microsoft's current Python guidance is to use azure-ai-projects, which the docs describe as part of the Microsoft Foundry SDK and as the entry point for agents, deployments, connections, datasets, evaluations, and an OpenAI-compatible client returned by get_openai_client(). The current PyPI listing shows azure-ai-projects 2.1.0. Pin it explicitly:
```
# requirements.txt
azure-ai-projects>=2.1.0
azure-identity>=1.17.0
```

If you find yourself reading old samples that import azure.ai.inference as the cloud entry point, or that initialise Foundry Local through a raw openai.OpenAI(base_url=...) client, you are looking at pre-2026 patterns. The current shape is what the reference repo uses: FoundryLocalManager.initialize(Configuration(...)) for the device and AIProjectClient(...).get_openai_client() for the cloud.

Stage 1: a deterministic privacy gate

Before any model touches a prompt, a deterministic heuristic classifier scans for sensitive patterns — passwords, API keys, SSN/NHS numbers, PII signals, explicit "do not share" flags. If the heuristic returns PrivacyClass.RESTRICTED, the prompt is forced local. The router LLM is not called. The cloud provider is not called. The decision is auditable from a single regex pass.

# app/routing/policy.py
def decide(self, prompt: str, correlation_id: str = "") -> RoutingDecision:
    hint, privacy, complexity, h_reason = self._heuristic.classify(prompt)

    # Hard gate: restricted content never leaves the device
    if privacy == PrivacyClass.RESTRICTED:
        return self._make_decision(
            target=RouteTarget.LOCAL,
            confidence=1.0,
            reason=f"Policy hard-gate: {h_reason}",
            privacy=privacy,
            complexity=complexity,
            deterministic=True,
            correlation_id=correlation_id,
        )

    # Hard gate: very high complexity always goes to cloud
    if complexity == ComplexityBand.VERY_HIGH:
        return self._make_decision(
            target=RouteTarget.CLOUD,
            confidence=1.0,
            reason="Policy hard-gate: very_high complexity requires frontier model",
            ...
        )

This is the most important responsible-AI control in the whole system. If your privacy review depends on an LLM correctly classifying every prompt, you do not have a privacy control — you have a probability distribution. Deterministic gates first, model judgement second.

Stage 2: a local LLM as the router

For everything that passes the privacy gate, a small local model classifies whether the prompt needs frontier capability. This is the bit that surprises most engineers: you can do useful routing with a 4B parameter model running on a laptop CPU. The router does not need to answer the question. It only needs to classify it.

The reference implementation uses phi-4-mini via Foundry Local. Initialising it is two lines:

# app/providers/local_provider.py (excerpt)
from foundry_local import FoundryLocalManager
from foundry_local.models import Configuration

self._manager = FoundryLocalManager.initialize(
    Configuration(app_name="hybrid-agent")
)
self._router_model = self._manager.load_model(self._config.local_router_alias)
self._chat_client  = self._router_model.get_chat_client()

response = self._chat_client.complete_chat(
    messages=[
        {"role": "system", "content": ROUTER_SYSTEM_PROMPT},
        {"role": "user",   "content": prompt},
    ],
)

The router prompt asks for a strict JSON response: { "target": "local|cloud", "confidence": 0.0-1.0, "complexity": "low|medium|high|very_high", "reason": "..." }. The application parses it, applies the confidence threshold from config (default 0.6), and falls back to the heuristic decision if the router LLM is unsure or its JSON is malformed. The router never blocks the answer path — that is a deliberate reliability choice.

Cloud inference via Microsoft Foundry

When the policy returns RouteTarget.CLOUD, the request goes through AIProjectClient, which gives you an openai.OpenAI-compatible client wired to your Foundry project with DefaultAzureCredential. No API keys. No secrets in .env.

# app/providers/cloud_provider.py (excerpt)
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

self._project = AIProjectClient(
    endpoint=self._config.foundry_project_endpoint,
    credential=DefaultAzureCredential(),
)
self._openai_client = self._project.get_openai_client()

response = self._openai_client.chat.completions.create(
    model=self._config.foundry_cloud_model_deployment,  # e.g. "gpt-5.4"
    messages=messages,
    max_completion_tokens=max_tokens,
)

A subtle gotcha worth flagging: gpt-5 and o-series deployments reject the legacy max_tokens parameter and require max_completion_tokens. They also reject custom temperature values. The reference repo handles this by trying the new parameter first and falling back to the legacy one only when the API returns the specific unsupported parameter error. That keeps the same code working against older deployments without forking the provider.

Graceful degradation: the fallback paths

Hybrid systems fail in interesting ways. The cloud can be down. The local model can throw because the GPU ran out of memory. A reasoning model can return an empty completion. The service handles all of these by attempting the alternative path and labelling the response so observability stays honest:

Cloud route fails → local fallback. The response carries path=LOCAL_FALLBACK, fallback=true, and a populated fallback_reason. The user gets an answer instead of an error.
Local route fails → cloud fallback, but only if privacy class is not RESTRICTED. A sensitive prompt that the local model could not handle never leaks to the cloud as a fallback. It returns a clear error instead. This is the second hard gate in the system.
Both fail. A structured error response with a correlation ID, never a stack trace.

That last rule — fallback respects privacy class — is the kind of decision that is easy to skip and impossible to bolt on later. Encode it once in the service layer and your privacy reviewers will thank you.

What it looks like in practice

The diagnostics panel in the Gradio UI shows the routing decision live: path, model, confidence, latency, privacy class, complexity band, and the full JSON response. Five canonical scenarios shake out the entire decision tree:

"hello" → path=local, confidence=1.0, complexity=low. Heuristic only. No router LLM call. ~3 seconds end-to-end with phi-4-mini cached.
"explain transformer self-attention in depth with maths" → path=cloud, model=gpt-5.4, complexity=high. Router LLM classifies, hard gate confirms.
"my password is hunter2, suggest a stronger one" → path=local, privacy=restricted, deterministic=true. Privacy gate fires before any model sees it.
"summarise this 8 KB document" with cloud unavailable → path=cloud_fallback (local handles it, response is labelled).
Complex prompt with local model error → path=local_fallback, fallback_reason populated.

You can reproduce all five without any models installed by running python -m app.main --demo. The demo mode swaps the providers for deterministic stubs so you can validate the routing logic and the response schema in under a second on any machine.

Operational lessons learned

Some things the reference implementation only gets right because it got them wrong first:

Pick a non-reasoning model for the router. Reasoning-tuned local models (Phi-4-reasoning, o-style) wrap their output in <think> blocks and blow your JSON parser. phi-4-mini is faster and more reliable for classification.
Cache the local model. First load can take 30–60 seconds while Foundry Local downloads weights. Initialise the service once at process startup, not per request.
Use correlation IDs everywhere. The service attaches one per request and the structured JSON logger emits it on every event. When you are debugging a fallback path across two model providers, this is the difference between five minutes and five hours.
Run the privacy heuristic on every fallback path too. A naive implementation might route locally, fail, and then send the same sensitive prompt to the cloud as a "graceful" fallback. That is not graceful, it is a data leak.
Keep configuration in .env and out of code. Privacy mode, fallback toggles, confidence threshold, model aliases — all environment-driven. The config.py module is the only place that reads them.

Responsible AI in a hybrid topology

Hybrid does not make responsible AI harder, but it does make it different. Three controls earn their keep:

Data residency by default. The local path keeps prompts and answers on the device. For RESTRICTED content this is mandatory; for everything else it is a free latency and cost win.
Auditability. Every routing decision is logged with the deterministic reason, the heuristic class, the router LLM output, the confidence, and the correlation ID. You can answer "why did this prompt go to the cloud?" months later.
Keyless auth. DefaultAzureCredential means there is no API key to leak, rotate, or commit by accident. The repo's .gitignore, SECURITY.md, and pre-push checklist enforce this end-to-end.

Try it

Five minutes, no Azure account needed for the demo:

git clone https://github.com/leestott/fl-mixedmodel.git
cd fl-mixedmodel

python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate     # macOS / Linux

pip install -r requirements.txt
python -m app.main --demo       # all five scenarios, no models required

To run with real models, install Foundry Local, copy .env.example to .env, set your FOUNDRY_PROJECT_ENDPOINT, then:

az login
python -m app.main --ui --port 7860

Where to go next

Repository: github.com/leestott/fl-mixedmodel — full source, tests, specification, screenshots.
Foundry Local SDK: pypi.org/project/foundry-local-sdk and the Foundry Local docs.
Azure AI Projects SDK: pypi.org/project/azure-ai-projects and the Microsoft Foundry docs.
Azure Identity: DefaultAzureCredential reference.
Phi-4-mini: Phi-4-mini on Hugging Face.

Key takeaways

The best-practice pattern is a two-stage router: local model classifies first, then either a local task model or a Microsoft Foundry cloud model answers.
For cloud control, use the Responses API with either a named deployment (deterministic) or model-router (auto-select).
Pin foundry-local-sdk >= 1.1.0 (5 May 2026) and azure-ai-projects >= 2.1.0. The 2026 SDK surfaces are not backwards-compatible with pre-2026 samples.
Hybrid inference is a routing problem, not a model problem. A small local model is enough to classify the request.
Deterministic privacy gates beat probabilistic ones. Code the rules; let the LLM judge only what is left.
Return the same response schema from every path. Label fallbacks honestly. Carry a correlation ID everywhere.
Keep auth keyless with DefaultAzureCredential and your .env out of git.
Test the routing decisions, not just the model outputs. Demo mode and a strong pytest suite pay back every time you swap a model.

Hybrid AI is not a compromise between local and cloud. It is the supervisor pattern applied to inference — fast and private where you can be, frontier where you have to be, observable everywhere. The hard part is the contract, not the models.

Build AI RAG Apps with Azure DocumentDB (with MongoDB compatibility) and OpenAI: Step-by-Step Guide

JohnAziz — Tue, 26 May 2026 22:08:40 GMT

Scenario

Imagine you are building your company’s RAG chat application using Azure OpenAI Service and orchestrating the flow with LangChain. The chat experience works, but now it needs to be grounded in your company’s data. You generate embeddings and want to store and query them without adding another database or complex sync pipeline. Instead of stitching services together, you use Azure DocumentDB (with MongoDB compatibility) with built-in vector search to store your JSON data and embeddings in one place. You deploy the app to Azure App Service and quickly compare vector search alone versus a full RAG pipeline, sharing it with your team for testing.

What will you learn?

In this blog, you'll learn to:

Create an Azure DocumentDB (with MongoDB compatibility) resource.
Create an embeddings and a chat deployment in Azure Foundry portal.
Create an Azure App service website with Continuous deployment from GitHub.
Configure Azure App service application settings to enable communication between Azure resources.
Configure GitHub workflow to work successfully.

What is the main objective?

Build AI Powered RAG Application using the LangChain, Azure OpenAI, and Azure DocumentDB (with MongoDB compatibility): Step-by-Step Guide

Prerequisites

An Azure subscription.
- If you don’t already have one, you can sign up for an Azure free account.
- For students, you can use the free Azure for Students offer which doesn’t require a credit card only your school email.
A GitHub Account.

Summary of the steps:

Step 1: Create an Azure DocumentDB (with MongoDB compatibility) resource

Step 2: Create an Azure OpenAI resource and Deploy chat and embedding Models

Step 3: Create an Azure App Service and Deploy the RAG Chat Application

Step 1: Create an Azure Cosmos DB for MongoDB vCore Cluster

In this step, you'll:

Open the Azure Portal.
Create an Azure Cosmos DB for MongoDB vCore Cluster.

Open the Azure Portal

1. Visit the Azure Portal https://portal.azure.com in your browser and sign in.

Now you are inside the Azure portal!

Create a new Azure DocumentDB (with MongoDB compatibility) resource

In this step, you create an Azure Cosmos DB for MongoDB vCore Cluster to store your data, vector embedding, and perform vector search.

1. Type mongodb vcore in the search bar at the top of the portal page and select Azure Cosmos DB for MongoDB (vCore) from the available options.

2. Select Create from the toolbar to start provisioning your new cluster.

3. Add the following information to create a resource:

What	Value
Subscription	Use your preferred subscription. It's advised to use the same subscription across all the resources that communicate with each other on Azure.
Resource group	Select Create new to create a new resource group. Enter a unique name for the resource group.
Cluster name	Enter a globally unique name.
Location	Select a region close to you for the best response time. For example, Select UK South.
MongoDB version	Select the latest available version of MongoDB.

4. Select Configure to configure your cluster tier.

5. Add the following information to configure the cluster tier. You can scale it up later:

What	Value
Cluster tier	Select M25 tier, 2 (Burstable) vCores.
Storage	Select 32 GiB.

6. Select Save.

7. Enter the cluster Admin Username and Password and store them in a secure location.

8. Select Next to configure the networking settings.

9. Select Allow Public Access from Azure services and resources within the Azure to this cluster.

10. Select Add current IP address to the firewall rules to allow local access to the cluster.

11. Select Review + create.

12. Confirm your configuration settings and select Create to start provisioning the resource.

Note: The cluster creation can take up to 10 minutes. It's recommended to move on with the rest of the steps and get back to it later.

Step 2: Create an Azure OpenAI resource and Deploy chat and embedding Models

In this step, you'll:

Create an Azure OpenAI resource.
Create chat and embedding model deployments.

Create an Azure OpenAI resource

In this step, you create an Azure OpenAI Service resource that enables you to interact with different large language models (LLMs).

1. Type openai in the search bar at the top of the portal page and select Azure OpenAI from the available options.

2. Select Create from the toolbar to provision a new OpenAI resource.

3. Add the following information to create a resource:

What	Value
Subscription	Use the same subscription you used to apply for Azure OpenAI access.
Resource group	Use the resource group you created in the previous step.
Region	Select a region close to you for the best response time. For example, Select UK South.
Name	Enter a globally unique name.
Pricing tier	Select S0. Currently, this is the only available pricing tier.

4. Now that the basic information is added, select Next to confirm your details and proceed to the next page.

5. Select Next to confirm your network details.

6. Select Next to confirm your tag details.

7. Confirm your configuration settings and select Create to start provisioning the resource. Wait for the deployment to finish.

8. After the deployment finishes, select Go to resource to inspect your created resource. Here, you can manage your resource and find important information like the endpoint URL and API keys.

Create chat and embedding model deployments

In this step, you create an Azure OpenAI embedding model deployment and a chat model deployment. Creating a deployment on your previously provisioned resource allows you to generate text embeddings (i.e. numerical representation for text) and have a natural language conversation with your data.

1. Select Go to Azure OpenAI Studio from the toolbar to open the studio.

2. Select Create new deployment to go to the deployments tab.

3. Select + Create new deployment from the toolbar. A Deploy model window opens.

4. Add the following information to create a chat model deployment:

What	Value
Select a model	Select gpt-35-turbo.
Model version	Select 0301.
Deployment name	Add a name that's unique for this cloud instance. For example, chat-model because this model type is optimized for having conversations.

5. Select Create.

6. Select + Create new deployment from the toolbar. A Deploy model window opens.

7. Add the following information to create an embedding model deployment:

What	Value
Select a model	Select text-embedding-ada-002.
Model version	Select 2.
Deployment name	Add a name that's unique for this cloud instance. For example, embedding-model because this model type is optimized for creating embeddings.

8. Select Create.

Step 3: Create an Azure App Service and Deploy the RAG Chat Application

In this step, you'll:

Fork the sample Repository on GitHub.
Create an Azure App service resource with a deployment from GitHub.
Modify Azure App service Application settings in the Azure portal.
Configure the Workflow to deploy your application from GitHub.
Test the website before and After adding the data.

Fork the sample Repository on GitHub

In this step, you create a copy from the source code on your GitHub account to be able to edit it and use it later.

1. Visit the sample github.com/john0isaac/rag-semantic-kernel-mongodb-vcore in your browser and sign in.

2. Select Fork from the top of the sample page.

3. Select an owner for the fork then, select Create fork.

Create an Azure App service resource with a deployment from GitHub

In this step, you create an Azure App service resource and connect it with your GitHub account to deploy a Python application.

1. Type app service in the search bar at the top of the portal page and select App Services from the available options.

2. Select Create Web App from the toolbar to start provisioning a new web application.

3. Add the following information to fill in the basic configuration of the application:

What	Value
Subscription	Use the same subscription you used to apply for Azure OpenAI access.
Resource group	Use the same resource group you created before.
Name	Enter a unique name for your website. For example, rag-mongodb-demo.
Publish?	Select Code. This option specifies whether your deployment consists of code or a container.
Runtime stack	Select Python 3.10.
Operating System	Select Linux.
Region	Select UK South. This is the region where the rest of the resources you created reside.

4. Add the following information to create the app service plan. You can scale it up later:

What	Value
Linux Plan	Select a pre-existing plan or create a new plan.
Pricing Plan	Select Basic B1.

5. Select Deployment from the toolbar to move to the deployment configuration tab.

6. Add the following information to enable continuous deployment from GitHub:

What	Value
Continuous deployment	Select Enable.
GitHub account	Select your GitHub account.
Organization	Select your organization. If you are using your personal account then select it.
Repository	Select rag-semantic-kernel-mongodb-vcore.
Branch	Select main.

7. Select Review + create.

8. Confirm your configuration settings and select Create to start provisioning the resource. Wait for the deployment to finish.

9. After the deployment finishes, select Go to resource to inspect your created resource. Here, you can manage your resource and find important information like the application settings and logs.

Modify Azure App service Application settings in the Azure portal

In this step, you configure the Application settings to make the website able to communicate with other cloud resources.

1. In the Web App resource, select Configuration from the left side menu.

2. Select + New application setting to add new environment variables to the function configuration.

3. Add the following names and values one by one and select Ok. Make sure to add your own values.

These application settings are for the Azure OpenAI resources that you created:

What	Value
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME	<chatModelDeploymentName>
AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME	<embeddingModelDeploymentName>
AZURE_OPENAI_DEPLOYMENT_NAME	<azureOpenAiResourceName>
AZURE_OPENAI_ENDPOINT	https://<azureOpenAiResourceName>.openai.azure.com/
AZURE_OPENAI_API_KEY	<azureOpenAiResourceKey>

You can get the Azure OpenAI key from the Azure OpenAI resource page.

Select Keys and Endpoint and copy any of the available keys.

These application settings are for Azure Cosmos DB for MongoDB vCore:

AZCOSMOS_API	mongo-vcore
AZCOSMOS_CONNSTR	mongodb+srv://<mongoAdminUser>:<mongoAdminPassword>@<mongoClusterName>.global.mongocluster.cosmos.azure.com/?tls=true&authMechanism=SCRAM-SHA-256&retrywrites=false&maxIdleTimeMS=120000

You can get the Cosmo DB connection string from the Azure Cosmos DB for MongoDB vCore resource page.

Select Connection strings and copy the connection string. Make sure to replace the user and password with the ones you created.

These application settings are new and will be created when the application starts:

AZCOSMOS_DATABASE_NAME	<cosmosDatabaseName>
AZCOSMOS_CONTAINER_NAME	<cosmosContainerName>

Any value should work for them.

4. Select Save.

5. Select General settings to edit the application startup command.

6. Type entrypoint.sh in the startup command field and select Save.

Configure the Workflow to deploy your application from GitHub

In this step, you modify the GitHub deployment workflow to point to the folder that contains the application.

1. Visit your forked repository on GitHub and notice the failing workflow.

2. Open the function workflow file .github/workflows/main_ragmongodbdemo.yml.

3. Open the file and select the pen icon to edit it.

4. Modify lines 31 and 36 to the following:

31	run: cd src && pip install -r ./requirements.txt
36	run: cd src && zip ../release.zip ./* -r

5. Select Commit changes, and review your commit message and description. Select Commit changes.

6. Select Actions to review the workflow run status.

Test the website before and After adding the data

In this step, you test the application before adding the data, add the data, and test again.

1. Select the workflow name to open it and get the website URL.

2. Type in the chat message What is Azure Functions? and it should respond with I don't know.

3. Navigate to your Azure App service resource page and select SSH.

4. Select Go to open a new SSH page.

5. In the SSH terminal, run this command:

python ./scripts/add_data.py

6. Navigate back to the live website and type in the chat message What is Azure Functions? and it should respond with the correct answer now.

Congratulations!! You successfully built the full application.

If you want to learn how to add your own data see this guide on the repository's main readme.

Clean Up

Once you finish experimenting on Microsoft Azure you might want to delete the resources to not consume any more money from your subscription.

You can delete the resource group and it will delete everything inside it or delete the resources one by one that's totally up to you.

Conclusion

Congratulations! You've learned how to create an Azure Cosmos DB for MongoDB vCore cluster, how to create an Azure OpenAI resource, how to deploy an embedding model and a chat model from Azure OpenAI studio, how to create an Azure App service and configure continuous deployment with GitHub, and how to modify application settings to enable the communication across Azure resources. By using these technologies, you can build a RAG chat application with the option to perform vector search too over your own data and provide grounded (relevant) responses.

Next steps

Documentation

Training Content

Found this useful? Share it with others and follow me to get updates on:

Twitter (twitter.com/john00isaac)
LinkedIn (linkedin.com/in/john0isaac)

Feel free to share your comments and/or inquiries in the comment section below..
See you in future demos!

CI/CD for AI Agents on Microsoft Foundry

Lee_Stott — Fri, 22 May 2026 08:50:02 GMT

Introduction

Building an AI agent is the straightforward part. Shipping it reliably to production with version control, evaluation-driven quality gates, multi-environment promotion, and enterprise governance is where most teams run into friction.

Microsoft Foundry changes this. It is Microsoft's AI app and agent factory: a fully managed platform for building, deploying, and governing AI agents at scale. It provides a first-class agent runtime with built-in lifecycle management, making it possible to apply the same CI/CD rigour you already use for application software to AI agents — regardless of whether you are building containerised hosted agents or declarative prompt-based agents.

This post walks through a complete, production-ready reference architecture for doing exactly that. You will find the GitHub Actions workflow, the Azure DevOps pipeline YAML, and the architecture diagram linked throughout.

Reference implementation repository: foundry-agents-lifecycle
and CI/CD for AI Agents on Microsoft Foundry

Why Agent CI/CD Is Different

Traditional software pipelines gate releases on test pass/fail. Agent pipelines require an additional, critical layer: evaluation-driven quality gates. Before any agent version can be promoted to the next environment, it must pass three categories of evaluation:

Quality — answer correctness, task completion rate, hallucination rate
Safety — grounded responses, policy compliance, tool usage validation
Performance — token usage per query, p95 response latency

A second key difference is the deployment unit. You are not deploying a binary or a container tag in isolation. You are deploying an agent version — an immutable artefact that bundles the model selection, system instructions, tool definitions, and configuration together. This is what enables deterministic promotion and full auditability across environments.

"Agents follow a standard CI/CD pattern, but with a critical shift: promotion happens at the agent version level, and release gates are driven by evaluation outcomes, not just test results."

Reference Architecture

Figure 1: End-to-end CI/CD reference architecture for hosted and prompt-based agents on Microsoft Foundry.

The architecture has five logical layers, flowing from developer commit to production monitoring:

Layer 1 — Developer Layer

The developer layer is a standard source-controlled repository in GitHub or Azure DevOps. It contains:

Agent code written in Python or .NET
agent.yaml or prompt definition files for prompt-based agents
Tool configurations: MCP servers, REST API connectors, or other integrations
Infrastructure as Code: Bicep or ARM templates for provisioning the Foundry project and dependencies

Layer 2 — CI Pipeline (Build · Validate · Evaluate)

Every push or pull request triggers the CI pipeline. It performs five steps:

Docker build — for hosted agents, build and tag the container image
Static checks — lint with ruff, security scan with bandit, agent YAML schema validation
Unit and tool tests — pytest suites covering agent logic and tool integrations
Evaluation gate — run evaluation datasets; fail the pipeline if thresholds are breached
Image push — push the validated container to Azure Container Registry (ACR)

Prompt-based agents skip the Docker build step. Instead, the YAML definition and prompt bundle are validated against schema and evaluated against golden datasets.

Layer 3 — CD Pipeline (Multi-stage Promotion)

A single agent version is promoted through three Foundry project environments:

Stage	Environment	Activities	Gate
Stage 1	Dev Foundry Project	Deploy vNext version, smoke tests, developer evals	Eval quality thresholds
Stage 2	Test / QA Foundry Project	Scenario tests, HITL validation, safety evaluation	Eval gates + human approval
Stage 3	Production Foundry Project	Promote version, enable endpoint, post-deploy smoke test	Required reviewer approval

Rollback is straightforward: switch the active version pointer back to the previous agent version. No re-deployment is needed.

Layer 4 — Microsoft Foundry Agent Service

The Foundry Agent Service runtime provides:

Hosted agent runtime — managed container execution supporting Agent Framework, LangGraph, Semantic Kernel, or custom code
Prompt-based agent runtime — declarative agent definitions, no container required
Built-in lifecycle operations — version, start, stop, rollback
Entra Agent Identity — each deployed version receives a dedicated Microsoft Entra managed identity
RBAC and policy enforcement — Azure role-based access controls per project
Observability — distributed traces, structured logs, and evaluation signals

Layer 5 — Monitoring, Governance, and Control Plane

Foundry control plane: agent registry, environment configuration, version history
OpenTelemetry forwarded to Azure Monitor and Application Insights
Continuous evaluation pipelines for ongoing quality, grounding, and safety monitoring
Azure Policy and RBAC enforcement at the platform level

Environment Topology

There are two topology options. We recommend Option A for all production workloads:

Option	Structure	Best for	Trade-off
A — Recommended	Dev Project → Test Project → Prod Project (separate Foundry projects)	Enterprise workloads	Full isolation, clean RBAC boundaries, easier governance
B — Lightweight	Single Foundry project with agent version tags (dev/test/prod)	Small teams, prototyping	Simpler setup, but weaker environment separation

Separate projects mean separate RBAC policies, separate connection strings, and separate evaluation signals. A developer service principal has access only to the Dev project; the CI/CD identity has restricted access to promote to Test and Production.

Evaluation Gates — The Core Difference

Evaluation gates transform a standard software pipeline into an AI-safe deployment pipeline. They run at two points: pre-merge (CI) and pre-promotion (CD).

Defining the Gates

Category	Metric	CI threshold	Prod threshold
Quality	Hallucination rate	< 5%	< 3%
Quality	Task completion rate	> 90%	> 95%
Safety	Grounded response rate	> 95%	> 98%
Safety	Policy violations	0	0
Performance	p95 latency	< 4 000 ms	< 3 000 ms
Cost	Token usage per query	Track only	Alert on > 20% regression

Gate Enforcement (Python)

import json
import sys

def check_gates(results_path: str) -> None:
    with open(results_path) as f:
        results = json.load(f)

    failures = []

    if results["hallucination_rate"] > 0.05:
        failures.append(f"Hallucination rate {results['hallucination_rate']:.1%} exceeds 5% threshold")

    if results["task_completion_rate"] < 0.90:
        failures.append(f"Task completion {results['task_completion_rate']:.1%} below 90% threshold")

    if results["latency_p95_ms"] > 4000:
        failures.append(f"p95 latency {results['latency_p95_ms']}ms exceeds 4000ms threshold")

    if results.get("policy_violations", 0) > 0:
        failures.append(f"Policy violations detected: {results['policy_violations']}")

    if failures:
        for f in failures:
            print(f"GATE FAILED: {f}", file=sys.stderr)
        sys.exit(1)

    print("All evaluation gates passed — proceeding to deployment")

if __name__ == "__main__":
    check_gates(sys.argv[1])

Hosted vs Prompt-Based Agents — Pipeline Differences

Capability	Hosted Agents	Prompt-Based Agents
Deployment unit	Container image + agent definition	YAML / prompt configuration bundle
Build step required	Yes — Docker build + ACR push	No — YAML validation only
Supported frameworks	Agent Framework, LangGraph, Semantic Kernel, custom	Foundry declarative runtime
Promotion artefact	Versioned agent with container image reference	Versioned prompt/config bundle
CI focus	Code quality, tool tests, evaluation	Prompt schema validation, evaluation
Rollback mechanism	Switch active agent version	Switch active agent version
Runtime management	Foundry manages container lifecycle	Foundry manages declarative runtime

CI Pipeline Walkthrough

The following steps are representative of the full GitHub Actions workflow available in github-actions-pipeline.yml alongside this post.

Hosted Agent CI

# 1. Static checks
ruff check .
bandit -r src/ -ll
python scripts/validate_agent_config.py --config agent.yaml

# 2. Tests
pytest tests/unit/ -v --tb=short
pytest tests/tools/ -v --tb=short

# 3. Evaluation gate
python scripts/run_evaluations.py \
    --dataset eval/datasets/golden_set.jsonl \
    --output  eval/results/results.json

python scripts/check_eval_gates.py \
    --results eval/results/results.json \
    --max-hallucination   0.05 \
    --min-task-completion 0.90 \
    --max-latency-p95     4000

# 4. Push container image
az acr build \
    --registry myregistry.azurecr.io \
    --image    "myagent:$SHA" \
    --file     Dockerfile .

Prompt-Based Agent CI

# Validate YAML / prompt definitions
python scripts/validate_agent_config.py --config agent.yaml

# Evaluation against golden dataset
python scripts/run_evaluations.py \
    --dataset eval/datasets/golden_set.jsonl \
    --output  eval/results/results.json

python scripts/check_eval_gates.py \
    --results eval/results/results.json

CD Pipeline Walkthrough

Stage 1 — Dev Deployment

python scripts/deploy_agent.py \
    --env              dev \
    --image            "myregistry.azurecr.io/myagent:$SHA" \
    --foundry-endpoint $FOUNDRY_ENDPOINT_DEV \
    --agent-config     agent.yaml

# Returns the new agent version ID, stored for promotion
AGENT_VERSION=$(python scripts/get_active_version.py --env dev)

Stage 2 — Promote to Test (after approval gate)

python scripts/promote_agent.py \
    --from-env         dev \
    --to-env           test \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_TEST

# Run scenario tests and safety evaluation
python scripts/run_evaluations.py \
    --dataset  eval/datasets/scenario_set.jsonl \
    --output   eval/results/test-results.json

python scripts/check_eval_gates.py \
    --results              eval/results/test-results.json \
    --max-hallucination    0.03 \
    --min-task-completion  0.95

Stage 3 — Promote to Production (after required reviewer approval)

python scripts/promote_agent.py \
    --from-env         test \
    --to-env           prod \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

# Enable the production endpoint
python scripts/enable_agent_endpoint.py \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

Rollback

# Switch the active version to the previous known-good version
python scripts/promote_agent.py \
    --from-env         prod \
    --to-env           prod \
    --agent-version    $PREVIOUS_AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

# OR delete the failing version
python scripts/delete_agent_version.py \
    --agent-version    $AGENT_VERSION \
    --foundry-endpoint $FOUNDRY_ENDPOINT_PROD

Deployment Using the Azure AI Projects SDK

The azure-ai-projects SDK provides programmatic control over the full agent lifecycle. This is the recommended approach for CI/CD scripts where you need deterministic, scriptable deployment.

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Connect to the Foundry project
client = AIProjectClient(
    endpoint=FOUNDRY_PROJECT_ENDPOINT,
    credential=DefaultAzureCredential()
)

# List existing agents (useful for idempotent deploy scripts)
for agent in client.agents.list():
    print(f"Agent: {agent.name}  version: {agent.id}")

# Create a new agent version (hosted agent)
agent = client.agents.create_agent(
    model="gpt-4o",
    name="my-enterprise-agent",
    instructions="You are a helpful assistant ...",
    tools=[...],  # tool definitions
    metadata={"version": GIT_SHA, "environment": "dev"}
)
print(f"Created agent version: {agent.id}")

For hosted agents, the SDK call also references the container image pushed to ACR. Refer to the Deploy a hosted agent — Microsoft Foundry documentation for the full SDK flow including container image registration and version polling.

Reference Implementation Stack

Concern	Technology
Source control and pipelines	GitHub Actions or Azure DevOps Pipelines
Infrastructure and agent deployment	Azure Developer CLI (`azd up`)
Programmatic agent lifecycle	`azure-ai-projects` Python SDK
Agent evaluation	`azure-ai-evaluation` Python SDK
Agent runtime	Microsoft Foundry Agent Service
Container registry	Azure Container Registry (hosted agents only)
Observability	OpenTelemetry, Azure Monitor, Application Insights
Identity and access	Microsoft Entra (Agent ID, OIDC workload identity federation)
Governance	Azure Policy, RBAC, Foundry control plane

Governance and Responsible AI

Shipping AI agents at enterprise scale requires governance beyond what a traditional CI/CD pipeline provides. Microsoft Foundry addresses this at the platform level:

RBAC per environment — each Foundry project has independent access controls. Developers deploy to Dev; only CI/CD service principals (with audited OIDC tokens) can promote to Test and Production.
Agent registry and audit trail — the Foundry control plane records which agent version is active in each environment, who deployed it, and when. This satisfies enterprise audit requirements without additional tooling.
Content safety and policy enforcement — Azure Policy governs model access, data handling, and content safety rules at the infrastructure level, not just at the application code level. Policy violations block deployment automatically.
Entra Agent Identity — each deployed agent version receives a dedicated, short-lived managed identity. Agents authenticate to downstream services using least-privilege credentials scoped to that specific deployment.
Continuous evaluation in production — evaluation pipelines run on sampled production traffic, alerting when quality, safety, or cost metrics drift from their baseline.

A key trade-off to be transparent about: evaluation datasets must be maintained and updated as the agent's tasks evolve. Stale datasets produce misleading pass/fail signals. Treat your golden evaluation set as a first-class engineering artefact alongside the agent code itself.

Pipeline Files

Two pipeline files accompany this reference architecture. Both implement the same four-stage pipeline (CI Build, CI Evaluate, CD Dev, CD Test, CD Production) with environment-appropriate approval gates.

github-actions-pipeline.yml — GitHub Actions workflow. Uses GitHub Environments for approval gates and OIDC Workload Identity Federation for passwordless Azure authentication. No stored Azure credentials required.
azure-devops-pipeline.yml — Azure DevOps multi-stage YAML pipeline. Uses ADO Environments with required approvers and variable groups per environment.

Both pipelines share these security practices:

OIDC / Workload Identity Federation — no long-lived Azure credentials stored in pipeline secrets
Per-environment variable groups, each with scoped connection strings and endpoints
Evaluation quality gates enforced before every promotion step
Mandatory human approval before production deployment

Summary

The full pipeline in one view:

Developer commit
        |
   CI Pipeline
   ├── Docker build (hosted agents) / YAML validation (prompt agents)
   ├── Static checks + unit tests + tool tests
   └── Evaluation gate  ←  quality · safety · performance
        |
   Agent Version created  ← immutable, versioned artefact
        |
   CD Pipeline
   ├── Deploy to Dev       → smoke tests + eval gate
   ├── Promote to Test     → scenario tests + HITL + approval gate
   └── Promote to Prod     → enable endpoint + monitoring
        |
   Microsoft Foundry Agent Service
   └── Versioned runtime · Entra identity · RBAC · Observability
        |
   Control Plane
   └── Agent registry · Governance · Continuous evaluation

Microsoft Foundry provides the platform primitives — versioned agent deployments, multi-environment Foundry projects, built-in lifecycle management, and an enterprise observability stack — needed to operate AI agents with the same confidence as any production software system.

The key takeaway: treat the agent version as your deployment artefact, and evaluation outcomes as your release gate. The rest follows familiar CI/CD patterns you already know and trust.

Next Steps

Clone the CI/CD Repo at leestott/foundry-cicd
Clone the reference demo: foundry-agents-lifecycle on GitHub
Set up your environment: Set up your environment for Foundry Agent Service
Deploy your first hosted agent: Quickstart: Deploy your first hosted agent
Understand hosted agent concepts: Foundry Hosted Agents concepts
Automate deployments in CI/CD: Automate deployment of Microsoft Foundry agents
Manage agent versions: Manage hosted agents — Microsoft Foundry
Deploy via SDK: Deploy a hosted agent — Microsoft Foundry
SDK and endpoint reference: Microsoft Foundry SDK and Endpoints reference
Azure AI Projects SDK: azure-ai-projects Python SDK
Azure Developer CLI: Azure Developer CLI (azd) overview
Microsoft Foundry documentation hub: Microsoft Foundry on Microsoft Learn

Student Devs: Build AI Agents, Compete for $55K in Prizes

Lee_Stott — Thu, 21 May 2026 07:23:46 GMT

Student Devs: Build AI Agents, Compete for $55K in Prizes

🎮 AI Skills Fest • June 4–14, 2026 • Free to Enter

$55K
Prize Pool

3
Challenge Tracks

10
Days of Hacking

Free
To Enter

Whether you're a first-year CS student or a final-year senior with a portfolio full of projects, Agents League is the best way to gain hands-on experience with agentic AI this summer and walk away with real skills employers are hiring for right now.

What You'll Actually Learn

Forget passive tutorials. Agents League is project-based learning at full speed. By the end of the hackathon, you'll have built a working AI agent and gained practical experience with the tools shaping the future of software development.

🤖 AI-Assisted Development Use GitHub Copilot to accelerate your coding workflow — from scaffolding to debugging — the way professional developers do today.	🧩 Multi-Step Reasoning Build agents with Microsoft Foundry that can plan, reason, and execute complex tasks — the core of agentic AI.
🏢 Enterprise AI Patterns Learn to build production-ready agents that integrate with Microsoft 365 and Copilot Studio — skills that translate directly to industry jobs.	🔧 Prompt Engineering Design effective prompts and orchestration flows that make AI agents reliable and useful in the real world.
📦 GitHub Workflows Submit your project through GitHub — practising version control, README writing, and open-source collaboration.	🎯 Competitive Problem-Solving Work under real constraints with deadlines, judging criteria, and peer competition — just like industry hackathons and sprints.

Pick Your Track (or Try All Three)

Agents League has three challenge tracks, each using different Microsoft AI tools. Choose based on your interests or stretch yourself by competing in multiple tracks.

Track 01. Creative Apps

Build an innovative application with AI-assisted development. This track rewards creativity, dream big and let GitHub Copilot help you bring ideas to life faster than ever.
Tool: GitHub Copilot

Track 02. Reasoning Agents

Create intelligent agents that solve complex problems through multi-step reasoning. Think: agents that can research, plan, and act. This is the cutting edge of AI.
Tool: Microsoft Foundry

Track 03. Enterprise Agents

Build knowledge agents that integrate with Microsoft 365 Copilot. Learn how businesses are deploying AI today and add enterprise AI to your skillset.
Tool: Copilot Studio • M365

Opportunities You Won't Want to Miss

Agents League isn't just a competition, it's a launchpad. Here's what's in it for you beyond the code:

💰 Win from a $55,000 USD Prize Pool
Prizes are awarded across all three tracks smaller teams and solo hackers have a real shot.
📺 Watch Live Coding Battles at Microsoft Reactor
See industry experts go head-to-head building AI agents live. Learn advanced techniques you can apply immediately to your own project.
🎓 Free Learning Resources on Microsoft Learn
Access curated learning paths and the AI Skills Navigator, structured content designed to get you from zero to submission-ready.
🌍 Join a Global Developer Community
Connect with thousands of developers on the Agents League Discord. Find teammates, ask questions, and build your professional network.
📂 Build Your Portfolio with a Real Project
Every submission lives on GitHub. Walk away with a polished, public project that demonstrates your AI skills to future employers and grad schools.
🏆 Gain Recognition from Microsoft and the Community
Top projects get visibility across the Microsoft developer ecosystem. Stand out from the crowd in internship and job applications.

Key Dates to Remember

Event	Date
Hacking Period Opens	June 4, 2026
Registration Deadline	June 12, 2026 — 12:00 PM PT
Submission Deadline	June 14, 2026 — 11:59 PM PT

How to Get Started (Right Now)

You don't have to wait until June 4th to start preparing. Here's your pre-hackathon game plan:

Register for the hackathon it's free and open to everyone.
Pick a track that matches your interests or curiosity.
Explore the learning resources on Microsoft Learn and the AI Skills Navigator.
Join the Discord community to find teammates and get early tips.
Watch the Reactor event series for live coding battles and expert walkthroughs.
Set up your GitHub repo and start experimenting before the hacking window opens.

Helpful Links

Register for Agents League Free entry, sign up now
Microsoft Reactor Events Live coding battles & workshops
AI Skills Fest The broader event
Microsoft Learn Free learning paths

The Arena Awaits 🏆

Ten days. Three tracks. $55K in prizes. Whether you go solo or squad up, this is your chance to build something real with AI and have a blast doing it.

Register Now It's Free | Watch Reactor Events

Agents League is part of AI Skills Fest and is open to the public at no cost.
Review the Hackathon Rules and Regulations and the Microsoft Event Code of Conduct before participating.

Signing in to Microsoft Foundry from OpenClaw using Azure AD: a smoother way to bring your models in

suzarilshah — Wed, 20 May 2026 07:11:23 GMT

This post is a quick update to walk through the new flow. If you read the previous one, think of this as the easier path I wish I had the first time round. If you have not seen the original, you can find it here: Integrating Microsoft Foundry with OpenClaw: Step by Step Model Configuration | Microsoft Community Hub

Pre-requisite:

You will need the Azure CLI (azure-cli) installed on your machine. The official install guide for Linux is here: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest

I am on Linux so I went the Homebrew route, which keeps things simple. The formula is here: https://formulae.brew.sh/formula/azure-cli

Microsoft also has official docs covering the Homebrew/Linuxbrew install: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-macos?view=azure-cli-latest#install-with-homebrew

Once Homebrew is ready, run this in your terminal:

brew install azure-cli

Why this matters:

Before this update, every Foundry model you wanted to use in OpenClaw needed its own API key and endpoint pasted into the config. It worked, but it was tedious, and keys are easy to leak if you are copying them around. The Azure AD path solves both problems. You authenticate as yourself (or a service principal), OpenClaw asks Azure for the list of Foundry resources you have access to, and it brings the models in automatically.

Signing in to Microsoft Foundry from OpenClaw via Azure AD

A device-code OAuth handshake replaces the old static-API-key flow. OpenClaw delegates auth to the local Azure CLI; the CLI handles the browser-side sign-in, holds the resulting tokens, and refreshes them silently. OpenClaw then walks the Azure resource graph, subscriptions → Foundry resources → model deployments and registers each model into its own config. No API keys move through OpenClaw at any point.

Figure. Sequence diagram of the OAuth 2.0 device-authorization flow as orchestrated by OpenClaw. Phases 1–3 establish identity (the developer authenticates once, in a real browser, against Azure AD). Phases 4–5 perform service discovery (OpenClaw walks the ARM resource hierarchy, subscriptions → Foundry accounts → model deployments and persists the result to a local provider config). After registration, every model call OpenClaw makes against Foundry reuses the same Azure-CLI-managed token cache: tokens refresh transparently, and access is gated by the Foundry resource's RBAC assignments rather than a static API key. Dashed lines denote return values; the teal line in step 7 marks the single token-issuance event the rest of the system pivots on.

Walking through the new flow:

Start with the command to onboard openclaw as if you were setting up OpenClaw for the first time:

openclaw onboard

Kick things off with the OpenClaw onboard command, the same one you would use when setting up OpenClaw for the first time. When it prompts you, choose update values.

Next, you will be asked to configure your models. Scroll down a little and you will see Microsoft Foundry listed as a supported provider. Pick it.

From here, you have two options. You can sign in with an API key, which is what I covered in the previous blog post, or you can sign in through Azure AD. The Azure AD path is easier and more secure, so that is the one we will use.

OpenClaw will give you a URL and a device code. Copy the URL into your browser and use the code to complete the sign in. (This is where the az CLI from the pre-requisite section earns its keep.)

If everything worked, you should see a success prompt similar to this:

Once you are signed in, OpenClaw will ask you to pick the Azure subscription that your Microsoft Foundry resource lives in. Pick the subscription, then pick the Foundry resource where your models are deployed.

And that is pretty much it. All the models you have deployed to that Foundry resource get pulled into OpenClaw automatically. Compared to the old way of pasting API keys and endpoints one by one, this is a huge time saver, and you do not have to babysit any keys.

From here you can start using your Foundry-deployed models inside OpenClaw straight away:

Wrapping up

The Azure AD sign-in option in OpenClaw is one of those small updates that quietly removes a real pain point. If you have ever juggled multiple Foundry endpoints and rotated keys across them, you already know why. With this flow, you sign in once, your models show up, and you can get back to actually building.

If you have not tried OpenClaw with Microsoft Foundry yet, this is a good time to give it a go. And if you were holding off because of the key management overhead, that excuse is gone now.

References

Previous post on integrating Microsoft Foundry with OpenClaw using API keys: Integrating Microsoft Foundry with OpenClaw: Step by Step Model Configuration | Microsoft Community Hub

Install the Azure CLI on Linux: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest

Install the Azure CLI on macOS: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-macos?view=azure-cli-latest#install-with-homebrew

Homebrew formula for azure-cli: https://formulae.brew.sh/formula/azure-cli

Spec-Driven Development for AI-Enabled Enterprise Systems

Lee_Stott — Mon, 18 May 2026 18:40:17 GMT

Spec-Driven Development for AI-Enabled Enterprise Systems

How to make specs the single source of truth for your React frontends, backend services, data, and AI agents.

If you are building an enterprise system with a React frontend, backend APIs and services, a database layer, and shared libraries, moving to Spec-Driven Development (SDD) can feel like a big cultural shift. For AI developers and engineers, though, it is a gift: structured, machine-readable specifications are exactly what both humans and AI coding agents need to stay aligned and productive.

This post walks through how to structure specs, version contracts, design workflows, and integrate AI agents in a way that scales. Along the way, it references Microsoft’s public guidance on microservices, APIs, DevOps, and architecture so you can go deeper where needed.

1. Structuring specifications for an enterprise system

For a serious enterprise system, treat specs as layered and modular rather than a single monolithic document. A good mental model is Domain-Driven Design (DDD) and bounded contexts (see https://learn.microsoft.com/azure/architecture/microservices/model/domain-analysis

Business and domain layer

This layer is technology-agnostic and captures:

Business capabilities and problem statements
Domain language and key entities
Business rules and workflows
Non-functional requirements (performance, security, compliance, SLAs)

Solution and architecture layer

Here you define how the system is shaped:

System context and C4-style diagrams
Service boundaries and ownership
Integration patterns and event flows
Data ownership and high-level models

Microsoft’s microservices guidance is a solid reference: https://learn.microsoft.com/azure/architecture/microservices/.

Implementation-oriented specs per component

For each concrete component, keep a focused spec:

Frontend / UI (React): screen catalogue, UX flows, state contracts, API dependencies, validation rules, accessibility and performance requirements.
APIs / services: OpenAPI or AsyncAPI contracts, error models, authentication and authorisation, rate limits, SLAs, observability requirements.
Database / schema: logical data model, ownership per service, migration strategy, retention, indexing, partitioning.
Shared libraries: responsibilities, versioning policy, supported runtimes, compatibility matrix.
Integrations: protocols, payloads, sequencing, idempotency, retry and backoff, SLAs, failure modes.

In practice, this usually means:

One “master” business and architecture spec per domain or product
Separate specs per service or module (frontend app, each backend service, shared library, integration)
Everything linked via IDs (for example REQ-123, SVC-ORDER-001) so you can trace from requirement to spec, implementation, and tests

2. Templates and standards that scale

To keep things consistent across teams, use a base template that all components share, then extend it with technology-specific sections. This works well for both human readers and AI agents consuming the specs.

Base specification template

Every spec, regardless of component type, should include:

Purpose and scope
Stakeholders and dependencies
Requirements mapping (list of requirement IDs covered)
Architecture and interaction overview
Contracts (APIs, events, data)
Non-functional requirements
Risks and open questions
Test and acceptance criteria

Extended templates per component

Frontend: UX flows, wireframes or Figma links, accessibility, performance budgets, offline behaviour, error states.
API / service: OpenAPI or AsyncAPI link, auth and authorisation, throttling, logging and metrics, health endpoints. See logging and monitoring guidance at https://learn.microsoft.com/azure/architecture/microservices/logging-monitoring
Database: schema definition, migration plan, backup and restore, data lifecycle, multi-tenant strategy.
Integration: sequence diagrams, error handling, retry and idempotency, message contracts, security.

3. Contracts, versioning, and change management

API contracts

For SDD, API contracts are first-class citizens. Define them via OpenAPI or AsyncAPI and treat the spec as the source of truth. Use contract testing to keep providers and consumers aligned, and version APIs explicitly (for example v1, v2) rather than breaking changes in place. Microsoft’s API design guidance is a good starting point: https://learn.microsoft.com/azure/architecture/best-practices/api-design and Azure API Management at https://learn.microsoft.com/azure/api-management/.

Database migrations

Any spec change that affects data should include a migration plan. Use migration tooling such as EF Core migrations, Flyway, or Liquibase, and treat migration scripts as code. Document backward-compatibility windows so APIs can support both old and new fields for a defined period.

Shared DTOs and models

Prefer sharing contracts (OpenAPI, JSON Schema) over large shared code libraries. If you must share code, version the shared library independently and document compatibility (for example, “Service A supports SharedLib 2.x”). Keep DTOs at the edges and map to internal domain models inside each service.

Cross-service dependencies

Capture dependencies explicitly in specs, such as “Order Service depends on Customer v1.3+ for endpoint /customers/{id}”. Use consumer-driven contracts and CI checks to prevent breaking changes. For event-driven systems, document event contracts and evolution rules. See event-driven architecture guidance at https://learn.microsoft.com/azure/architecture/reference-architectures/event-driven/event-driven-architecture-overview.

Spec versioning and change management

Version specs semantically (for example OrderServiceSpec v1.2.0) and record what changed, why, impact, and migration steps. Link spec versions to releases or tags in Git and to work items in Azure DevOps or GitHub Issues. Azure Boards is useful here: https://learn.microsoft.com/azure/devops/boards/?view=azure-devops.

4. A mature Spec-Driven Development workflow

A realistic SDD workflow for AI-enabled teams might look like this:

Discovery and domain analysis: capture business capabilities, domain language, and high-level workflows.
Business and architecture specs: define bounded contexts, service boundaries, integration patterns, and NFRs.
Contract design: design API specs (OpenAPI or AsyncAPI), event schemas, data models, and validation rules.
Task generation: derive work items from specs, such as “Implement endpoint X”, “Add migration Y”, “Add UI flow Z”. This is a great place to use AI agents to read specs and generate tasks.
Implementation: code is generated or written to satisfy the spec; the spec remains the reference, not the code.
Validation and testing: contract tests, unit tests, integration tests, and end-to-end tests all trace back to spec IDs. Use quality gates in CI and CD, as described in Https://learn.microsoft.com/azure/architecture/framework/devops/devops-quality
Review and sign-off: architecture and product review against the spec; update the spec if reality diverges.
Release and observability: dashboards and alerts tied to specified SLIs and SLOs.

5. Governance, traceability, and avoiding drift

Traceability across the lifecycle

Use IDs everywhere: requirements, spec sections, tasks, tests, and deployment artefacts. In Azure DevOps or GitHub, link:

Requirement (for example Azure DevOps Feature)
Spec (stored in the repo)
User stories and tasks
Pull requests
Tests
Releases

For key decisions, adopt Architecture Decision Records (ADRs). Microsoft’s guidance on ADRs is here: Https://learn.microsoft.com/azure/architecture/framework/devops/adrs

Keeping humans and AI agents aligned

To avoid implementation drift:

Make specs as machine-readable as possible (OpenAPI, JSON Schema, YAML, BPMN).
Enforce spec checks in CI: API implementation must match OpenAPI, DB schema must match migration plan, generated clients must be up to date.
For AI coding agents, always provide the relevant spec files as context and constrain them to files linked to specific spec IDs.
Add automated checks that compare generated code to contracts and fail builds when they diverge.

6. Enterprise best practices for repos and governance

Example repository structure

/docs
  /business
  /architecture
  /decisions (ADRs)
/specs
  /frontend
  /services
    /orders
    /customers
  /integrations
  /data
/src
  /frontend
  /services
  /shared
/tests
/ops
  /pipelines
  /infra-as-code

Governance practices

An architecture review group that reviews spec changes, not just code changes.
Definition of Done includes: spec updated, tests linked, contracts validated.
Regular “spec health” reviews to identify what is out of date or drifting.

For broader architectural guidance, see:

Azure microservices and DDD: https://learn.microsoft.com/azure/architecture/microservices/
Cloud design patterns: https://learn.microsoft.com/azure/architecture/patterns/
Azure Well-Architected Framework: https://learn.microsoft.com/azure/well-architected/

7. Integrating AI and agentic workflows into SDD

Spec-Driven Development is a natural fit for AI and multi-agent systems because specs provide structured, reliable context. Here are some practical patterns.

LangGraph and multi-agent orchestration using Microsoft Agent Framework

You can design a graph where:

A “spec agent” reads and validates specs.
An “implementation agent” writes or updates code based on those specs.
A “test agent” generates tests from contracts and acceptance criteria.

The graph flow can mirror your SDD workflow: Spec → Contract → Code → Tests → Review, with each agent responsible for a stage.

MCP (Model Context Protocol)

Expose your spec repository, OpenAPI definitions, and ADRs as MCP tools so agents can query the true source of truth instead of hallucinating. For example, provide a tool that returns the OpenAPI for a given service and version, or a tool that returns the ADRs relevant to a particular domain. Learn more about MCP at https://aka.ms/mcp-for-beginners

BPMN and process flows

Store BPMN diagrams as part of the spec. Agents can read them to generate workflow code, state machines, or tests. For process-oriented integrations, see Azure Logic Apps guidance at https://learn.microsoft.com/azure/logic-apps/.

CI/CD pipelines on Azure

In your pipelines, validate that implementation matches the spec:

Contract tests for APIs and events
Schema checks for databases
Linting and static analysis for spec conformance

Use pipeline gates to block deployments if contracts or migrations are out of sync.
Azure Pipelines https://learn.microsoft.com/azure/devops/pipelines/?view=azure-devops
GitHub Agentic Workflow Patterns https://github.github.com/gh-aw/

Where to start

The key is not to boil the ocean. Pick one domain, such as “Orders”, and design a thin but end-to-end SDD flow: spec → contract → tasks → code → tests. Run it with your AI agents in the loop, learn where the friction is, and iterate. Once that feels natural, you can roll the patterns out across the rest of your system.

For AI developers and engineers, SDD is more than process hygiene. It is how you give your agents high-quality, unambiguous context so they can generate code, tests, and documentation that actually match what the business needs.

Building the Solution Teams Need to Secure AI Against Prompt Injection

teo-montero — Mon, 25 May 2026 17:24:52 GMT

As artificial intelligence continues to evolve, teams are prioritising rapid advancements and deployment of applications while often overlooking security considerations. Emerging threats such as prompt injection remain poorly understood, and this is putting systems, users, and infrastructure at serious risk.

Much of the expertise required to mitigate these risks is currently fragmented and inaccessible, concentrated among a small group of cybersecurity specialists. Meanwhile, developers, under pressure to ship quickly, often lack both the tools and frameworks needed to systematically test their AI systems for vulnerabilities.

This disconnect is creating a significant gap between the development and security assurance of AI applications.

To address this gap, we developed a unified Prompt Injection Testing Platform and knowledge base, powered by Microsoft Foundry, designed to make LLM security testing accessible, structured, and understandable for developers.

Project Overview

Developers are rapidly integrating LLMs and agents into applications, but:

Security testing is not standardised
Prompt injection risks are increasingly understood in research, but poorly mitigated in practice by developers
There is a lack of accessible, actionable tooling

This creates a dangerous gap: applications are being deployed faster than they are being secured.

As part of our UCL Industry Exchange Network (IXN) project in collaboration with Avanade, we built a Prompt Injection Testing Platform designed to solve this exact issue by:

Providing a knowledge base of vulnerabilities and mitigations
Helping teams identify vulnerabilities within their AI systems
Enabling custom and automated testing pipelines
Integrating tools like Garak for adversarial testing

With this, we aim to make prompt injection testing accessible, standard, and understandable.

Project Journey

We divided our project into several phases:

Phase 1: Understanding our Users’ Needs.

We began by identifying the core users of our platform: AI developers and broader stakeholders across development, security, and safety disciplines integrating LLMs into their applications. By meeting with them, we uncovered a few key challenges:

Developers have limited awareness of prompt injection risks
There is a generalised lack of accessible tools for testing

This first exploration set a core principle: We must build a developer-first solution which does not depend on extensive technical knowledge to be used. We concluded that to be as useful as possible, our solution should not require prior prompt injection knowledge.

In order to solve the two challenges presented by our users, we concluded a platform would be the best approach, as it enables us to centralise fragmented knowledge while providing a structured, scalable environment for testing LLM vulnerabilities in practice.

Phase 2: Understanding the Threat Landscape

Building on our user research, we focused on developing a deep understanding of the prompt injection threat landscape to inform the design of our platform. This phase involved researching:

Different types of prompt injection vulnerabilities
Common attack scenarios and override techniques
Existing mitigation strategies used in practice
Tools and methodologies for prompt injection security testing
The most widely used models to ensure our platform would be compatible with real-world systems.
We consolidated these findings into a structured technical report, designed to be shared with developers, security testers, and semi-technical stakeholders. The goal was not only to guide our own implementation, but also to contribute to making prompt injection more standard and understandable.

From our research, we realised prompt injection is not a single vulnerability, but a rapidly evolving attack surface that requires continuous, scalable testing rather than one-time validation.

Phase 3: Building the Platform

Guided by both our user insights and the threat landscape analysis, we moved to designing and developing a unified prompt injection testing platform and knowledge base.

To do this, we defined three core principles:

Developer first: no deep security knowledge would be required
Unified: combines education (knowledge base) and execution (testing tools)
Scalable: Expert users could extend the platform by bringing their own models, tests, and mitigations.

During this stage, we built a platform which allows teams to:

Connect their own LLM endpoints
Run custom prompt injection tests
Execute automated adversarial testing through Garak
Access a centralised knowledge base of vulnerabilities and mitigation strategies.
Export knowledge base information and test results as PDFs.

By the end, we had developed a unified platform that enables developers to systematically test, understand, and mitigate prompt injection vulnerabilities in their AI applications.

To understand how our platform works in practice, you can view our demo video.

Figure 4: Platform home interface presenting an overview of prompt injection concepts and a structured vulnerability catalogue for exploring attack types and mitigation strategies.

Key Features

Model Integration and Configuration

Users can use models included in the platform or connect their own LLM endpoints, allowing the platform to work across different providers:

Supports multiple model providers through Microsoft Foundry
Supports custom model integration via HTTP endpoints
Enables model configurations such as custom system prompts and mitigation layers.
Ensures flexibility as new models and mitigations emerge

Testing Suite

The platform allows users to create and run custom prompt injection tests tailored to their applications. This involves:

Creating and executing targeted prompts
Simulating real-world attack scenarios
Running predefined adversarial testing suites (integrating NVIDIA Garak)

Figure 2: Testing interface showing configuration of prompt injection tests and execution of automated scans, with results and risk evaluation displayed.

Knowledge Base

A core component of our platform is a structured knowledge base, which is designed to make prompt injection concepts accessible and understandable. This is divided into two key areas:

Vulnerabilities: Provides information on different types of prompt injection attacks, including explanations of how each vulnerability works, with real-world examples and scenarios, as well as references to reputable external sources
Mitigations: Focuses on how to defend against these vulnerabilities, and it includes clear implementation strategies and code examples demonstrating how to integrate each mitigation.

To support exploration, we also included a chatbot interface, which answers questions using knowledge base data and trusted sources. This helps users quickly navigate vulnerabilities and mitigation strategies by providing contextual, reliable information and redirecting users to the appropriate page of our platform.

Figure 3: Direct prompt injection analysis view, where users can explore attack techniques, observe unsafe model responses, and review corresponding mitigation approaches.

Prompt Enhancer

In addition to testing and learning, our platform integrates a prompt enhancer, designed to help users actively improve the security of their system prompts. It works in the following way:

Takes an existing prompt as input
Draws on the knowledge base insights and best practices
Restructures the prompt to improve clarity and robustness
Incorporates selected prompt-layer mitigations to reduce prompt injection risk

Figure 4: Prompt Enhancer interface showing the application of prompt-layer mitigations (e.g. delimiter tokens, instruction hierarchy enforcement) to restructure and secure a system prompt against prompt injection attacks.

Technical Details

To support a flexible and scalable testing system, we designed our platform with a modular, layered architecture. This allows different components to operate independently while remaining integrated through clearly defined interfaces, ensuring both extensibility and maintainability.

System Architecture

We divided our platform into four main layers:

Frontend Layer

An interactive user interface that allows developers to:

Explore the prompt injection knowledge base
Configure and run tests
View results and vulnerability analysis

API Layer

The API layer acts as the orchestration and communication layer between the frontend and the core system.

Handles requests from the frontend to create and run tests.
Provides frontend with available models, mitigations, and configurations.
Ensures any newly added models and mitigations can be automatically reflected in the frontend without requiring manual updates.

Domain Layer

The layer which defines the core structure and logic of the system:

Defines interfaces for key components such as mitigations, models, and test runners
Establishes the test structure and data models
Encapsulates logic to ensure consistency

Integration Layer

The layer which implements the abstractions defined in the domain layer and connects the platform to external services

Implements model providers such as OpenAI, Anthropic, and other external HTTP-based endpoints
Implements test runners, including custom prompt runners and external tools such as Garak.
Implements database connections and repository classes.

Results and Outcomes

Through the research and development of our platform, we were able to gain several key insights into the behaviour and security of LLM-based applications:

Prompt injection vulnerabilities are more prevalent than expected. Even simple prompts with carefully crafted inputs can unsafely manipulate a model’s behaviour.
Lack of structured testing leads to hidden risks. Without a systematic approach, many vulnerabilities remain undetected. It is sometimes time consuming to manually craft unsafe prompts.
Combining custom testing with framework-based testing improves coverage. Using both custom prompts (targeted and application-specific scenarios) and framework-driven testing (e.g. Garak) enables a more comprehensive evaluation of model safety, as both expected and unexpected vulnerabilities can be captured
Structured prompts can significantly improve robustness. We observed that prompts with a clear structure and embedded mitigations are less susceptible to injection attacks.

By the end of our project, we successfully developed a platform that:

Bridges the gap between prompt injection knowledge and practical testing.
Enables repeatable and structured testing of prompt injection vulnerabilities
Provides a unified workflow for learning, testing, and improving prompt security.
Supports multiple models and testing approaches, to cover the entire vulnerability safety.

We demonstrated that prompt injection risks can be systematically identified, tested, and mitigated through a structured and repeatable approach.

Lessons Learned

Throughout the project, we identified several key insights that shaped both our technical approach and our understanding of AI security.

AI is rapidly evolving, and systems must be designed accordingly. AI models and attack techniques are advancing extremely fast. As a result, static solutions are quickly becoming obsolete. We learned that it is essential to design a platform that is modular, extensible and adaptable. Through well-defined interfaces and generic services, we ensured our platform can evolve alongside attacks and mitigations.

Security must be built into development, not considered at testing. Many developers are focusing on functionality first and security often takes a backseat. In the context of LLMs, vulnerabilities can fundamentally affect the security of the system and its users. As such, security should be treated as a core part of the development cycle. Models and external tools should only be connected if their safety is guaranteed.

Bridging the gap between developers and security testers is necessary. We identified a major disconnect between developers building AI applications and the security testers evaluating them. These groups often operate with different priorities and levels of knowledge. We are bridging this gap by making prompt injection knowledge more accessible and creating workflows that are usable by developers while still grounded in robust security practices.

Further Development

While our platform provides a strong foundation for prompt injection testing and knowledge, there are several areas for future exploration:

Expanding our testing framework integrations, by adding a broader coverage of attack techniques
Integration with MCP servers and external systems, supporting interactions with tools, APIs and external data sources.
Addressing additional indirect prompt injection vulnerabilities, including file uploads, website scraping, and multi-step workflows.

Looking ahead, we also aim to integrate our platform more deeply into development workflows by introducing CI/CD integrations for continuous security testing and versioned tracking of model robustness over time.

Our goal is to evolve the platform into a comprehensive security layer, capable of testing entire AI-driven systems in dynamic, real-world contexts.

Conclusion

As AI becomes increasingly integrated into real-world applications, ensuring their security is essential. As our research highlights, current practices have not kept pace with the rapid evolution of AI systems and attack techniques.

Through our work, we demonstrated that prompt injection risks can be systematically identified, tested, and mitigated using a structured approach. By combining a unified knowledge base with a flexible testing platform powered by Microsoft Foundry, we are taking a step towards making AI systems safer and more reliable.

More importantly, our project reinforces a broader idea: a developer-first approach to security, supported by collaboration across development, security, and safety disciplines, is essential for building AI at scale. Security should not remain confined to specialist teams but should be embedded directly into the development process, alongside practices such as red-teaming and continuous testing.

Our project empowers teams with the knowledge and tools they need to build safer and more reliable AI systems. If you’re interested in building more secure AI systems or exploring prompt injection in practice, we invite you to join us through the Foundry Community on the 3rd of June at 2pm BST, when we will be showcasing our platform live, walking through real-world examples, and discussing how teams can integrate prompt injection testing into their development workflows.

Team

Teo Montero Bonet, UCL Computer Science

Mario Mojarro Ruiz, UCL Computer Science

David Thomas Garcia, UCL Computer Science

Nathaniel Gibbon, UCL Computer Science

With support from Josh McDonald, Avanade

Stop Hallucinating, Start Evaluating — A Tour of AgentEval

joslat — Wed, 13 May 2026 06:12:50 GMT

Your AI agent works great… until it doesn't. AgentEval catches the failures before your users do.

Your AI agent works great… until it doesn't. AgentEval catches the failures before your users do.

The biggest problem for Agentic Engineers

Shipping an AI agent feels great — until you hit the wall every Agentic Engineer eventually hits: non-determinism. In September 2025 OpenAI researchers published "Why Language Models Hallucinate" (Kalai, Nachum, Vempala, Zhang). The punchline matches what every agent developer has lived through at 2 a.m.: even the best models confidently produce plausible falsehoods. So how do we — .NET developers shipping production AI agents — actually KNOW our agents work?

Four questions a traditional unit test cannot answer:

Did the agent call the right tools, in the right order, with the right arguments?
Is the answer actually grounded in the retrieved context — or is the model making it up?
Will it survive a prompt injection next Tuesday?
And — if we repeat this execution — will it work consistently every single time?

That last question is the one that keeps me up at night. It's the one that started this whole journey for me.

What is AgentEval?

AgentEval is the .NET evaluation toolkit for AI agents. Open-source, MIT-licensed, available on NuGet, and built first for Microsoft Agent Framework (MAF) and Microsoft.Extensions.AI. What RAGAS and DeepEval do for Python, AgentEval does for .NET — with the fluent assertion APIs we actually expect.

AgentEval Public Beta — built on MAF and MEAI.

Today it ships with over 30 runnable samples, 192 red-team probes covering 6 of the 10 OWASP LLM Top 10 categories, 5 memory metrics (more on that later — it's the "one more thing" of this post), and exports to JSON, JUnit XML, Markdown, SARIF and PDF.

Docs live at https://agenteval.dev. Public Beta — moving fast, APIs may shift, but already battle-tested by the .NET AI community.

Why I built AgentEval

Honest version: AgentEval started as a side effect of another side project.

I was building an auto-investment system for fun. Aggregate financial news, market data, a couple of feeds — then asked an agent to produce a "bull/bear" assessment on individual stocks. Sometimes it worked. Sometimes — same prompt, same data — a completely different conclusion. Worse, the inconsistencies didn't fail loudly. They quietly biased the next decision.

So I went down the loophole, as we say. I had to understand why a state-of-the-art model gave different answers to the same question — and how to systematically catch that before it touched real positions.

Talking AI non-determinism — the talk that started it all.

I started small. Tool-call assertions. RAG faithfulness. Then stochastic runs — because the real question was never "did it pass?" but "how often does it pass?". Then model comparison. Then red teaming. It kept growing.

In mid-December 2025 I made the call: I paused the auto-investment system (still on halt — I'll come back to it one day) and went all-in on AgentEval.

The ecosystem challenge

Here's the bit that bothered me the most. Python had RAGAS, DeepEval, PromptFoo. The Python community had a whole shelf of mature evaluation libraries to pick from. .NET had… nothing comparable. The community deserved better. And honestly, I needed it for my own work.

Building WITH the Microsoft Agent Framework team — not just FOR them.

One quick credibility note: the MAF integration didn't ship by accident. It came from a deep dive — the official docs, a 12,000+ line source-code audit of MAF itself, and direct collaboration with the framework team. The rigor is intentional.

Today AgentEval is open source, MIT-licensed, and built with love for the .NET AI community.

Built for the Microsoft AI Stack

Three reasons MAF and MEAI developers care:

Native first-class integration. AgentEval works directly with AIAgent and IChatClient. Tool calls are tracked automatically. Token counts, cost estimates, latencies — captured for you. No manual wiring.

Universal adapter for any provider. One line:

var agent = chatClient.AsEvaluableAgent( name: "PolicyAssistant", systemPrompt: SYSTEM_PROMPT); services.AddAgentEval(); // DI-friendly setup

That chatClient can be Azure OpenAI, OpenAI, Foundry Local, Ollama, Groq, vLLM — anything that speaks IChatClient. Same evaluation code, swappable provider.

Record once. Replay forever. Trace recording captures a full execution as JSON. Replay it deterministically in CI — no live API calls, no surprise costs, identical results every run. This is how AI evaluations finally become reproducible.

Semantic Kernel users aren't left out — there's a bridge via AIFunctionFactory.Create() so SK tools surface in AgentEval just like MAF tools.

The Capability Tour

Tool chains, asserted.

Tool calls are where agents earn their keep — and where they fail in subtle ways. AgentEval gives us a fluent grammar to assert on them:

result.ToolUsage!.Should() .HaveCalledTool("AuthenticateUser", because: "security first") .BeforeTool("FetchUserData") .WithArgument("method", "OAuth2") .And() .HaveCalledTool("SendNotification").AtLeastTimes(1) .And() .HaveNoErrors();

No more grep-ing logs to confirm a function fired. The assertions read like requirements — because that's exactly what they are.

Is your agent hallucinating?

Use a separate evaluator LLM to score the response against the retrieved context — faithfulness, relevance, correctness.

var faithfulness = await new FaithfulnessMetric(evaluator).EvaluateAsync(context); if (faithfulness.Score < 70) throw new HallucinationDetectedException($"Faithfulness: {faithfulness.Score}");

The trick is using two IChatClients — one for the agent, a separate one for the judge. The judge has clean context, so it can't be biased by the original conversation.

Stochastic evaluation, or: the bull/bear problem

Remember my auto-investment system? Same prompt, different answers. That isn't a bug — that's how LLMs are. So let's stop asking "did it pass?" and start asking "how often does it pass?"

var result = await stochasticRunner.RunStochasticTestAsync(agent, testCase, new StochasticOptions(Runs: 20, SuccessRateThreshold: 0.85, ScoreThreshold: 75)); result.Statistics.Mean.Should().BeGreaterThan(80); result.Statistics.StandardDeviation.Should().BeLessThan(10);

Run it twenty times. Assert on pass rate and standard deviation. The evaluation that never flakes — because pass/fail isn't based on a single lucky run, it's based on the rate. That is the answer to my bull/bear problem.

And here's where it gets interesting: I've used this exact technique to figure out which agentic architectures are more deterministic than others — what makes some agents stable and others wobble. You can do the same. Apply your own hypotheses — "does adding a planner reduce variance?", "is GPT-4o-mini consistent enough for this task?", "does my system prompt fix the spread?" — run them through AgentEval, and engineer your way to a more deterministic agent. This is what real Agentic Engineering looks like.

Stochastic evaluations

Stochastic evaluations

Pick the right model — with evidence.

"GPT-4o or GPT-4o-mini?" used to be a vibes-based debate. Now it's a function call:

var result = await comparer.CompareModelsAsync( factories: new[] { gpt4o, gpt4oMini, claude }, testCases: testSuite, metrics: new[] { new ToolSuccessMetric(), new RelevanceMetric(eval) }, options: new ComparisonOptions(RunsPerModel: 5)); Console.WriteLine(result.ToMarkdown());

Model comparison output — winner + best-value recommendation

The output is a leaderboard — pass rate, average score, latency, cost — plus a recommendation. "Why are we using GPT-4o?" Answer: this report.

Latency and cost — as code.

result.Performance!.Should() .HaveFirstTokenUnder(TimeSpan.FromMilliseconds(500)) .HaveEstimatedCostUnder(0.05m) .HaveTokenCountUnder(2000);

If your UX needs sub-500ms time-to-first-token, write it down. Make it fail the build. SLAs as executable evaluations — that's the dream.

Red team your agent before users do.

192 probes across 9 attack types — prompt injection, jailbreaks, PII leakage, excessive agency, insecure output handling, system prompt extraction, indirect injection, inference abuse, encoding evasion. Coverage: 6 of the 10 OWASP LLM Top 10 (2025), mapped to 6 MITRE ATLAS techniques.

// One line. var result = await agent.QuickRedTeamScanAsync(); result.Should() .HavePassed() .And().HaveMinimumScore(80) .And().HaveASRBelow(0.05); // Attack Success Rate under 5%

Need a full pipeline? Use AttackPipeline.Create() with specific attacks and intensities. Need to ship findings to the GitHub Security tab? Export to SARIF. Need a PDF for the security review board? Export to PDF.

Does this actually work in the wild? Yes. Jernej Kavka (Microsoft MVP, SSW) tried AgentEval against one of his agents in a single evening. The initial system prompt let through a 40% attack success rate. After tightening the prompt and re-running the red-team scan, he brought it to zero. One evening. That is the difference between catching this in dev and learning about it from a postmortem. Link to his experience blog at the bottom references.

Toxicity, bias, misinformation.

Responsible AI isn't an afterthought. Counterfactual bias testing — the methodology academics will recognise — is one line:

var bias = await new BiasMetric(evaluator) .EvaluateCounterfactualAsync(originalContext, counterfactualContext, "gender");

Toxicity and misinformation metrics live right next to it. Run them in the same suite as your quality metrics, because they ARE quality metrics.

Multi-agent flows, as assertions.

Agents don't live alone anymore. MAF's WorkflowBuilder graphs route work between multiple executors — and AgentEval asserts against the whole graph.

var testCase = new WorkflowTestCase { Name = "TripPlanner", ExpectedExecutors = ["TripPlanner", "FlightReservation", "HotelReservation", "Presenter"], StrictExecutorOrder = true, ExpectedTools = ["SearchFlights", "BookHotel"], }; var result = await new WorkflowEvaluationHarness() .RunWorkflowTestAsync(workflowAdapter, testCase); result.ExecutionResult!.Should() .HaveExecutedInOrder("TripPlanner", "FlightReservation", "HotelReservation", "Presenter") .HaveTraversedEdge("TripPlanner", "FlightReservation") .HaveAnyExecutorCalledTool("SearchFlights") .HaveCompletedWithin(TimeSpan.FromMinutes(2));

Four agents. Real WorkflowBuilder. One test. Edge traversal, executor order, per-graph tool calls — all observable, all assertable. Try doing that in Python.

Don't write C#? Use the CLI.

Not every reader is a .NET developer — and AgentEval has a no-code path for everyone else. The CLI runs against any OpenAI-compatible endpoint, fits inside your CI/CD pipeline, and gives you the same evaluations the SDK does. Install once:

dotnet tool install -g AgentEval.Cli --prerelease

1. Scaffold a test dataset

agenteval init -o agenteval.yaml

This drops a YAML template you can edit — inputs, expected behaviours, evaluation criteria. Treat it like any other test file: commit it, version it, code-review it.

2. Run an evaluation

# Against Azure OpenAI agenteval eval --azure \ --endpoint https://myresource.openai.azure.com/ \ --deployment-name gpt-4o \ --dataset agenteval.yaml --runs 5 --success-threshold 0.9 # Against OpenAI directly agenteval eval --endpoint https://api.openai.com/v1 \ --model gpt-4o --dataset agenteval.yaml # Against a local Ollama model — no API key needed agenteval eval --endpoint http://localhost:11434/v1 \ --model llama3 --dataset agenteval.yaml

Works with Azure OpenAI, OpenAI, Foundry Local, Ollama, Groq, vLLM, LM Studio, Together.ai — anything that speaks the OpenAI protocol. Same dataset, swappable provider.

3. Run a red-team scan

# All 9 attack types at moderate intensity agenteval redteam --azure \ --endpoint https://myresource.openai.azure.com/ \ --deployment-name gpt-4o --intensity moderate # Or pick specific attacks and export SARIF for GitHub Security agenteval redteam --azure \ --endpoint https://myresource.openai.azure.com/ \ --deployment-name gpt-4o \ --attacks PromptInjection,Jailbreak --format sarif

The CLI exports to JSON, Markdown, JUnit XML, SARIF and PDF — so you can wire results into GitHub Actions, Azure DevOps, dashboards, or compliance reports. And here is the real point: the sooner we find these issues, the cheaper they are to fix. Shifting evaluation left — into the CLI, into the build, into the pull request — is the difference between a 5-minute fix and a 5-day incident.

Shift-left for AI quality

Behavioral policies and compliance — as code, in your test suite.

Evaluations belong in your build, not in a notebook. Here is how that looks in practice:

Write evaluations as xUnit tests — they run on every PR alongside your unit tests.
Export JUnit XML — results show up in the GitHub Actions / Azure DevOps test tab like any other test.
Export SARIF — red-team findings land in the GitHub Security tab automatically.
Auto-saved trace artifacts on failure — record once, replay forever, debug at leisure.

Catch hallucinations, tool-call regressions, and prompt-injection vulnerabilities before they reach production. That is shift-left for AI quality in .NET — and the sooner you do it, the cheaper every fix gets.

And one more thing… memory.

First AgentEval memory component and dynamic report showcase, at the MVP Summit

First memory component and dynamic report showcase, at the MVP Summit

Every serious agent now has "memory" — context providers, chat history compaction, cross-session recall. But here is the uncomfortable question: does it actually remember?

When your compaction strategy drops a fact at the 50K-token mark, do you notice? When your agent confidently contradicts something the user said three turns ago, can you catch it before they do?

Does your agent actually remember? Or is it just guessing convincingly?

Enter AgentEval.Memory — a complete toolkit for evaluating retention, recall depth, temporal reasoning, fact updates, cross-session persistence and resistance to noise. Five memory metrics, five benchmark presets (Quick → Overflow at 192K tokens), and an HTML pentagon report that overlays multiple models for side-by-side comparison.

The headline: LongMemEval — the ICLR 2025 academic memory benchmark — is fully re-implemented in .NET. That means .NET teams can produce paper-comparable scores against the same benchmark academics are publishing on. For coursework or research, this matters.

var runner = MemoryBenchmarkRunner.Create(chatClient); var result = await runner.RunBenchmarkAsync(agent, MemoryBenchmark.Standard); Console.WriteLine($"Memory: {result.OverallScore:F1}% ({result.Grade})"); await result.ExportHtmlReportAsync("memory-report.html");

MAF-native — it plugs into AIContextProvider, ChatHistoryProvider and CompactionStrategy out of the box. Wire it up and find out what your agent actually forgets.

The community is already using it

Jernej Kavka (Microsoft MVP, SSW) wrote up his experience getting started with AgentEval — Testing .NET AI Agents with AgentEval. In one evening he caught real issues in his agents before they had a chance to escape — and walked through hardening his system prompt until the red-team attack success rate dropped from 40% to zero. Read the post; it is a great hands-on introduction.

Want a full end-to-end sample? Check out ECS2026MAF in the samples folder. Or run all the samples in one shot:

dotnet run --project samples/AgentEval.Samples

I have also taken this talk on the road — "Stop Hallucinating, Start Evaluating" at the MVP Summit, ECS 2026 and EMPOWER conferences and a handful of community events. If you want to bring it to your .NET user group, your university lab, or your faculty seminar — get in touch.

Try AgentEval in 5 minutes

★ Five minutes, four commands

1. Install the CLI:

dotnet tool install -g AgentEval.Cli --prerelease

2. Scaffold a test dataset:

agenteval init -o agenteval.yaml

3. Run an evaluation:

agenteval eval --endpoint https://api.openai.com/v1 --model gpt-4o --dataset agenteval.yaml

4. Run a red-team scan:

agenteval redteam --endpoint ... --model gpt-4o --intensity quick

Where to go next

⭐ Star the repo: https://github.com/AgentEvalHQ/AgentEval
🧰 CLI source: https://github.com/AgentEvalHQ/AgentEval.Cli
📖 Read the docs: https://agenteval.dev
📦 NuGet: https://www.nuget.org/packages/AgentEval
🎓 Adopt AgentEval in your courses or research labs — fluent, type-safe, LongMemEval-comparable.
🤝 Contribute — issues, attack probes, samples, ideas. Pull requests are welcome.
💬 Discuss — questions and feedback on https://github.com/AgentEvalHQ/AgentEval/discussions

Stop guessing if your AI agent works. Start proving it. And use AgentEval to make them work better.

City simulator with AI agents for traffic congestion

Shoib — Mon, 11 May 2026 07:00:00 GMT

Project Overview

Hi I'm Shoib Muhammad an MEng Mathematical Computation student at UCL, I set out to tackle one of the most persistent challenges in modern cities: traffic congestion. In London alone, drivers lose around 141 hours each year to congestion, with the wider UK economy losing approximately £5.1 billion in productivity. Traditional fixed-cycle traffic lights cannot adapt to changing demand, and even adaptive systems such as SCOOT and SCATS tend to focus on single intersections rather than understanding the full network.

The aim of this project was to build something practical and experimental: a high fidelity simulation environment where multi agent AI systems can be tested, broken, and improved without affecting real world traffic. It is a safe space to explore how intelligent systems could one day manage entire city networks.

The Stack

The project brings together a mix of gaming, backend, and AI technologies. Microsoft Foundry using GPT-4o handles both vision and reasoning, allowing the system to count vehicles from camera feeds and make decisions from a single cloud endpoint. The Microsoft Agent Framework enables multi agent coordination, structuring a pipeline from camera input through to intersection decisions and orchestration, with up to 20 agents running in parallel. Unreal Engine 5 provides the simulation itself, modelling a 20 intersection city with realistic vehicle movement and camera views. FastAPI acts as the backend, linking everything together with schema validation, a polling loop, and a deterministic testing suite to ensure reliability.

Technology	Core Function	Impact
Microsoft Foundry (GPT-4o)	Vision + Reasoning	Counted vehicles from camera frames and made per-intersection decisions through one cloud endpoint.
Microsoft Agent Framework	Multi-Agent Orchestration	Structured the Camera → Intersection → Orchestrator pipeline and fanned out 20 parallel agent runs per cycle.
Unreal Engine 5	3D Simulation Environment	Hosted the 20-intersection city, vehicle physics, and per-intersection cameras.
FastAPI	Backend & Schema Validation	Bridged UE5 and Azure with a polling loop, schema contracts, and a 32-test deterministic suite.

How a Decision Cycle Flows

The simulation models a small city with 20 intersections. Each decision cycle is powered by three types of AI agents working together. A Camera Agent acts as the perception layer, using GPT-4o vision to count vehicles on each road approaching an intersection. Each intersection is controlled by its own agent, which evaluates traffic pressure and decides whether to change the signal phase. An Orchestrator Agent then coordinates these decisions across the network, ensuring that neighbouring intersections do not switch at the same time and cause gridlock.

At the start of each cycle, Unreal Engine generates screenshots from each intersection along with traffic flow data. The backend processes and validates this data, then sends it to the Camera Agent. The output is distributed to all intersection agents in parallel, and their decisions are combined by the orchestrator. The final decision is written back into the simulation, updating the traffic lights. If anything fails, the system simply holds its current state and retries on the next cycle, making the system robust even with non deterministic AI outputs.

Camera Agent – the perception layer, using GPT-4o vision to count vehicles on every approach of every intersection.
Intersection Agents (×20, parallel) – each owns a single junction, evaluates queue pressure and recommends whether to switch the signal phase.
Orchestrator Agent – the coordinator, applying an adjacency-aware rule that prevents two neighbouring intersections from switching simultaneously and dumping traffic into each other's red phases.

Agent Topology

When a decision cycle begins, UE5 writes 20 PNG screenshots and a flow-direction JSON to a shared directory. The FastAPI backend polls the directory, validates the batch against a 20-unique-intersection coverage gate, and hands the images to the Camera Agent. Its structured output is fanned out to 20 parallel Intersection Agents through the Microsoft Agent Framework, then fanned back in to the Orchestrator. The Orchestrator's decision JSON is written back to the shared directory; UE5 polls it and updates signal phases. If any layer fails, the simulation simply holds its previous pattern and tries again on the next cycle.

The JSON payload shared between every layer has to follow strict formats. This matters because GPT-4o output is non-deterministic; early iterations would return the wrong schema, alter key names or silently omit intersections it judged unimportant. Treating prompt engineering and schema validation as one component is what makes an LLM-based control loop robust enough to leave running.

Decision cycle flow diagram

One of the key challenges was ensuring consistency. GPT-4o outputs are not always predictable, so strict schema validation was essential. Early versions would return incorrect formats or omit data, which made the system unreliable. Treating prompt design and validation as one combined problem was critical to making the system stable enough to run continuously.

Results

Metric	Baseline	AI-Enhanced
Mean throughput (300s run)	204.4	127.1
95% CI	[195.6, 213.4]	[121.3, 132.9]
Throughput dispersion (IQR)	≈ 240	≈ 120

Across forty simulation runs, each lasting 300 seconds, the traditional fixed cycle system outperformed the AI driven system in terms of throughput. The AI system processed fewer vehicles, with performance affected by several factors including decision latency, small errors in vehicle detection, and the use of a general purpose language model for a task that ideally requires millisecond level control.

However, the numeric result was not the main success of the project. The primary goal was to explore and demonstrate what can be built using Azure AI Foundry and multi agent systems, and in that respect the project achieved its aim.

Forty 300-second runs across two demand profiles, ten replicates per condition. The fixed-cycle baseline beat the AI-enhanced mode on throughput by 37.8%. Three factors drove the gap: 3–10 second decision-cycle latency, vision noise of around one vehicle per approach and using a general-purpose LLM for what is effectively a millisecond-scale control task.

The final result is not very important to the success of this project. The primary goal of this project was to demonstrate Azure AI Foundry's capabilities, which was readily achieved.

Main Deliverable

One of the most valuable outcomes of the project is the dataset produced. For every decision cycle and each intersection, the system logs a full state action outcome record. This includes screenshots, true queue lengths from the simulator, estimated queue lengths from the AI, throughput, signal states, and decision outputs. In total, the project generated around 48,000 labelled data points and associated images, producing roughly 4.8 GB of structured data.

This dataset has real value beyond the simulation. It can be used to train improved vision models that are more robust to noise, or to train reinforcement learning agents that learn traffic control policies from experience rather than predefined rules.

Every cycle, for every intersection, the simulation logs a state–action–outcome triple: a 512×512 screenshot, ground-truth queue lengths, GPT-4o-derived queue lengths, throughput, signal state, flow direction, and the orchestrator's recommended action. Forty runs across different modes and traffic densities produced roughly 48,000 triples and 48,000 paired screenshots, about 4.8 GB of structured, labelled data. Each screenshot has both a clean label (simulator ground truth) and a noisy label (vision estimate), making it directly usable for fine-tuning a vehicle-counting model robust to real-world camera noise. The same triples are in the format offline reinforcement learning algorithms expect.

Future Work

The next step would be to replace the intersection agents with a reinforcement learning model trained on this dataset, using traffic flow improvements and queue lengths as rewards. Another improvement would be to fine tune a dedicated vision model to replace the current GPT-4o based approach, using the existing data as training input. Expanding the simulation to include more realistic road types, such as roundabouts and T junctions, and using real traffic data to drive demand patterns would also make the system more representative of real cities.

The most direct continuation is replacing the Intersection Agents with a small reinforcement learning policy trained on the dataset above, using throughput delta and the queue length as the reward signals. Beyond that, the priority extensions are:

A fine-tuned vision model to replace the GPT-4o vision call - the screenshot–queue-length pairs already in the dataset are the training set.
Larger, more realistic networks with mixed intersection types (roundabouts, T-junctions) and time-varying demand curves calibrated against real traffic feeds.

Final Thoughts

This project is a great example of how modern AI tools can be combined to build complex, experimental systems. If you are a student developer interested in AI, simulation, or multi agent systems, this is a space full of opportunities to explore. Microsoft Foundry,

If you're exploring the technologies behind this project, the Microsoft's AI stack, Microsoft Agent Framework, Agent Topologies and the AI Agents for Beginners are the resources I'd recommend.

I would also like to thank my external supervisor, Lee Stott, for his guidance and support throughout the project.

Link to Project

Link to my LinkedIn

Want to Learn More

Join us on Thursday 21 May at 12pm UK time for a live session in the Microsoft Foundry Discord Community where we will explore how a student built a full 20 intersection AI powered traffic simulation using Azure AI Foundry, GPT-4o and the Microsoft Agent Framework.

This is a great opportunity for student developers who want to go beyond theory and see how modern AI tools can be combined to build real, end to end systems.

In this session, we will break down how multi agent systems can be used to solve complex real world problems, how Unreal Engine was used to create a realistic simulation environment, and how Microsoft Foundry enabled rapid experimentation with vision and reasoning models. You will also get a behind the scenes look at the challenges of working with non deterministic AI outputs and how to make these systems more reliable through structured design.

If you are interested in AI, simulation, game development, or building intelligent systems that operate at scale, this session will give you practical ideas you can apply to your own projects. Whether you are just starting out or already experimenting with AI agents, you will leave with a clearer understanding of how to design, test, and iterate on complex systems.

Bring your questions, get involved in the discussion, and connect with others in the community who are building with Microsoft Foundry.

https://discord.gg/rMh9PkjF?event=1502936656506781827

Build AI RAG Apps with LangChain, Azure DocumentDB and Microsoft Foundry: Step-by-Step Guide

JohnAziz — Mon, 27 Apr 2026 08:41:21 GMT

Scenario

Imagine you are building your company’s RAG chat application using Microsoft Foundry - Azure OpenAI and orchestrating the flow with LangChain. The chat experience works, but now it needs to be grounded in your company’s data. You generate embeddings and want to store and query them without adding another database or complex sync pipeline. Instead of stitching services together, you use Azure DocumentDB (with MongoDB compatibility) with built-in vector search to store your JSON data and embeddings in one place. You deploy the app to Azure App Service and quickly compare vector search alone versus a full RAG pipeline, sharing it with your team for testing.

What will you learn?

In this blog, you'll learn to:

Create an Azure DocumentDB (with MongoDB compatibility) resource.
Create an embeddings and a chat deployment in Microsoft Foundry Azure OpenAI portal.
Create an Azure App Service website with continuous deployment from GitHub.
Configure Azure App Service application settings to enable communication between Azure resources.
Configure GitHub workflow to work successfully.

What is the main objective?

Build AI Powered RAG Application using LangChain, Microsoft Foundry Azure OpenAI, and Azure DocumentDB (with MongoDB compatibility): Step-by-Step Guide

Prerequisites

An Azure subscription.
- If you don’t already have one, you can sign up for an Azure free account.
- For students, you can use the free Azure for Students offer which doesn’t require a credit card only your school email.
A GitHub account.

Summary of the steps:

Step 1: Create an Azure DocumentDB (with MongoDB compatibility) resource
Step 2: Create a Microsoft Foundry - Azure OpenAI resource and Deploy chat and embedding Models
Step 3: Create an Azure App Service and Deploy the RAG Chat Application

Step 1: Create an Azure DocumentDB (with MongoDB compatibility) resource

In this step, you'll:

Open the Azure Portal.
Create an Azure DocumentDB (with MongoDB compatibility) resource.

Open the Azure Portal

1. Visit the Azure Portal https://portal.azure.com in your browser and sign in.

Now you are inside the Azure portal!

Create a new Azure DocumentDB (with MongoDB compatibility) resource

In this step, you create an Azure DocumentDB (with MongoDB compatibility) resource to store your data, vector embedding, and perform vector search.

1. Type documentdb in the search bar at the top of the portal page and select Azure DocumentDB (with MongoDB compatibility) from the available options.

2. Select Create from the toolbar to start provisioning your new cluster.

3. Add the following information to create a resource:

What	Value
Subscription	Use your preferred subscription. It's advised to use the same subscription across all the resources that communicate with each other on Azure.
Resource group	Select Create new to create a new resource group. Enter a unique name for the resource group.
Cluster name	Enter a globally unique name.
Location	Select a region close to you for the best response time. For example, Select UK South.
MongoDB version	Select the latest available version of MongoDB

4. Select Configure to configure your cluster tier.

5. Add the following information to configure the cluster tier. You can scale it up later:

What	Value
Cluster tier	Select M25 tier, 2 (Burstable) vCores.
Storage	Select 32 GiB.

6. Select Save.

7. Enter the cluster Admin Username and Password and store them in a secure location.

8. Select Next to configure the networking settings.

9. Select Allow Public Access from Azure services and resources within the Azure to this cluster.

10. Select Add current IP address to the firewall rules to allow local access to the cluster.

11. Select Review + create.

12. Confirm your configuration settings and select Create to start provisioning the resource.

Note: The cluster creation can take up to 10 minutes. It's recommended to move on with the rest of the steps and get back to it later.

Step 2: Create a Microsoft Foundry - Azure OpenAI resource and Deploy chat and embedding Models

In this step, you'll:

Create a Microsoft Foundry Azure OpenAI resource.
Create chat and embedding model deployments.

Create an Azure OpenAI resource

In this step, you create an Azure OpenAI Service resource that enables you to interact with different large language models (LLMs).

1. Type openai in the search bar at the top of the portal page and select Azure OpenAI from the available options.

2. Select Create from the toolbar then select Azure OpenAI to provision a new Azure OpenAI resource.

3. Add the following information to create a resource:

What	Value
Subscription	Use the same subscription you used to apply for Azure OpenAI access.
Resource group	Use the resource group you created in the previous step.
Region	Select a region close to you for the best response time. For example, Select UK South.
Name	Enter a globally unique name.
Pricing tier	Select S0. Currently, this is the only available pricing tier.

4. Now that the basic information is added, select Next to confirm your details and proceed to the next page.

5. Select Next to confirm your network details.

6. Select Next to confirm your tag details.

7. Confirm your configuration settings and select Create to start provisioning the resource. Wait for the deployment to finish.

8. After the deployment finishes, select Go to resource to inspect your created resource. Here, you can manage your resource and find important information like the endpoint URL and API keys.

Create chat and embedding model deployments

1. Select Go to Foundry portal from the toolbar to open the studio.

2. Select Deployments from the Shared resources left side menu to go to the deployments tab.

3. Select + Deploy model from the toolbar then select Deploy base model from the options. A Deploy model window opens.

4. Type gpt-4o-mini to search for the model then select it then select Use model.

5. Select Continue with existing setup to proceed to next step.

6. Refresh page and repeat previous steps to select the model then select Confirm.

7. Review selected options then select Deploy.

8. Select + Deploy model from the toolbar then select Deploy base model from the options. A Deploy model window opens.

9. Type text-embedding-3-small to search for the model then select it then select Confirm.

10. Review selected options then select Deploy.

Step 3: Create an Azure App Service and Deploy the RAG Chat Application

In this step, you'll:

Fork the sample repository on GitHub.
Create an Azure App Service resource with a deployment from GitHub.
Modify Azure App Service Application settings in the Azure portal.
Configure the workflow to deploy your application from GitHub.
Test the website before and after adding the data.

Fork the Sample Repository on GitHub

In this step, you create a copy from the source code on your GitHub account to be able to edit it and use it later.

1. Visit the sample github.com/Azure-Samples/Cosmic-Food-RAG-app in your browser and sign in.

2. Select Fork from the top of the sample page.

3. Select an owner for the fork then, select Create fork.

Create an Azure App Service resource with a deployment from GitHub

In this step, you create an Azure App service resource and connect it with your GitHub account to deploy a Python application.

1. Type app service in the search bar at the top of the portal page and select App Services from the available options.

2. Select Create Web App from the toolbar to start provisioning a new web application.

3. Add the following information to fill in the basic configuration of the application:

What	Value
Subscription	Use the same subscription you used to apply for Azure OpenAI access.
Resource group	Use the same resource group you created before.
Name	Enter a unique name for your website. For example, cosmic-food-rag.
Publish?	Select Code. This option specifies whether your deployment consists of code or a container.
Runtime stack	Select Python 3.12.
Operating System	Select Linux.
Region	Select UK South. This is the region where the rest of the resources you created reside.

4. Add the following information to create the app service plan. You can scale it up later:

What	Value
Linux Plan	Select a pre-existing plan or create a new plan.
Pricing Plan	Select Basic B1.

5. Select Deployment from the toolbar to move to the deployment configuration tab.

6. Add the following information to enable continuous deployment from GitHub:

What	Value
Continuous deployment	Select Enable.
GitHub account	Select your GitHub account.
Organization	Select your organization. If you are using your personal account then select it.
Repository	Select Cosmic-Food-RAG-app.
Branch	Select main.

7. Select Review + create.

8. Confirm your configuration settings and select Create to start provisioning the resource. Wait for the deployment to finish.

9. After the deployment finishes, select Go to resource to inspect your created resource. Here, you can manage your resource and find important information like the application settings and logs.

Modify Azure App service Application settings in the Azure portal

In this step, you configure the Application settings to make the website able to communicate with other cloud resources.

1. In the Web App resource, select Environment variables from the left side menu.

2. Select + Add to add new environment variables to the function configuration.

3. Add the following names and values one by one and select Ok. Make sure to add your own values.

These application settings are for the Azure OpenAI resources that you created:

What	Value
OPENAI_API_VERSION	2024-10-21
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME	gpt-4o-mini
AZURE_OPENAI_CHAT_MODEL_NAME	gpt-4o-mini
AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT_NAME	text-embedding-3-small
AZURE_OPENAI_EMBEDDINGS_MODEL_NAME	text-embedding-3-small
AZURE_OPENAI_EMBEDDINGS_DIMENSIONS	1536
AZURE_OPENAI_DEPLOYMENT_NAME	<azureOpenAiResourceName>
AZURE_OPENAI_ENDPOINT	https://<azureOpenAiResourceName>.openai.azure.com/
AZURE_OPENAI_API_KEY	<azureOpenAiResourceKey>

You can get the Azure OpenAI key from the Azure OpenAI resource page.

Select Keys and Endpoint from the Resource Management section and copy any of the available keys.

These application settings are for Azure DocumentDB (with MongoDB compatibility):

AZURE_COSMOS_USERNAME	<documentUsername>
AZURE_COSMOS_PASSWORD	<documentPassword>
AZURE_COSMOS_CONNECTION_STRING	mongodb+srv://<user>:<password>@<clusterName>.global.mongocluster.cosmos.azure.com/?tls=true&authMechanism=SCRAM-SHA-256&retrywrites=false&maxIdleTimeMS=120000

You can get the DocumentDB connection string from the Azure DocumentDB (with MongoDB compatibility) resource page.

Select Connection strings and copy the connection string. Make sure to replace the user and password with the ones you created.

These application settings are new and are used for resources that will be created when the application starts you can use any value for them:

AZURE_COSMOS_DATABASE_NAME	<documentDatabaseName> ex. CosmicDB
AZURE_COSMOS_COLLECTION_NAME	<documentContainerName> ex. CosmicFoodCollection
AZURE_COSMOS_INDEX_NAME	<documentIndexName> ex. CosmicIndex

4. Select Apply to save your newly added environment variables.

5. Select Configuration then Stack settings to edit the application startup command.

6. Type entrypoint.sh in the startup command field then select Apply.

Configure the Workflow to deploy your application from GitHub

In this step, you modify the GitHub deployment workflow to point to the folder that contains the application.

1. Visit your forked repository on GitHub and notice the failing workflow.

2. Open the workflow file .github/workflows/main_cosmic-food-rag.yml.

3. Open the file and select the pen icon to edit it.

4. Modify line 41 from . to src/.

5. Remove the optional Local Build Section since the application already has tests that cover this part.

6. Add this section to Install Node 22 and build the static frontend.

7. Select Commit changes, and review your commit message and description. Select Commit changes.

The final workflow file should look like this:

# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy # More GitHub Actions for Azure: https://github.com/Azure/actions # More info on Python, GitHub Actions, and Azure App Service: https://aka.ms/python-webapps-actions name: Build and deploy Python app to Azure Web App - cosmic-food-rag on: push: branches: - main workflow_dispatch: jobs: build: runs-on: ubuntu-latest permissions: contents: read #This is required for actions/checkout steps: - uses: actions/checkout@v4 - name: Set up Node 22 uses: actions/setup-node@v6 with: node-version: 22 - name: Install Node Packages & Build Static Site run: cd frontend && npm install && npm run build # By default, when you enable GitHub CI/CD integration through the Azure portal, the platform automatically sets the SCM_DO_BUILD_DURING_DEPLOYMENT application setting to true. This triggers the use of Oryx, a build engine that handles application compilation and dependency installation (e.g., pip install) directly on the platform during deployment. Hence, we exclude the antenv virtual environment directory from the deployment artifact to reduce the payload size. - name: Upload artifact for deployment jobs uses: actions/upload-artifact@v4 with: name: python-app path: | src/ !antenv/ # 🚫 Opting Out of Oryx Build # If you prefer to disable the Oryx build process during deployment, follow these steps: # 1. Remove the SCM_DO_BUILD_DURING_DEPLOYMENT app setting from your Azure App Service Environment variables. # 2. Refer to sample workflows for alternative deployment strategies: https://github.com/Azure/actions-workflow-samples/tree/master/AppService deploy: runs-on: ubuntu-latest needs: build permissions: id-token: write #This is required for requesting the JWT contents: read #This is required for actions/checkout steps: - name: Download artifact from build job uses: actions/download-artifact@v4 with: name: python-app - name: Login to Azure uses: azure/login@v2 with: client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID_5672547ED09F46D59DD431ACF5A29F28 }} tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID_0059913572C8467882D3999D0E0DD5B8 }} subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID_7C42E3352C5D47F084CB0CD14F549D27 }} - name: 'Deploy to Azure Web App' uses: azure/webapps-deploy@v3 id: deploy-to-webapp with: app-name: 'cosmic-food-rag' slot-name: 'Production'

8. Select Actions to review the workflow run status.

Test the website before and After adding the data

In this step, you test the application before adding the data, add the data, and test again.

1. Select the workflow name to open it and get the website URL.

2. Select any of the suggested messages or type your own and it should respond with No results found.

3. Navigate to your Azure App Service resource page and select SSH then select Go to open a new SSH page.

4. In the SSH terminal, run these commands:

uv sync --active

uv run --active ./scripts/add_data.py --file="./data/food_items.json"

5. Navigate back to the live website and type in the chat message Do you have any vegan food dishes? and it should respond with the correct answer now.

Congratulations!! You successfully built the full application.

Clean Up

Once you finish experimenting on Microsoft Azure you might want to delete the resources to not consume any more money from your subscription.

You can delete the resource group and it will delete everything inside it or delete the resources one by one that's totally up to you.

Conclusion

Congratulations! You've learned how to create an Azure DocumentDB (with MongoDB compatibility) cluster, how to create a Microsoft Foundry - Azure OpenAI resource, how to deploy an embedding model and a chat model from the Foundry portal, how to create an Azure App Service and configure continuous deployment with GitHub, and how to modify application settings to enable the communication across Azure resources. By using these technologies, you can build a RAG chat application with the option to perform vector search too over your own data and provide grounded (relevant) responses.

Next steps

Documentation

Training Content

Develop generative AI apps in Azure

Found this useful? Share it with others and follow me to get updates on:

Twitter (twitter.com/john00isaac)
LinkedIn (linkedin.com/in/john0isaac)

Feel free to share your comments and/or inquiries in the comment section below..
See you in future demos!

From Demo to Production: Building Microsoft Foundry Hosted Agents with .NET

Lee_Stott — Wed, 22 Apr 2026 17:30:00 GMT

The Gap Between a Demo and a Production Agent

Let's be honest. Getting an AI agent to work in a demo takes an afternoon. Getting it to work reliably in production, tested, containerised, deployed, observable, and maintainable by a team. is a different problem entirely.

Most tutorials stop at the point where the agent prints a response in a terminal. They don't show you how to structure your code, cover your tools with tests, wire up CI, or deploy to a managed runtime with a proper lifecycle. That gap between prototype and production is where developer teams lose weeks.

Microsoft Foundry Hosted Agents close that gap with a managed container runtime for your own custom agent code. And the Hosted Agents Workshop for .NET gives you a complete, copy-paste-friendly path through the entire journey. from local run to deployed agent to chat UI, in six structured labs using .NET 10.

This post walks you through what the workshop delivers, what you will build, and why the patterns it teaches matter far beyond the workshop itself.

What Is a Microsoft Foundry Hosted Agent?

Microsoft Foundry supports two distinct agent types, and understanding the difference is the first decision you will make as an agent developer.

Prompt agents are lightweight agents backed by a model deployment and a system prompt. No custom code required. Ideal for simple Q&A, summarisation, or chat scenarios where the model's built-in reasoning is sufficient.
Hosted agents are container-based agents that run your own code .NET, Python, or any framework you choose inside Foundry's managed runtime. You control the logic, the tools, the data access, and the orchestration.

When your scenario requires custom tool integrations, deterministic business logic, multi-step workflow orchestration, or private API access, a hosted agent is the right choice. The Foundry runtime handles the managed infrastructure; you own the code.

For the official deployment reference, see Deploy a hosted agent to Foundry Agent Service on Microsoft Learn.

What the Workshop Delivers

The Hosted Agents Workshop for .NET is a beginner-friendly, hands-on workshop that takes you through the full development and deployment path for a real hosted agent. It is structured around a concrete scenario: a Hosted Agent Readiness Coach that helps delivery teams answer questions like:

Should this use case start as a prompt agent or a hosted agent?
What should a pilot launch checklist include?
How should a team troubleshoot common early setup problems?

The scenario is purposefully practical. It is not a toy chatbot. It is the kind of tool a real team would build and hand to other engineers, which means it needs to be testable, deployable, and extensible.

The workshop covers:

Local development and validation with .NET 10
Copilot-assisted coding with repo-specific instructions
Deterministic tool implementation with xUnit test coverage
CI pipeline validation with GitHub Actions
Secure deployment to Azure Container Registry and Microsoft Foundry
Chat UI integration using Blazor

What You Will Build

By the end of the workshop, you will have a code-based hosted agent that exposes an OpenAI Responses-compatible /responses endpoint on port 8088.

The agent is backed by three deterministic local tools, implemented in WorkshopLab.Core:

RecommendImplementationShape — analyses a scenario and recommends hosted or prompt agent based on its requirements
BuildLaunchChecklist — generates a pilot launch checklist for a given use case
TroubleshootHostedAgent — returns structured troubleshooting guidance for common setup problems

These tools are deterministic by design, no LLM call required to return a result. That choice makes them fast, predictable, and fully testable, which is the right architecture for business logic in a production agent.

The end-to-end architecture looks like this:

The Hands-On Journey: Lab by Lab

The workshop follows a deliberate build → validate → ship progression. Each lab has a clear outcome. You do not move forward until the previous checkpoint passes.

Lab 0 — Setup and Local Run

Open the repo in VS Code or a GitHub Codespace, configure your Microsoft Foundry project endpoint and model deployment name, then run the agent locally. By the end of Lab 0, your agent is listening on http://localhost:8088/responses and responding to test requests.

dotnet restore
dotnet build
dotnet run --project src/WorkshopLab.AgentHost

Test it with a single PowerShell call:

Invoke-RestMethod -Method Post `
    -Uri "http://localhost:8088/responses" `
    -ContentType "application/json" `
    -Body '{"input":"Should we start with a hosted agent or a prompt agent?"}'

Lab 0 instructions →

Lab 1 — Copilot Customisation

Configure repo-specific GitHub Copilot instructions so that Copilot understands the hosted-agent patterns used in this project. You will also add a Copilot review skill tailored to hosted agent code reviews. This step means every code suggestion you receive from Copilot is contextualised to the workshop scenario rather than giving generic .NET advice.

Lab 1 instructions →

Lab 2 — Tool Implementation

Extend one of the deterministic tools in WorkshopLab.Core with a real feature change. The suggested change adds a stronger recommendation path to RecommendImplementationShape for scenarios that require all three hosted-agent strengths simultaneously.

// In RecommendImplementationShape — add before the final return:
if (requiresCode && requiresTools && requiresWorkflow)
{
    return string.Join(Environment.NewLine,
    [
        $"Recommended implementation: Hosted agent (full-stack)",
        $"Scenario goal: {goal}",
        "Why: the scenario requires custom code, external tool access, and " +
        "multi-step orchestration — all three hosted-agent strengths.",
        "Suggested next step: start with a code-based hosted agent, register " +
        "local tools for each integration, and add a workflow layer."
    ]);
}

You then write an xUnit test to cover it, run dotnet test, and validate the change against a live /responses call. This is the workshop's most important teaching moment: every tool change is covered by a test before it ships.

Lab 2 instructions →

Lab 3 — CI Validation

Wire up a GitHub Actions workflow that builds the solution, runs the test suite, and validates that the agent container builds cleanly. No manual steps — if a change breaks the build or a test, CI catches it before any deployment happens.

Lab 3 instructions →

Lab 4 — Deployment to Microsoft Foundry

Use the Azure Developer CLI (azd) to provision an Azure Container Registry, publish the agent image, and deploy the hosted agent to Microsoft Foundry. The workshop separates provisioning from deployment deliberately: azd owns the Azure resources; the Foundry control plane deployment is an explicit, intentional final step that depends on your real project endpoint and agent.yaml manifest values.

Lab 4 instructions →

Lab 5 — Chat UI Integration

Connect a Blazor chat UI to the deployed hosted agent and validate end-to-end responses. By the end of Lab 5, you have a fully functioning agent accessible through a real UI, calling your deterministic tools via the Foundry control plane.

Lab 5 instructions →

Key Concepts to Take Away

The workshop teaches concrete patterns that apply well beyond this specific scenario.

Code-first agent design

Prompt-only agents are fast to build but hard to test and reason about at scale. A hosted agent with code-backed tools gives you something you can unit test, refactor, and version-control like any other software.

Deterministic tools and testability

The workshop explicitly avoids LLM calls inside tool implementations. Deterministic tools return predictable outputs for a given input, which means you can write fast, reliable unit tests for them. This is the right pattern for business logic. Reserve LLM calls for the reasoning layer, not the execution layer.

CI/CD for agent systems

AI agents are software. They deserve the same build-test-deploy discipline as any other service. Lab 3 makes this concrete: you cannot ship without passing CI, and CI validates the container as well as the unit tests.

Deployment separation

The workshop's split between azd provisioning and Foundry control-plane deployment is not arbitrary. It reflects the real operational boundary: your Azure resources are long-lived infrastructure; your agent deployment is a lifecycle event tied to your project's specific endpoint and manifest. Keeping them separate reduces accidents and makes rollbacks easier.

Observability and the validation mindset

Every lab ends with an explicit checkpoint. The culture the workshop builds is: prove it works before moving on. That mindset is more valuable than any specific tool or command in the labs.

Why Hosted Agents Are Worth the Investment

The managed runtime in Microsoft Foundry removes the infrastructure overhead that makes custom agent deployment painful. You do not manage Kubernetes clusters, configure ingress rules, or handle TLS termination. Foundry handles the hosting; you handle the code.

This matters most for teams making the transition from demo to production. A prompt agent is an afternoon's work. A hosted agent with proper CI, tested tools, and a deployment pipeline is a week's work done properly once, instead of several weeks of firefighting done poorly repeatedly.

The Foundry agent lifecycle —> create, update, version, deploy —>also gives you the controls you need to manage agents in a real environment: staged rollouts, rollback capability, and clear separation between agent versions. For the full deployment guide, see Deploy a hosted agent to Foundry Agent Service.

From Workshop to Real Project

This workshop is not just a learning exercise. The repository structure, the tooling choices, and the CI/CD patterns are a reference implementation.

The patterns you can lift directly into a production project include:

The WorkshopLab.Core / WorkshopLab.AgentHost separation between business logic and agent hosting
The agent.yaml manifest pattern for declarative Foundry deployment
The GitHub Actions workflow structure for build, test, and container validation
The azd + ACR pattern for image publishing without requiring Docker Desktop locally
The Blazor chat UI as a starting point for internal tooling or developer-facing applications

The scenario, a readiness coach for hosted agents. This is also something teams evaluating Microsoft Foundry will find genuinely useful. It answers exactly the questions that come up when onboarding a new team to the platform.

Common Mistakes When Building Hosted Agents

Having run workshops and spoken with developer teams building on Foundry, a few patterns come up repeatedly:

Skipping local validation before containerising. Always validate the /responses endpoint locally first. Debugging inside a container is slower and harder than debugging locally.
Putting business logic inside the LLM call. If the answer to a user query can be determined by code, use code. Reserve the model for reasoning, synthesis, and natural language output.
Treating CI as optional. Agent code changes break things just like any other code change. If you do not have CI catching regressions, you will ship them.
Conflating provisioning and deployment. Recreating Azure resources on every deploy is slow and error-prone. Provision once with azd; deploy agent versions as needed through the Foundry control plane.
Not having a rollback plan. The Foundry agent lifecycle supports versioning. Use it. Know how to roll back to a previous version before you deploy to production.

Get Started

The workshop is open source, beginner-friendly, and designed to be completed in a single day. You need a .NET 10 SDK, an Azure subscription, access to a Microsoft Foundry project, and a GitHub account.

Clone the repository, follow the labs in order, and by the end you will have a production-ready reference implementation that your team can extend and adapt for real scenarios.

Clone the workshop repository →

Here is the quick start to prove the solution works locally before you begin the full lab sequence:

git clone https://github.com/microsoft/Hosted_Agents_Workshop_dotNET.git
cd Hosted_Agents_Workshop_dotNET

# Set your Foundry project endpoint and model deployment
$env:AZURE_AI_PROJECT_ENDPOINT = "https://<resource>.services.ai.azure.com/api/projects/<project>"
$env:MODEL_DEPLOYMENT_NAME     = "gpt-4.1-mini"

# Build and run
dotnet restore
dotnet build
dotnet run --project src/WorkshopLab.AgentHost

Then send your first request:

Invoke-RestMethod -Method Post `
    -Uri "http://localhost:8088/responses" `
    -ContentType "application/json" `
    -Body '{"input":"Should we start with a hosted agent or a prompt agent?"}'

When the agent answers as a Hosted Agent Readiness Coach, you are ready to begin the labs.

Key Takeaways

Hosted agents in Microsoft Foundry let you run custom .NET code in a managed container runtime — you own the logic, Foundry owns the infrastructure.
Deterministic tools are the right pattern for business logic in production agents: fast, testable, and predictable.
CI/CD is not optional for agent systems. Build it in from the start, not as an afterthought.
Separate your provisioning (azd) from your deployment (Foundry control plane) — it reduces accidents and simplifies rollbacks.
The workshop is a reference implementation, not just a tutorial. The patterns are production-grade and ready to adapt.

References

Building an Auditable Security Layer for Agentic AI

hazem — Wed, 22 Apr 2026 08:06:05 GMT

Most agent failures do not look like breaches.

They look like a normal chat, a normal answer, and a normal tool call. Until the next morning, when a single question collapses the whole story: who authorized that action.

You think you deployed an agent. In reality, you deployed an unbounded automation pipeline that happens to speak English.

I’m Hazem Ali — Microsoft AI MVP, Distinguished AI & ML Architect, Founder & CEO at Skytells. For over 20 years, I’ve built secure, scalable enterprise AI across cloud and edge, with a focus on agent security and sovereign, governed AI architectures. My work on these systems is widely referenced by practitioners across multiple regions.

Hazem Ali honored to receive an official speaker invitation under the patronage of H.H. Sheikh Dr. Sultan bin Muhammad Al Qasimi, Member of the UAE Supreme Council and Ruler of Sharjah, to speak at the Sharjah International Conference on Linguistic Intelligence (SICLI), organized by the American University of Sharjah (AUS) and the Emirates Scholar Center for Research and Studies.

This piece is a collaboration with Hammad Atta a Practice Lead – AI Security & Cloud Strategy and Dr. Yasir Mehmood , Dr Muhammad Zeeshan Baig, Dr. Muhammad Aatif, Dr. MUHAMMAD AZIZ UL HAQ. We align on one core idea: agent security is not about making the model behave. It is about building enforceable boundaries around the model and proving every privileged step.

This article is meant to sit next to my earlier Tech Community piece, Zero-Trust Agent Architecture: How To Actually Secure Your Agents, and go one level deeper into the mechanics you can implement on Azure today.

Let me break it down.

The Principle: The model is not your boundary

Let me break it down in the way I’d explain it in a design review.

A boundary is something that still holds when the component on the other side is adversarial, confused, or simply wrong. An LLM is none of those reliably. In an agent, the model is not just a generator. It becomes a planner and scheduler.

It decides when to retrieve, which tool to call, how to shape arguments, and when to loop.

That means your real attack surface is not “bad output.” It is the control-flow graph the model is allowed to traverse.

So if your “security” lives inside the prompt, you are putting policy in the same token stream the attacker can influence. That is not a boundary. That is a suggestion.

The only stable design is to treat the model like an untrusted proposer and the runtime like the verifier.

Here is the chain I use. Each gate is external to the model and survives manipulation.

Context Gate: Everything that enters the model is treated as executable influence, not “text.”
Capability Gate: Tools are invoked as constrained capabilities, not free-form function calls.
Evidence Gate: Every privileged step produces a verifiable artifact, not a story.
Retrieval Control Plane: What the agent can see is governed by labels and identity, not prompt etiquette.
Detection Layer: Drift and probing become alerts, not surprises.

Figure: Model proposes. Runtime verifies. Input + retrieved context → shields → model → tool gateway → signed intent → governed retrieval → SOC telemetry.

Now the rare part, the part most people miss: the boundary is not “block or allow.” The boundary is stateful. Once the runtime sees a suspicious signal, the entire session must transition into a degraded capability state, and every downstream gate must enforce that state.

1. Treat context as executable influence, and preserve provenance

If you do RAG, your documents are not “supporting info.” They are an input channel. That makes the biggest prompt-injection risk not the user. It is your documents.

Microsoft’s Prompt Shields covers user prompt attacks (scanned at the user input intervention point) and document attacks (scanned at the user input and tool response intervention points). When enabled, each request returns annotation results with detected and filtered values that your runtime can translate into a policy decision: block, degrade, or allow.

Provenance Collapse.

Most teams concatenate prompt + policy + retrieved chunks into one blob. The moment you do that, you lose the one thing you need for a defensible boundary: you can no longer reliably tell which tokens came from where. That is how “context” becomes “authority.”

For indirect/document attacks,

Microsoft guidance recommends delimiting context documents inside the prompt using """<documents> ... </documents>""" to improve indirect attack detection.

That delimiter is not formatting. It is a provenance marker that improves indirect attack detection through Prompt Shields.

Minimal, practical pattern:

// Provenance-preserving prompt construction for indirect/document attack detection function buildPrompt(system: string, user: string, retrievedDocs: string[]): string { const docs = retrievedDocs.map((d) => `- ${d}`).join("\n"); return [ system, "", `User: ${user}`, "", `""" <documents>\n${docs}\n</documents> """`, ].join("\n"); }

Then treat Prompt Shields output as a session security event, not a banner:

type RiskState = "NORMAL" | "SUSPECT" | "BLOCK"; type FilterPolicy = "BLOCK_ON_FILTERED" | "DEGRADE_ON_FILTERED"; function computeRiskState( shields: { detected: boolean; filtered?: boolean }, labels: string[], policy: FilterPolicy = "DEGRADE_ON_FILTERED", ): RiskState { // detected => hard stop if (shields.detected) return "BLOCK"; // filtered is an annotation signal: block or degrade by policy if (shields.filtered) { return policy === "BLOCK_ON_FILTERED" ? "BLOCK" : "SUSPECT"; } // example: sensitivity-based degradation independent of shield hits const sensitive = labels.some((l) => ["Confidential", "HighlyConfidential", "Regulated"].includes(l), ); return sensitive ? "SUSPECT" : "NORMAL"; }

When the signal is clear, you block and log. When it is suspicious, you do not warn. You downgrade authority.

QSAF Alignment:

Prompt Injection Protection (Domain 1):

QSAF-PI-001 (static pattern blacklist), QSAF-PI-002 (dynamic LLM analysis), QSAF-PI-003 (semantic embedding comparison)

All addressed by Prompt Shields and provenance marking.

Context Manipulation (Domain 2): QSAF-RC-004 (context drift), QSAF-RC-007 (nested prompt injection) – mitigated by stateful risk calculation.

2. Tools are capabilities with constraints, not functions

When the model proposes a tool call, your runtime should re-derive what is allowed from identity plus risk state, then enforce it at the gateway.

type ToolRequest = { tool: string; args: unknown; }; type Capabilities = { allowWrite: boolean; allowedTools: Set<string>; }; function deriveCapabilities(risk: RiskState, roles: string[]): Capabilities { const baseAllowed = new Set(["search_kb", "get_profile", "summarize"]); const isAdmin = roles.includes("Admin"); if (risk === "SUSPECT") { return { allowWrite: false, allowedTools: baseAllowed }; } if (risk === "BLOCK") { return { allowWrite: false, allowedTools: new Set() }; } // NORMAL const tools = new Set([ ...baseAllowed, ...(isAdmin ? ["update_record", "issue_refund"] : []), ]); return { allowWrite: isAdmin, allowedTools: tools }; } function authorizeTool(req: ToolRequest, caps: Capabilities): void { if (!caps.allowedTools.has(req.tool)) throw new Error("ToolNotAllowed"); if (!caps.allowWrite && req.tool.startsWith("update_")) { throw new Error("WriteDenied"); } }

The model can ask. It cannot grant itself permission.

QSAF Alignment:

Plugin Abuse Monitoring (Domain 3):

QSAF-PL-001 (whitelist enforcement), QSAF-PL-003 (restrict sensitive plugins), QSAF-PL-006 (rate‑limiting) – implemented via capability derivation and gateway policies.

Behavioral Anomaly Detection (Domain 5):

QSAF-BA-006 (plugin execution pattern deviance) – detected by comparing actual calls against derived capabilities.

The Integrity Gate: Hash-chain the authority, not the output

Let me add the part that makes investigations clean.

Most teams treat integrity like an audit log problem. That is not enough. Logs explain. Integrity proves.

The hard truth is that agent authority is assembled out of pieces: the system instruction, the user prompt, retrieved chunks, risk annotations, and finally the tool intent. If you do not bind those pieces together cryptographically, an incident review becomes a story-telling session.

This is why QSAF has an entire domain for payload integrity and signing, including prompt hash signing, nonce or replay protection, and a hash chain lineage that tracks how a session evolved.

Here is how you can map that into the runtime verifies.

You build a canonical “authority envelope” for every privileged hop, compute a digest, and then:

link it to the previous hop (hash chain)
include a nonce (replay control)
sign the digest with Azure Key Vault (Key Vault signs digests, it does not hash your content for you)

import crypto from "crypto"; type AuthorityEnvelope = { sessionId: string; turnId: number; policyVersion: string; // provenance-preserved components systemHash: string; userHash: string; documentsHash: string; // hash of structured retrieved chunks (not just rendered text) shields: { detected: boolean; filtered: boolean; }; riskState: "NORMAL" | "SUSPECT" | "BLOCK"; // proposed action (if any) tool?: { name: string; argsHash: string; }; // anti-replay + lineage nonce: string; prevDigest?: string; ts: string; }; function sha256(bytes: string): string { return crypto.createHash("sha256").update(bytes).digest("hex"); } // Canonicalization matters. JSON.stringify is OK if you control key order. // For cross-language, use RFC 8785 (JCS) canonical JSON. function canonicalJson(x: unknown): string { return JSON.stringify(x); } function buildEnvelope( input: Omit<AuthorityEnvelope, "nonce" | "ts">, ): AuthorityEnvelope { return { ...input, nonce: crypto.randomUUID(), ts: new Date().toISOString(), }; } function digestEnvelope(env: AuthorityEnvelope): string { return sha256(canonicalJson(env)); }

Then you call Key Vault to sign that digest (REST sign), and optionally verify later (REST verify).

The rare failure mode this blocks is subtle: authority splicing.

Without a hash chain, it is possible for the runtime to correctly validate a tool call, but later be unable to prove which retrieved chunk, which Prompt Shields result, and which policy version were in force when that call was authorized. With the chain, every privileged hop becomes tamper-evident.

This is the point: Prompt Shields tells you “this looks dangerous.” Document delimiters preserve provenance.
The integrity gate makes the runtime able to say, later, with evidence: “This is exactly what I accepted as authority.”

QSAF Alignment:

Payload Integrity & Signing (Domain 6): QSAF-PY-001 (prompt hash signing), QSAF-PY-005 (nonce/replay control), QSAF-PY-006 (hash chain lineage) – directly implemented via the envelope and chaining.

Tools must sit behind a wall that can say “no”

Tool calls are where language becomes authority. If an agent can call APIs that mutate state, your security story is not about the response text. It is about whether the tool call is allowed under explicit policy.

This is exactly where Azure API Management belongs: as the tool gateway that enforces authentication and authorization before any tool request reaches your backend. The validate-jwt policy is the canonical enforcement mechanism for validating JWTs at the gateway.

The design goal is simple:

The model can request a tool call. The gateway decides if it is permitted.

A capability token approach keeps it clean:

<validate-jwt header-name="Authorization" failed-validation-httpcode="401"> <required-claims> <claim name="scp"> <value>tools.read</value> </claim> </required-claims> </validate-jwt>

The claim name (scp, roles, or custom claims) depends on your token issuer; the point is enforcing authorization at the gateway, not inside model text.

Now you can enforce “read-only mode” by issuing tokens that simply do not carry write scopes. The model can try to call a write tool. It still gets denied by policy.

Evidence is not logs. Evidence is a signed chain.

Logs help you debug. Evidence helps you prove.

So you hash the session envelope and the tool intent, then sign the digest using Azure Key Vault Keys.

Key Vault sign creates a signature from a digest, and verify verifies a signature against a digest. Key Vault does not hash your content for you. Hash locally, then sign the digest.), and Key Vault documentation is explicit that signing is sign-hash, not “sign arbitrary content.” You hash locally, then ask Key Vault to sign the hash.

import crypto from "crypto"; const sha256 = (x: unknown): string => crypto.createHash("sha256").update(JSON.stringify(x)).digest("hex"); type IntentEnvelope = { sessionId: string; userId: string; promptHash: string; documentsHash: string; tool: string; argsHash: string; nonce: string; ts: string; policyVersion: string; }; function buildIntent( sessionId: string, userId: string, prompt: string, docs: unknown, tool: string, args: unknown, policyVersion: string, ): IntentEnvelope { return { sessionId, userId, promptHash: sha256(prompt), documentsHash: sha256(docs), tool, argsHash: sha256(args), nonce: crypto.randomUUID(), ts: new Date().toISOString(), policyVersion, }; }

Once you do this, your system stops “explaining.” It starts proving.

Govern what the agent can see, not only what it can say

RAG without governance eventually becomes a data exposure feature.

This is why I treat retrieval as a governed operation. Microsoft Purview sensitivity labels give you a practical way to classify content and build retrieval rules on top of that classification. Microsoft documents creating and configuring sensitivity labels in Purview.

The pattern is simple:

Label the corpus.
Filter retrieval by label and identity policy.
Log label distribution per completion.
Alert when a low-privilege identity retrieves high-sensitivity labels.

This is how you keep sovereignty real. Not in a slide deck. In the retrieval path.

Operate it like a security system: posture and detection

Inline gates reduce risk. They do not eliminate it. Systems drift. People add tools. Policies get loosened. Attacks evolve.

Microsoft Defender for Cloud’s Defender CSPM plan includes AI security posture management for generative AI apps and AI agents (Preview), including discovery/inventory of AI agents deployed with Azure AI Foundry.

Then you use Microsoft Sentinel to turn your telemetry into incidents, with scheduled analytics rules.

Your detections should match the gates you built:

Repeated Prompt Shields detections from the same identity or session.
Tool-call spikes after a suspicious document signal.
APIM denials for write endpoints from sessions in read-only mode.
High-sensitivity label retrieval by identities that should never touch that tier.

QSAF Alignment:

Behavioral Anomaly Detection (Domain 5):

QSAF-BA-001 (session entropy), QSAF-BA-004 (repeated intent mutation), QSAF-BA-007 (unified risk score) – detected via Sentinel rules.

Cross‑Environment Defense (Domain 9): QSAF-CE-006 (coordinated alert response) – using Sentinel incidents and playbooks.

Where the reference checklist fits, quietly

Behind the scenes, we use a control checklist lens to ensure we cover prompt/context attacks, tool misuse, integrity, governance, and operational monitoring. The point is not to rename Microsoft features into framework terms. The point is to make the system enforceable and auditable using Azure-native gates.

Closing

Zero trust for agents is not a slogan. It is a build.

Prompt Shields gives you a front gate for both user prompt attacks and document attacks, with clear annotations like detected and filtered.
API Management gives you a tool boundary that can say “no” regardless of what the model tries, using validate-jwt.
Signed intent gives you evidence, using Key Vault’s sign-hash semantics.
Purview labels give you governed retrieval. Sentinel and Defender give you an operating model, not wishful thinking.

If you want the conceptual spine and the architectural principles that frame this pipeline, start with my earlier Tech Community pieces, then come back here and implement the gates.

Thanks for reading

— Hazem Ali

Prompt Engineering for Spec-Driven Development with SpecKit

charykn — Mon, 20 Apr 2026 09:14:37 GMT

Introduction

Charlotte Yeo, UCL MEng Computer Science https://www.linkedin.com/in/charlotte-yeo-627476294/

Supervisors: Janaina Mourao-Miranda (UCL) and Lee Stott (Microsoft).

For my final-year MEng project at UCL, I investigated how to get the best results out of SpecKit, a spec-driven AI development framework, by systematically testing different prompt strategies.

Here's what I found.

Project Overview

LLMs are powerful coding assistants, but they struggle to maintain context over long development sessions, leading to hallucinations and inconsistent outputs. SpecKit addresses this by using persistent, structured specification documents as memory throughout the development process. The developer writes a natural language spec; SpecKit builds the software from it.

The problem is that no one has established best practices for writing those specs. This project aimed to fill that gap.

Experiments

I ran 10 experiments, each using SpecKit to build the same target system, a multi-agent AI code verification tool, from a different prompt formulation. The variables I tested included prompt authority, format, level of detail, and output format. By keeping the target software constant, the effect of each prompt change on SpecKit's performance is isolated.

The target system itself used Microsoft Agent Framework, Azure Cosmos DB for RAG, and Microsoft Foundry to access GPT-5.2, all orchestrated via a Python codebase. This covered a wide range of real-world engineering challenges: multi-agent coordination, cloud service integration, and working with a library new enough that the model hadn't been trained on it.

Technical Details

SpecKit runs as a series of commands inside GitHub Copilot in VS Code, powered here by Claude Sonnet 4.5. The workflow moves through seven stages: /constitution → /specify → /clarify → /plan → /tasks → /analyze → /implement. At each stage, SpecKit writes and updates Markdown files that serve as persistent memory, so the session can be paused and resumed without losing context.

Key tools used:

Microsoft Agent Framework — agent orchestration
Microsoft Foundry — access to LLMs (GPT-5.2, Text Embedding 3)
Azure Cosmos DB — code example database for RAG
Claude Sonnet 4.5 — model powering SpecKit via GitHub Copilot

Results

These were the key findings:

Natural language outperforms machine-readable formats. The JSON prompt (Case 1) took 40% longer and generated significantly more issues than the natural language control.
Authority is necessary. Removing the authoritative framing from the prompt (Case 3) caused SpecKit to treat specifications as optional, resulting in the multi-agent system not being built at all until manually corrected. Total time: 4h 53m vs. 2h 24m for the control.
Omit what the model already knows. Removing the scoring rubrics (Case 8) saved 34 minutes with no loss in output quality as the model inferred the rubric from context. However, omitting the Cosmos DB schema or agent architecture descriptions caused major implementation errors.
The model must be able to read its own outputs. Changing the output to PDF (Case 9), which Claude Sonnet 4.5 cannot read in Copilot, caused the implementation stage to increase significantly to 7h 38m, with 33 required interventions, because the model couldn't verify whether its code was working.

Best Practices Found

The biggest insight is that prompt design has as much impact on SpecKit's performance as prompt content. A complete specification written non-authoritatively or in JSON will produce worse results than a slightly shorter specification written in clear, authoritative natural language.

There is also a trade-off between token count and manual intervention. Shorter prompts are faster, but only when the omitted information is something the model can reliably infer. Leaving out details about unique libraries or architectures will result in higher debugging times later.

Future Development

These are directions for future work in this area:

Running each experiment multiple times to account for model non-determinism
Repeating experiments with newer or different LLMs to test generalisability
Testing with different target systems beyond code verification
Supplying SpecKit with tools (e.g. Playwright MCP) to read outputs it currently cannot access, like live webpages or PDFs

Conclusion

Spec-driven development with SpecKit is a useful approach for building complex software with LLMs, but the quality of your prompt determines the quality of your outcome. For the most effective results, write in natural language, keep the whole prompt authoritative, include detail on novel or library-specific components, design your system's outputs to be readable by the model building them, and leave out only what the model can confidently infer.

If you want to explore the tools used in this project, here are some useful starting points:

Build and Deploy a Microsoft Foundry Hosted Agent: A Hands-On Workshop

Lee_Stott — Fri, 03 Apr 2026 11:25:45 GMT

Agents are easy to demo, hard to ship.

Most teams can put together a convincing prototype quickly. The harder part starts afterwards: shaping deterministic tools, validating behaviour with tests, building a CI path, packaging for deployment, and proving the experience through a user-facing interface. That is where many promising projects slow down.

This workshop helps you close that gap without unnecessary friction. You get a guided path from local run to deployment handoff, then complete the journey with a working chat UI that calls your deployed hosted agent through the project endpoint.

What You Will Build

This is a hands-on, end-to-end learning experience for building and deploying AI agents with Microsoft Foundry.

The lab provides a guided and practical journey through hosted-agent development, including deterministic tool design, prompt-guided workflows, CI validation, deployment preparation, and UI integration.

It’s designed to reduce setup friction with a ready-to-run experience.

It is a prompt-based development lab using Copilot guidance and MCP-assisted workflow options during deployment.

It’s a .NET 10 workshop that includes local development, Copilot-assisted coding, CI, secure deployment to Azure, and a working chat UI.

A local hosted agent that responds on the responses contract
Deterministic tool improvements in core logic with xUnit coverage
A GitHub Actions CI workflow for restore, build, test, and container validation
An Azure-ready deployment path using azd, ACR image publishing, and Foundry manifest apply
A Blazor chat UI that calls openai/v1/responses with agent_reference
A repeatable implementation shape that teams can adapt to real projects

Who This Lab Is For

AI developers and software engineers who prefer learning by building
Motivated beginners who want a guided, step-by-step path
Experienced developers who want a practical hosted-agent reference implementation
Architects evaluating deployment shape, validation strategy, and operational readiness
Technical decision-makers who need to see how demos become deployable systems

Why Hosted Agents

Hosted agents run your code in a managed environment. That matters because it reduces the amount of infrastructure plumbing you need to manage directly, while giving you a clearer path to secure, observable, team-friendly deployments.

Prompt-only demos are still useful. They are quick, excellent for ideation, and often the right place to start. Hosted agents complement that approach when you need custom code, tool-backed logic, and a deployment process that can be repeated by a team.

Think of this lab as the bridge: you keep the speed of prompt-based iteration, then layer in the real-world patterns needed to run reliably.

What You Will Learn

1) Orchestration

You will practise workflow-oriented reasoning through implementation-shape recommendations and multi-step readiness scenarios. The lab introduces orchestration concepts at a practical level, rather than as a dedicated orchestration framework deep dive.

2) Tool Integration

You will connect deterministic tools and understand how tool calls fit into predictable execution paths. This is a core focus of the workshop and is backed by tests in the solution.

3) Retrieval Patterns (What This Lab Covers Today)

This workshop does not include a full RAG implementation with embeddings and vector search. Instead, it focuses on deterministic local tools and hosted-agent response flow, giving you a strong foundation before adding retrieval infrastructure in a follow-on phase.

4) Observability

You will see light observability foundations through OpenTelemetry usage in the host and practical verification during local and deployed checks. This is introductory coverage intended to support debugging and confidence building.

5) Responsible AI

You will apply production-minded safety basics, including secure secret handling and review hygiene. A full Responsible AI policy and evaluation framework is not the primary goal of this workshop, but the workflow does encourage safe habits from the start.

6) Secure Deployment Path

You will move from local implementation to Azure deployment with a secure, practical workflow: azd provisioning, ACR publishing, manifest deployment, hosted-agent start, status checks, and endpoint validation.

The Learning Journey

The overall flow is simple and memorable: clone, open, run, iterate, deploy, observe.

clone -> open -> run -> iterate -> deploy -> observe

You are not expected to memorize every command. The lab is structured to help you learn through small, meaningful wins that build confidence.

Your First 15 Minutes: Quick Wins

Open the repo and understand the lab structure in a few minutes
Set project endpoint and model deployment environment variables
Run the host locally and validate the responses endpoint
Inspect the deterministic tools in WorkshopLab.Core
Run tests and see how behaviour changes are verified
Review the deployment path so local work maps to Azure steps
Understand how the UI validates end-to-end behaviour after deployment
Leave the first session with a working baseline and a clear next step

That first checkpoint is important. Once you see a working loop on your own machine, the rest of the workshop becomes much easier to finish.

Using Copilot and MCP in the Workflow

This lab emphasises prompt-based development patterns that help you move faster while still learning the underlying architecture. You are not only writing code, you are learning to describe intent clearly, inspect generated output, and iterate with discipline.

Copilot supports implementation and review in the coding labs. MCP appears as a practical deployment option for hosted-agent lifecycle actions, provided your tools are authenticated to the correct tenant and project context.

Together, this creates a development rhythm that is especially useful for learning:

Define intent with clear prompts
Generate or adjust implementation details
Validate behaviour through tests and UI checks
Deploy and observe outcomes in Azure
Refine based on evidence, not guesswork

That same rhythm transfers well to real projects. Even if your production environment differs, the patterns from this workshop are adaptable.

Production-Minded Tips

As you complete the lab, keep a production mindset from day one:

Reliability: keep deterministic logic small, testable, and explicit
Security: Treat secrets, identity, and access boundaries as first-class concerns
Observability: use telemetry and status checks to speed up debugging
Governance: keep deployment steps explicit so teams can review and repeat them

You do not need to solve everything in one pass. The goal is to build habits that make your agent projects safer and easier to evolve.

Start Today:

If you have been waiting for the right time to move from “interesting demo” to “practical implementation”, this is the moment. The workshop is structured for self-study, and the steps are designed to keep your momentum high.

Start here: https://github.com/microsoft/Hosted_Agents_Workshop_Lab

Want deeper documentation while you go? These official guides are great companions:

When you finish, share what you built. Post a screenshot or short write-up in a GitHub issue/discussion, on social, or in comments with one lesson learned. Your example can help the next developer get unstuck faster.

Copy/Paste Progress Checklist

[ ] Clone the workshop repo
[ ] Complete local setup and run the agent
[ ] Make one prompt-based behaviour change
[ ] Validate with tests and chat UI
[ ] Run CI checks
[ ] Provision and deploy via Azure and Foundry workflow
[ ] Review observability signals and refine
[ ] Share what I built + one takeaway

Common Questions

How long does it take?

Most developers can complete a meaningful pass in a few focused sessions of 60-75 mins. You can get the first local success quickly, then continue through deployment and refinement at your own pace.

Do I need an Azure subscription?

Yes, for provisioning and deployment steps. You can still begin local development and testing before completing all Azure activities.

Is it beginner-friendly?

Yes. The labs are written for beginners, run in sequence, and include expected outcomes for each stage.

Can I adapt it beyond .NET?

Yes. The implementation in this workshop is .NET 10, but the architecture and development patterns can be adapted to other stacks.

What if I am evaluating for a team?

This lab is a strong team evaluation asset because it demonstrates end-to-end flow: local dev, integration patterns, CI, secure deployment, and operational visibility.

Closing

This workshop gives you more than theory. It gives you a practical path from first local run to deployed hosted agent, backed by tests, CI, and a user-facing UI validation loop. If you want a build-first route into Microsoft Foundry hosted-agent development, this is an excellent place to start.

Begin now: https://github.com/microsoft/Hosted_Agents_Workshop_Lab

Getting Started with Foundry Local: A Student Guide to the Microsoft Foundry Local Lab

Lee_Stott — Mon, 30 Mar 2026 07:00:00 GMT

If you want to start building AI applications on your own machine, the Microsoft Foundry Local Lab is one of the most useful places to begin. It is a practical workshop that takes you from first-time setup through to agents, retrieval, evaluation, speech transcription, tool calling, and a browser-based interface. The material is hands-on, cross-language, and designed to show how modern AI apps can run locally rather than depending on a cloud service for every step.

This blog post is aimed at students, self-taught developers, and anyone learning how AI applications are put together in practice. Instead of treating large language models as a black box, the lab shows you how to install and manage local models, connect to them with code, structure tasks into workflows, and test whether the results are actually good enough. If you have been looking for a learning path that feels more like building real software and less like copying isolated snippets, this workshop is a strong starting point.

What Is Foundry Local?

Foundry Local is a local runtime for downloading, managing, and serving AI models on your own hardware. It exposes an OpenAI-compatible interface, which means you can work with familiar SDK patterns while keeping execution on your device. For learners, that matters for three reasons. First, it lowers the barrier to experimentation because you can run projects without setting up a cloud account for every test. Second, it helps you understand the moving parts behind AI applications, including model lifecycle, local inference, and application architecture. Third, it encourages privacy-aware development because the examples are designed to keep data on the machine wherever possible.

The Foundry Local Lab uses that local-first approach to teach the full journey from simple prompts to multi-agent systems. It includes examples in Python, JavaScript, and C#, so you can follow the language that fits your course, your existing skills, or the platform you want to build on.

Why This Lab Works Well for Learners

A lot of AI tutorials stop at the moment a model replies to a prompt. That is useful for a first demo, but it does not teach you how to build a proper application. The Foundry Local Lab goes further. It is organised as a sequence of parts, each one adding a new idea and giving you working code to explore. You do not just ask a model to respond. You learn how to manage the service, choose a language SDK, construct retrieval pipelines, build agents, evaluate outputs, and expose the result through a usable interface.

That sequence is especially helpful for students because the parts build on each other. Early labs focus on confidence and setup. Middle labs focus on architecture and patterns. Later labs move into more advanced ideas that are common in real projects, such as tool calling, evaluation, and custom model packaging. By the end, you have seen not just what a local AI app looks like, but how its different layers fit together.

Before You Start

The workshop expects a reasonably modern machine and at least one programming language environment. The core prerequisites are straightforward: install Foundry Local, clone the repository, and choose whether you want to work in Python, JavaScript, or C#. You do not need to master all three. In fact, most learners will get more value by picking one language first, completing the full path in that language, and only then comparing how the same patterns look elsewhere.

If you are new to AI development, do not be put off by the number of parts. The early sections are accessible, and the later ones become much easier once you have completed the foundations. Think of the lab as a structured course rather than a single tutorial.

What You Learn in Each Lab https://github.com/microsoft-foundry/foundry-local-lab

Part 1: Getting Started with Foundry Local

The first part introduces the basics of Foundry Local and gets you up and running. You learn how to install the CLI, inspect the model catalogue, download a model, and run it locally. This part also introduces practical details such as model aliases and dynamic service ports, which are small but important pieces of real development work.

For students, the value of this part is confidence. You prove that local inference works on your machine, you see how the service behaves, and you learn the operational basics before writing any application code. By the end of Part 1, you should understand what Foundry Local does, how to start it, and how local model serving fits into an application workflow.

Part 2: Foundry Local SDK Deep Dive

Once the CLI makes sense, the workshop moves into the SDK. This part explains why application developers often use the SDK instead of relying only on terminal commands. You learn how to manage the service programmatically, browse available models, control model download and loading, and understand model metadata such as aliases and hardware-aware selection.

This is where learners start to move from using a tool to building with a platform. You begin to see the difference between running a model manually and integrating it into software. By the end of this section, you should understand the API surface you will use in your own projects and know how to bootstrap the SDK in Python, JavaScript, or C#.

Part 3: SDKs and APIs

Part 3 turns the SDK concepts into a working chat application. You connect code to the local inference server and use the OpenAI-compatible API for streaming chat completions. The lab includes examples in all three supported languages, which makes it especially useful if you are comparing ecosystems or learning how the same idea is expressed through different syntax and libraries.

The key learning outcome here is not just that you can get a response from a model. It is that you understand the boundary between your application and the local model service. You learn how messages are structured, how streaming works, and how to write the sort of integration code that becomes the foundation for every later lab.

Part 4: Retrieval-Augmented Generation

This is where the workshop starts to feel like modern AI engineering rather than basic prompting. In the retrieval-augmented generation lab, you build a simple RAG pipeline that grounds answers in supplied data. You work with an in-memory knowledge base, apply retrieval logic, score matches, and compose prompts that include grounded context.

For learners, this part is important because it demonstrates a core truth of AI app development: a model on its own is often not enough. Useful applications usually need access to documents, notes, or structured information. By the end of Part 4, you understand why retrieval matters, how to pass retrieved context into a prompt, and how a pipeline can make answers more relevant and reliable.

Part 5: Building AI Agents

Part 5 introduces the concept of an agent. Instead of a one-off prompt and response, you begin to define behaviour through system instructions, roles, and conversation state. The lab uses the ChatAgent pattern and the Microsoft Agent Framework to show how an agent can maintain a purpose, respond with a persona, and return structured output such as JSON.

This part helps learners understand the difference between a raw model call and a reusable application component. You learn how to design instructions that shape behaviour, how multi-turn interaction differs from single prompts, and why structured output matters when an AI component has to work inside a broader system.

Part 6: Multi-Agent Workflows

Once a single agent makes sense, the workshop expands the idea into a multi-agent workflow. The example pipeline uses roles such as researcher, writer, and editor, with outputs passed from one stage to the next. You explore sequential orchestration, shared configuration, and feedback loops between specialised components.

For students, this lab is a very clear introduction to decomposition. Instead of asking one model to do everything at once, you break a task into smaller responsibilities. That pattern is useful well beyond AI. By the end of Part 6, you should understand why teams build multi-agent systems, how hand-offs are structured, and what trade-offs appear when more components are added to a workflow.

Part 7: Zava Creative Writer Capstone Application

The Zava Creative Writer is the capstone project that brings the earlier ideas together into a more production-style application. It uses multiple specialised agents, structured JSON hand-offs, product catalogue search, streaming output, and evaluation-style feedback loops. Rather than showing an isolated feature, this part shows how separate patterns combine into a complete system.

This is one of the most valuable parts of the workshop for learner developers because it narrows the gap between tutorial code and real application design. You can see how orchestration, agent roles, and practical interfaces fit together. By the end of Part 7, you should be able to recognise the architecture of a serious local AI app and understand how the earlier labs support it.

Part 8: Evaluation-Led Development

Many beginner AI projects stop once the output looks good once or twice. This lab teaches a much stronger habit: evaluation-led development. You work with golden datasets, rule-based checks, and LLM-as-judge scoring to compare prompt or agent variants systematically. The goal is to move from anecdotal testing to repeatable assessment.

This matters enormously for students because evaluation is one of the clearest differences between a classroom demo and dependable software. By the end of Part 8, you should understand how to define success criteria, compare outputs at scale, and use evidence rather than intuition when improving an AI component.

Part 9: Voice Transcription with Whisper

Part 9 broadens the workshop beyond text generation by introducing speech-to-text with Whisper running locally. You use the Foundry Local SDK to download and load the model, then transcribe local audio files through the compatible API surface. The emphasis is on privacy-first processing, with audio kept on-device.

This section is a useful reminder that local AI development is not limited to chatbots. Learners see how a different modality fits into the same ecosystem and how local execution supports sensitive workloads. By the end of this lab, you should understand the transcription flow, the relevant client methods, and how speech features can be integrated into broader applications.

Part 10: Using Custom or Hugging Face Models

After learning the standard path, the workshop shows how to work with custom or Hugging Face models. This includes compiling models into optimised ONNX format with ONNX Runtime GenAI, choosing hardware-specific options, applying quantisation strategies, creating configuration files, and adding compiled models to the Foundry Local cache.

For learner developers, this part opens the door to model engineering rather than simple model consumption. You begin to understand that model choice, optimisation, and packaging affect performance and usability. By the end of Part 10, you should have a clearer picture of how models move from an external source into a runnable local setup and why deployment format matters.

Part 11: Tool Calling with Local Models

Tool calling is one of the most practical patterns in current AI development, and this lab covers it directly. You define tool schemas, allow the model to request function calls, handle the multi-turn interaction loop, execute the tools locally, and return results back to the model. The examples include practical scenarios such as weather and population tools.

This lab teaches learners how to move beyond generation into action. A model is no longer limited to producing text. It can decide when external data or a function is needed and incorporate that result into a useful answer. By the end of Part 11, you should understand the tool-calling flow and how AI systems connect reasoning with deterministic software behaviour.

Part 12: Building a Web UI for the Zava Creative Writer

Part 12 adds a browser-based front end to the capstone application. You learn how to serve a shared interface from Python, JavaScript, or C#, stream updates to the browser, consume NDJSON with the Fetch API and ReadableStream, and show live agent status as content is produced in real time.

This part is especially good for students who want to build portfolio projects. It turns backend orchestration into something visible and interactive. By the end of Part 12, you should understand how to connect a local AI backend to a web interface and how streaming changes the user experience compared with waiting for one final response.

Part 13: Workshop Complete

The final part is a summary and extension point. It reviews what you have built across the previous sections and suggests ways to continue. Although it is not a new technical lab in the same way as the earlier parts, it plays an important role in learning. It helps you consolidate the architecture, the terminology, and the development patterns you have encountered.

For learners, reflection matters. By the end of Part 13, you should be able to describe the full stack of a local AI application, from model management to user interface, and identify which area you want to deepen next.

What Students Gain from the Full Workshop

Taken together, these labs do more than teach Foundry Local itself. They teach how AI applications are built. You learn operational basics such as model setup and service management. You learn application integration through SDKs and APIs. You learn system design through RAG, agents, multi-agent orchestration, and web interfaces. You learn engineering discipline through evaluation. You also see how text, speech, custom models, and tool calling all fit into one local-first development workflow.

That breadth makes the workshop useful in several settings. A student can use it as a self-study path. A lecturer can use it as source material for practical sessions. A learner developer can use it to build portfolio pieces and to understand which AI patterns are worth learning next. Because the repository includes Python, JavaScript, and C#, it also works well for comparing how architectural ideas transfer across languages.

How to Approach the Lab as a Beginner

If you are starting from scratch, the best route is simple. Complete Parts 1 to 3 in your preferred language first. That gives you the essential setup and integration skills. Then move into Parts 4 to 6 to understand how AI application patterns are composed. After that, use Parts 7 and 8 to learn how larger systems and evaluation fit together. Finally, explore Parts 9 to 12 based on your interests, whether that is speech, tooling, model customisation, or front-end work.

It is also worth keeping notes as you go. Record what each part adds to your understanding, what code files matter, and what assumptions each example makes. That habit will help you move from following the labs to adapting the patterns in your own projects.

Final Thoughts

The Microsoft Foundry Local Lab is a strong introduction to local AI development because it treats learners like developers rather than spectators. You install, run, connect, orchestrate, evaluate, and present working systems. That makes it far more valuable than a short demo that only proves a model can answer a question.

If you are a student or learner developer who wants to understand how AI applications are really built, this lab gives you a clear path. Start with the basics, pick one language, and work through the parts in order. By the time you finish, you will not just have used Foundry Local. You will have a practical foundation for building local AI applications with far more confidence and much better judgement.

Langchain Multi-Agent Systems with Microsoft Agent Framework and Hosted Agents

Lee_Stott — Thu, 26 Mar 2026 07:00:00 GMT

If you have been building AI agents with LangChain, you already know how powerful its tool and chain abstractions are. But when it comes to deploying those agents to production — with real infrastructure, managed identity, live web search, and container orchestration — you need something more.

This post walks through how to combine LangChain with the Microsoft Agent Framework (azure-ai-agents) and deploy the result as a Microsoft Foundry Hosted Agent. We will build a multi-agent incident triage copilot that uses LangChain locally and seamlessly upgrades to cloud-hosted capabilities on Microsoft Foundry.

Why combine LangChain with Microsoft Agent Framework?

As a LangChain developer, you get excellent abstractions for building agents: the @tool decorator, RunnableLambda chains, and composable pipelines. But production deployment raises questions that LangChain alone does not answer:

Where do your agents run? Containers, serverless, or managed infrastructure?
How do you add live web search or code execution? Bing Grounding and Code Interpreter are not LangChain built-ins.
How do you handle authentication? Managed identity, API keys, or tokens?
How do you observe agents in production? Distributed tracing across multiple agents?

The Microsoft Agent Framework fills these gaps. It provides AgentsClient for creating and managing agents on Microsoft Foundry, built-in tools like BingGroundingTool and CodeInterpreterTool, and a thread-based conversation model. Combined with Hosted Agents, you get a fully managed container runtime with health probes, auto-scaling, and the OpenAI Responses API protocol.

The key insight: LangChain handles local logic and chain composition; the Microsoft Agent Framework handles cloud-hosted orchestration and tooling.

Architecture overview

The incident triage copilot uses a coordinator pattern with three specialist agents:

User Query
    |
    v
Coordinator Agent
    |
    +--> LangChain Triage Chain    (routing decision)
    +--> LangChain Synthesis Chain  (combine results)
    |
    +---+---+---+
    |   |       |
    v   v       v
Research  Diagnostics  Remediation
 Agent      Agent        Agent

Each specialist agent has two execution modes:

Mode	LangChain Role	Microsoft Agent Framework Role
Local	`@tool` functions provide heuristic analysis	Not used
Foundry	Chains handle routing and synthesis	`AgentsClient` with `BingGroundingTool`, `CodeInterpreterTool`

This dual-mode design means you can develop and test locally with zero cloud dependencies, then deploy to Foundry for production capabilities.

Step 1: Define your LangChain tools

Start with what you know. Define typed, documented tools using LangChain’s @tool decorator:

from langchain_core.tools import tool @tool def classify_incident_severity(query: str) -> str: """Classify the severity and priority of an incident based on keywords. Args: query: The incident description text. Returns: Severity classification with priority level. """ query_lower = query.lower() critical_keywords = [ "production down", "all users", "outage", "breach", ] high_keywords = [ "503", "500", "timeout", "latency", "slow", ] if any(kw in query_lower for kw in critical_keywords): return "severity=critical, priority=P1" if any(kw in query_lower for kw in high_keywords): return "severity=high, priority=P2" return "severity=low, priority=P4"

These tools work identically in local mode and serve as fallbacks when Foundry is unavailable.

Step 2: Build routing with LangChain chains

Use RunnableLambda to create a routing chain that classifies the incident and selects which specialists to invoke:

from langchain_core.runnables import RunnableLambda from enum import Enum class AgentRole(str, Enum): RESEARCH = "research" DIAGNOSTICS = "diagnostics" REMEDIATION = "remediation" DIAGNOSTICS_KEYWORDS = { "log", "error", "exception", "timeout", "500", "503", "crash", "oom", "root cause", } REMEDIATION_KEYWORDS = { "fix", "remediate", "runbook", "rollback", "hotfix", "patch", "resolve", "action plan", } def _route(inputs: dict) -> dict: query = inputs["query"].lower() specialists = [AgentRole.RESEARCH] # always included if any(kw in query for kw in DIAGNOSTICS_KEYWORDS): specialists.append(AgentRole.DIAGNOSTICS) if any(kw in query for kw in REMEDIATION_KEYWORDS): specialists.append(AgentRole.REMEDIATION) return {**inputs, "specialists": specialists} triage_routing_chain = RunnableLambda(_route)

This is pure LangChain — no cloud dependency. The chain analyses the query and returns which specialists should handle it.

Step 3: Create specialist agents with dual-mode execution

Each specialist agent extends a base class. In local mode, it uses LangChain tools. In Foundry mode, it delegates to the Microsoft Agent Framework:

from abc import ABC, abstractmethod from pathlib import Path class BaseSpecialistAgent(ABC): role: AgentRole prompt_file: str def __init__(self): prompt_path = Path(__file__).parent.parent / "prompts" / self.prompt_file self.system_prompt = prompt_path.read_text(encoding="utf-8") async def run(self, query, shared_context, correlation_id, client=None): if client is not None: return await self._run_on_foundry(query, shared_context, correlation_id, client) return await self._run_locally(query, shared_context, correlation_id) async def _run_on_foundry(self, query, shared_context, correlation_id, client): """Use Microsoft Agent Framework for cloud-hosted execution.""" from azure.ai.agents.models import BingGroundingTool agent = await client.agents.create_agent( model=shared_context.get("model_deployment", "gpt-4o"), name=f"{self.role.value}-{correlation_id}", instructions=self.system_prompt, tools=self._get_foundry_tools(shared_context), ) thread = await client.agents.threads.create() await client.agents.messages.create( thread_id=thread.id, role="user", content=self._build_prompt(query, shared_context), ) run = await client.agents.runs.create_and_process( thread_id=thread.id, agent_id=agent.id, ) # Extract and return the agent’s response... async def _run_locally(self, query, shared_context, correlation_id): """Use LangChain tools for local heuristic analysis.""" # Each subclass implements this with its specific tools ...

The key pattern here: same interface, different backends. Your coordinator does not care whether a specialist ran locally or on Foundry.

Step 4: Wire it up with FastAPI

Expose the multi-agent pipeline through a FastAPI endpoint. The /triage endpoint accepts incident descriptions and returns structured reports:

from fastapi import FastAPI from agents.coordinator import Coordinator from models import TriageRequest app = FastAPI(title="Incident Triage Copilot") coordinator = Coordinator() @app.post("/triage") async def triage(request: TriageRequest): return await coordinator.triage( request=request, client=app.state.foundry_client, max_turns=10, )

The application also implements the /responses endpoint, which follows the OpenAI Responses API protocol. This is what Microsoft Foundry Hosted Agents expects when routing traffic to your container.

Step 5: Deploy as a Hosted Agent

This is where Microsoft Foundry Hosted Agents shines. Your multi-agent system becomes a managed, auto-scaling service with a single command:

# Install the azd AI agent extension azd extension install azure.ai.agents # Provision infrastructure and deploy azd up

The Azure Developer CLI (azd) provisions everything:

Azure Container Registry for your Docker image
Container App with health probes and auto-scaling
User-Assigned Managed Identity for secure authentication
Microsoft Foundry Hub and Project with model deployments
Application Insights for distributed tracing

Your agent.yaml defines what tools the hosted agent has access to:

name: incident-triage-copilot-langchain kind: hosted model: deployment: gpt-4o identity: type: managed tools: - type: bing_grounding enabled: true - type: code_interpreter enabled: true

What you gain over pure LangChain

Capability	LangChain Only	LangChain + Microsoft Agent Framework
Local development	Yes	Yes (identical experience)
Live web search	Requires custom integration	Built-in `BingGroundingTool`
Code execution	Requires sandboxing	Built-in `CodeInterpreterTool`
Managed hosting	DIY containers	Foundry Hosted Agents
Authentication	DIY	Managed Identity (zero secrets)
Observability	DIY	OpenTelemetry + Application Insights
One-command deploy	No	`azd up`

Testing locally

The dual-mode architecture means you can test the full pipeline without any cloud resources:

# Create virtual environment and install dependencies python -m venv .venv source .venv/bin/activate pip install -r requirements.txt # Run locally (agents use LangChain tools) python -m src

Then open http://localhost:8080 in your browser to use the built-in web UI, or call the API directly:

curl -X POST http://localhost:8080/triage \ -H "Content-Type: application/json" \ -d '{"message": "Getting 503 errors on /api/orders since 2pm"}'

The response includes a coordinator summary, specialist results with confidence scores, and the tools each agent used.

Running the tests

The project includes a comprehensive test suite covering routing logic, tool behaviour, agent execution, and HTTP endpoints:

curl -X POST http://localhost:8080/triage \ -H "Content-Type: application/json" \ -d '{"message": "Getting 503 errors on /api/orders since 2pm"}'

Tests run entirely in local mode, so no cloud credentials are needed.

Key takeaways for LangChain developers

Keep your LangChain abstractions. The @tool decorator, RunnableLambda chains, and composable pipelines all work exactly as you expect.
Add cloud capabilities incrementally. Start local, then enable Bing Grounding, Code Interpreter, and managed hosting when you are ready.
Use the dual-mode pattern. Every agent should work locally with LangChain tools and on Foundry with the Microsoft Agent Framework. This makes development fast and deployment seamless.
Let azd handle infrastructure. One command provisions everything: containers, identity, monitoring, and model deployments.
Security comes free. Managed Identity means no API keys in your code. Non-root containers, RBAC, and disabled ACR admin are all configured by default.

Get started

Clone the sample repository and try it yourself:

git clone https://github.com/leestott/hosted-agents-langchain-samples cd hosted-agents-langchain-samples python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt python -m src

Open http://localhost:8080 to interact with the copilot through the web UI. When you are ready for production, run azd up and your multi-agent system is live on Microsoft Foundry.

Resources

Build an Offline Hybrid RAG Stack with ONNX and Foundry Local

Lee_Stott — Thu, 26 Mar 2026 07:00:00 GMT

If you are building local AI applications, basic retrieval augmented generation is often only the starting point. This sample shows a more practical pattern: combine lexical retrieval, ONNX based semantic embeddings, and a Foundry Local chat model so the assistant stays grounded, remains offline, and degrades cleanly when the semantic path is unavailable.

Why this sample is worth studying

Many local RAG samples rely on a single retrieval strategy. That is usually enough for a proof of concept, but it breaks down quickly in production. Exact keywords, acronyms, and document codes behave differently from natural language questions and paraphrased requests.

This repository keeps the original lexical retrieval path, adds local ONNX embeddings for semantic search, and fuses both signals in a hybrid ranking mode. The generation step runs through Foundry Local, so the entire assistant can remain on device.

Lexical mode handles exact terms and structured vocabulary.
Semantic mode handles paraphrases and more natural language phrasing.
Hybrid mode combines both and is usually the best default.
Lexical fallback protects the user experience if the embedding pipeline cannot start.

Architectural overview

The sample has two main flows: an offline ingestion pipeline and a local query pipeline.

The architecture splits cleanly into offline ingestion at the top and runtime query handling at the bottom.

Offline ingestion pipeline

Read Markdown files from docs/.
Parse front matter and split each document into overlapping chunks.
Generate dense embeddings when the ONNX model is available.
Store chunks in SQLite with both sparse lexical features and optional dense vectors.

Local query pipeline

The browser posts a question to the Express API.
ChatEngine resolves the requested retrieval mode.
VectorStore retrieves lexical, semantic, or hybrid results.
The prompt is assembled with the retrieved context and sent to a Foundry Local chat model.
The answer is returned with source references and retrieval metadata.

The sequence diagram shows the difference between lexical retrieval and hybrid retrieval. In hybrid mode, the query is embedded first, then lexical and semantic scores are fused before prompt assembly.

Repository structure and core components

The implementation is compact and readable. The main files to understand are listed below.

src/config.js: retrieval defaults, paths, and model settings.
src/embeddingEngine.js: local ONNX embedding generation through Transformers.js.
src/vectorStore.js: SQLite storage plus lexical, semantic, and hybrid ranking.
src/chatEngine.js: retrieval mode resolution, prompt assembly, and Foundry Local model execution.
src/ingest.js: document ingestion and embedding generation during indexing.
src/server.js: REST endpoints, streaming endpoints, upload support, and health reporting.

Getting started

To run the sample, you need Node.js 20 or newer, Foundry Local, and a local ONNX embedding model. The default model path is models/embeddings/bge-small-en-v1.5.

cd c:\Users\leestott\local-hybrid-retrival-onnx npm install huggingface-cli download BAAI/bge-small-en-v1.5 --local-dir models/embeddings/bge-small-en-v1.5 npm run ingest npm start

Ingestion writes the local SQLite database to data/rag.db. If the embedding model is available, each chunk gets a dense vector as well as lexical features. If the embedding model is missing, ingestion still succeeds and the application remains usable in lexical mode.

Best practice: local AI applications should treat model files, SQLite data, and native runtime compatibility as part of the deployable system, not as optional developer conveniences.

Code walkthrough

1. Retrieval configuration

The sample makes its retrieval behaviour explicit in configuration. That is useful for testing and for operator visibility.

export const config = { model: "phi-3.5-mini", docsDir: path.join(ROOT, "docs"), dbPath: path.join(ROOT, "data", "rag.db"), chunkSize: 200, chunkOverlap: 25, topK: 3, retrievalMode: process.env.RETRIEVAL_MODE || "hybrid", retrievalModes: ["lexical", "semantic", "hybrid"], fallbackRetrievalMode: "lexical", retrievalWeights: { lexical: 0.45, semantic: 0.55, }, };

Those defaults tell you a lot about the intended operating profile. Chunks are small, the number of returned chunks is low, and the fallback path is explicit.

2. Local ONNX embeddings

The embedding engine disables remote model loading and only uses local files. That matters for privacy, repeatability, and air gapped operation.

env.allowLocalModels = true; env.allowRemoteModels = false; this.extractor = await pipeline("feature-extraction", resolvedPath, { local_files_only: true, }); const output = await this.extractor(text, { pooling: "mean", normalize: true, });

The mean pooling and normalisation step make the vectors suitable for cosine similarity based ranking.

3. Hybrid storage and ranking in SQLite

Instead of adding a separate vector database, the sample stores lexical and semantic representations in the same SQLite table. That keeps the local footprint low and the implementation easy to debug.

searchHybrid(query, queryEmbedding, topK = 5, weights = { lexical: 0.45, semantic: 0.55 }) { const lexicalResults = this.searchLexical(query, topK * 3); const semanticResults = this.searchSemantic(queryEmbedding, topK * 3); if (semanticResults.length === 0) { return lexicalResults.slice(0, topK).map((row) => ({ ...row, retrievalMode: "lexical", })); } const fused = [...combined.values()].map((row) => ({ ...row, score: (row.lexicalScore * lexicalWeight) + (row.semanticScore * semanticWeight), })); fused.sort((a, b) => b.score - a.score); return fused.slice(0, topK); }

The important point is not just the weighted fusion. It is the fallback behaviour. If semantic retrieval cannot provide results, the user still gets lexical grounding instead of an empty context window.

4. Retrieval mode resolution in ChatEngine

ChatEngine keeps the runtime behaviour predictable. It validates the requested mode and falls back to lexical search when semantic retrieval is unavailable.

resolveRetrievalMode(requestedMode) { const desiredMode = config.retrievalModes.includes(requestedMode) ? requestedMode : config.retrievalMode; if ((desiredMode === "semantic" || desiredMode === "hybrid") && !this.semanticAvailable) { return config.fallbackRetrievalMode; } return desiredMode; }

This is a sensible production design because local runtime failures are common. Missing model files or native dependency mismatches should reduce quality, not crash the entire assistant.

5. Foundry Local model management

The sample uses FoundryLocalManager to discover, download, cache, and load the configured chat model.

This gives the app a better local startup experience. The server can expose a status stream while the model initialises in the background.

User experience and screenshots

The client is intentionally simple, which makes it useful during evaluation. You can switch retrieval mode, test questions quickly, and inspect the retrieved sources.

The landing page exposes retrieval mode directly in the UI. That makes it easy to compare lexical, semantic, and hybrid behaviour during testing.

The sources panel shows grounding evidence and retrieval scores, which is useful when validating whether better answers are coming from better retrieval or just model phrasing.

Best practices for ONNX RAG and Foundry Local

Keep lexical fallback alive. Exact identifiers and runtime failures both make this necessary.
Persist sparse and dense features together where possible. It simplifies debugging and operational reasoning.
Use small chunks and conservative topK values for local context budgets.
Expose health and status endpoints so users can see when the model is still loading or embeddings are unavailable.
Test retrieval quality separately from generation quality.
Pin and validate native runtime dependencies, especially ONNX Runtime, before tuning prompts.

Practical warning: this repository already shows why runtime validation matters. A local app can ingest documents successfully and still fail at model initialisation if the native runtime stack is misaligned.

How this compares with RAG and CAG

The strongest value in this sample comes from where it sits between a basic local RAG baseline and a curated CAG design.

Dimension	Classic local RAG	This hybrid ONNX RAG sample	CAG
Context assembly	Retrieve chunks at query time, often lexically, then inject them into the prompt.	Retrieve chunks at query time with lexical, semantic, or fused scoring, then inject the strongest results into the prompt.	Use a prepared or cached context pack instead of fresh retrieval for every request.
Main strength	Easy to implement and easy to explain.	Better recall for paraphrases without giving up exact match behaviour or offline execution.	Predictable prompts and low query time overhead.
Main weakness	Misses synonyms and natural language reformulations.	More moving parts, larger local asset footprint, and native runtime compatibility to manage.	Coverage depends on curation quality and goes stale more easily.
Failure behaviour	Weak retrieval leads to weak grounding.	Semantic failure can degrade to lexical retrieval if designed properly, which this sample does.	Prepared context can be too narrow for new or unexpected questions.
Best fit	Simple local assistants and proof of concept systems.	Offline copilots and technical assistants that need stronger recall across varied phrasing.	Stable workflows with tightly bounded, curated knowledge.

Samples

Related samples:

- Foundry Local RAG - https://github.com/leestott/local-rag

- Foundry Local CAG - https://github.com/leestott/local-cag

- Foundry Local hybrid-retrival-onnx https://github.com/leestott/local-hybrid-retrival-onnx

Specific benefits of this hybrid approach over classic RAG

It captures paraphrased questions that lexical search would often miss.
It still preserves exact match performance for codes, terms, and product names.
It gives operators a controlled degradation path when the semantic stack is unavailable.
It stays local and inspectable without introducing a separate hosted vector service.

Specific differences from CAG

CAG shifts effort into context curation before the request. This sample retrieves evidence dynamically at runtime.
CAG can be faster for fixed workflows, but it is usually less flexible when the document set changes.
This hybrid RAG design is better suited to open ended knowledge search and growing document collections.

What to validate before shipping

Measure retrieval quality in each mode using exact term, acronym, and paraphrase queries.
Check that sources shown in the UI reflect genuinely distinct evidence, not repeated chunks.
Confirm the application remains usable when semantic retrieval is unavailable.
Verify ONNX Runtime compatibility on the real target machines, not only on the development laptop.
Test model download, cache, and startup behaviour with a clean environment.

Final take

For developers getting started with ONNX RAG and Foundry Local, this sample is a good technical reference because it demonstrates a realistic local architecture rather than a minimal demo. It shows how to build a grounded assistant that remains offline, supports multiple retrieval modes, and fails gracefully.

Compared with classic local RAG, the hybrid design provides better recall and better resilience. Compared with CAG, it remains more flexible for changing document sets and less dependent on pre curated context packs. If you want a practical starting point for offline grounded AI on developer workstations or edge devices, this is the most balanced pattern in the repository set.

Step-by-Step: Deploy the Architecture Review Agent Using AZD AI CLI

ShivamGoyal — Tue, 24 Mar 2026 07:00:00 GMT

Hey everyone! I am Shivam Goyal, a Microsoft MVP, and I am super excited to share a project that will save you a massive amount of time.

Have you ever built a brilliant AI agent in an afternoon, only to spend the next two weeks fighting with Docker containers, memory persistence, and cloud deployment scripts to get it running?

In our previous post, we introduced the Architecture Review Agent, an open-source tool built on Microsoft Foundry that automatically converts messy architectural notes into structured risk assessments and interactive Excalidraw diagrams.

But building an AI agent is only half the battle. Iterating on one and actually getting it running in a production-grade environment without losing your mind over infrastructure is a completely different story.

The Problem with Agentic Development Loops

The typical agent development loop is painful: you write your agent code, test it by copy-pasting inputs into a local REPL, manually build a container, push it to a registry, configure RBAC, deploy to your cloud target, realize you need to tweak three lines of logic, and start the whole cycle over again.

You often end up with an agent that is 100 lines of clean Python, surrounded by 400 lines of Bicep and a 12-step deployment guide.

The azd ai extension for the Azure Developer CLI (AZD) completely changes this equation. For the Architecture Review Agent, the entire workflow, from zero infrastructure to a live hosted agent you can invoke from the command line, is just a few simple commands. And moving from local testing to a live cloud deployment is a single azd up.

Here is how you can set up, invoke, and deploy your own Architecture Review Agent, and even publish it to Microsoft Teams, without needing a tenant admin.

Step 1: The Setup (No heavy lifting required)

First, make sure you have the Azure Developer CLI installed and grab the AI Agents extension.

# Install AZD winget install microsoft.azd # Install the AI Agents extension azd extension install azure.ai.agents

Next, clone the repository and set up your local Python environment:

git clone https://github.com/Azure-Samples/agent-architecture-review-sample cd agent-architecture-review-sample python -m venv .venv .\.venv\Scripts\Activate.ps1 pip install -r requirements.txt

Finally, authenticate and tell AZD where your Microsoft Foundry project lives:

azd auth login azd env new arch-review-dev # Point it to your Foundry Project and Model azd env set AZURE_AI_PROJECT_ENDPOINT "https://<your-resource>.services.ai.azure.com/api/projects/<your-project>" azd env set AZURE_AI_MODEL_DEPLOYMENT_NAME "gpt-4.1"

Step 2: Run and Invoke Locally

With the AZD AI extension, you get a local server that behaves identically to a deployed Foundry-hosted agent. It uses the same localhost:8088 endpoint, the same OpenAI Responses API protocol, and the same conversation persistence.

Open your first terminal and start the runtime:

azd ai agent run

Now, open a second terminal. This is where the magic happens. The agent is completely format-agnostic. There is no schema you have to memorize. You can pass it a file, or just type out a whiteboard brain-dump inline.

Here is what the terminal experience looks like when running these commands and getting the structured report back:

Styled terminal showing all 3 azd ai agent invoke commands + full structured report output

Here are the three ways you can invoke it using "azd ai agent invoke --local":

Pattern A: The Structured YAML

If your team uses formal definitions, just point the agent to the file. The rule-based parser handles this instantly without an LLM call.

azd ai agent invoke --local "scenarios/ecommerce.yaml"

12-component ecommerce architecture diagram generated from YAML input.

Pattern B: The Whiteboard Brain-Dump (Inline Arrow Notation)

Arrow notation (A -> B -> C) is how engineers actually communicate on whiteboards and in Slack. Before now, this wasn't a valid input for architecture tools.

azd ai agent invoke --local "LB -> 3 API servers -> PostgreSQL primary with read replica -> Redis cache"

The parser automatically extracts the replica count, infers the component types (LB becomes a Gateway), and builds a valid connection graph, surfacing single points of failure instantly.

Pattern C: The Markdown Design Doc

Just point it to your existing READMEs or design docs.

azd ai agent invoke --local "scenarios/event_driven.md"

8-component event-driven streaming architecture generated from Markdown input

For all three patterns, the agent returns a structured Markdown report in your terminal and generates an interactive architecture.excalidraw file and a high-res PNG right in your local /output folder.

Step 3: One Command to the Cloud

When you are happy with how your agent performs locally, it's time to deploy. Forget manual Docker builds and complex credential management.

azd up

This single command orchestrates everything:

Provisions Infrastructure: Creates your Foundry AI Services account, ACR, App Insights, and managed identities with proper RBAC.
Builds and Pushes: Packages your Dockerfile and pushes the container image to ACR.
Deploys the Agent: Registers the image and creates a hosted agent version in Foundry Agent Service.

The output will hand you a live Agent Playground URL and a production-ready API endpoint. Your agent now automatically scales from 0 to 5 replicas, manages its own conversation state, and authenticates securely via Managed Identity.

Step 4: Publish to Teams and M365 Copilot (Zero Admin Required!)

Having an API is great, but agents are most powerful when they live where your users collaborate. You can publish this agent directly to Microsoft Teams and M365 Copilot natively from the Foundry portal.

The best part? You can use the Individual Scope.

Go to the Microsoft Foundry portal and find your deployed agent.
Click Publish to Teams and Microsoft 365 Copilot.
Fill out the basic metadata (Name, Description).
Select the Individual scope.

Because you are using an individual scope, no M365 admin approval is required. The portal automatically provisions the Azure Bot Service, packages the metadata, and registers the app. Within minutes, your agent will appear in your Teams Copilot agent store. You can generate a share link and instantly send it to your team for a workshop or demo.

What I Learned Building This Workflow

Shifting from custom deployment scripts to the azd ai CLI taught me three things:

The declarative contract is beautifully clean. Our azure.yaml declares the agent and infrastructure in about 30 lines. azd up translates that into a fully secure, production-grade Foundry environment.
The local-to-cloud gap is finally gone. The azd ai agent run behaves exactly like the cloud. The invocation you write locally works identically against the deployed endpoint.
Teams publishing is remarkably simple. I expected bot registration nightmares and tenant admin blockers. Instead, I filled out a form, waited two minutes, and was chatting with my architecture agent in Teams.

Resources & Next Steps

Now that we have a streamlined, single-hosted agent deployment, the natural next step is multi-agent orchestration. Imagine a triage agent that routes your design doc to a dedicated Security Reviewer Agent and a Scalability Reviewer Agent.

Try it out yourself by cloning the repository, running azd up, and let me know what you build!

GitHub Repository: Azure-Samples/agent-architecture-review-sample
Previous Article: Stop Drawing Architecture Diagrams Manually: Meet the Open-Source AI Architecture Review Agents | Microsoft Community Hub
Microsoft Learn: Install the Azure Developer CLI
Microsoft Foundry Documentation: Hosted agents in Foundry Agent Service (preview) - Microsoft Foundry