Educator Developer Blog articles

Adding a Fallback Model to Hermes with Microsoft Foundry

suzarilshah — Thu, 27 Aug 2026 07:00:00 GMT

So the plan was simple. Leave the Bedrock configuration untouched, then wire Microsoft Foundry in behind it as a fallback, so Hermes always has somewhere else to go when the primary provider is not responding. A few other reasons pushed me towards Foundry in particular:

Redundancy that does not need me. If Bedrock is throttled or out of quota, I want Hermes to fail over on its own rather than waiting for me to notice.
A catalogue I already pay for. Foundry puts the latest GPT models next to open-weight and partner models in one place, so I can pick a model that suits the task instead of settling for whatever a single provider happens to offer.
Enterprise controls out of the box. Region pinning, private networking, content filters and per-deployment quota all sit in the same portal, which makes the setup far easier to defend to a security reviewer.
Learning the mechanics before I need them. Working out how Hermes handles a provider chain is much nicer on a quiet Tuesday than during a live outage.

Here is the short version, if you are deciding whether to read on. Time: about thirty minutes if nothing goes wrong. Cost: pay-as-you-go tokens only, and none at all while the fallback sits idle. Result: an assistant that keeps answering when your primary provider stops.

Before you start, you will need three things: a machine with Hermes already installed and a working primary provider configured, an Azure subscription with access to Microsoft Foundry in a region you can actually deploy into, and enough quota in that region to create a deployment.

One thing that made this easy to justify: Foundry deployments bill per token on the standard pay-as-you-go tier. A fallback provider that never gets invoked costs nothing beyond the requests it actually serves, so the insurance is close to free until the day you need it.

Chat surface

(CLI, messaging)

→

Hermes Gateway

→

Primary

Amazon Bedrock

→

Fallback

Microsoft Foundry

Figure 1: Where the fallback sits. Every request goes through the Hermes gateway to the primary provider; only when that provider is unavailable does the chain continue to Microsoft Foundry.

Part 1: Deploying a Model on Microsoft Foundry

The first half of this job happens entirely inside the Microsoft Foundry portal and has nothing to do with Hermes yet. All you are really doing here is making sure your Azure subscription can serve a model, and that you hold an endpoint and key Hermes can authenticate with later.

1. Deploy model in Foundry	→	2. Copy endpoint + key	→	3. hermes fallback add	→	4. Authenticate	→	5. Select models, test

Figure 2: The whole setup in five moves. The first two happen in the Microsoft Foundry portal (orange); the rest happen on the Hermes machine (blue).

Go to Microsoft Foundry > Build > Models > Deploy > Deploy a base model. You can deploy a fine-tuned model instead if you already have one, which works just as well with Hermes. Check the region shown at the top of the portal before you commit, because both model availability and deployment quota differ from one region to the next. Then deploy the model you have selected:

In this case I deployed gpt-5.6-sol, which is the model Hermes will fall back to.

The choice was deliberate rather than exciting. My primary model on Bedrock is a general-purpose chat model, and a fallback is only useful if the answers it gives feel like a continuation of the same conversation rather than a different assistant wearing the same name. The gpt-5.6-sol deployment matches that behaviour closely, it was available in the region I wanted to pin, and the quota I was granted comfortably covers a day of normal use. If a fallback surprises you the first time it fires, it is the wrong fallback.

Once the deployment finishes, open it and take note of two values: the target endpoint URI and the API key. Copy both somewhere safe now, because you will be pasting them into Hermes in the next part. If your organisation rotates keys on a schedule, use a key with the longest life you are allowed, since a fallback secured with a credential that expires quietly stops being a fallback.

What to copy	Where it lives in the portal	Where Hermes asks for it
Target endpoint URI	Deployment > Endpoint > Target URI	"Endpoint" prompt in hermes fallback add
API key	Deployment > Endpoint > Key	"API key" prompt, or choose Entra ID instead
Deployment name	Deployment > Details > Name	Shown in the model list Hermes returns
Region	Top of the portal, next to the resource	Must match the region you deployed into

Figure 3: Everything Hermes will ask for, and where to find each value before you leave the portal.

Part 2: Adding Foundry to Hermes as a Fallback

With the Foundry side sorted, everything from here happens in the Hermes CLI. One thing worth knowing before you start: this is the fallback command, not the primary model command, so your existing Bedrock configuration is left completely alone. Nothing in this section can break what is already working, which makes it a good one to try on a live setup.

Run the Hermes fallback command:

When Hermes asks which provider to add, choose Azure Foundry. The picker still carries the old name; it is the same service that the portal now calls Microsoft Foundry.

Paste the target endpoint URI from your deployment, then authenticate with the API key you copied earlier. Hermes also offers Microsoft Entra ID at this prompt, which is the better option if your organisation would rather not have a static key sitting on the machine. If authentication fails here, check the endpoint before you start suspecting the key. In my experience the endpoint is wrong far more often than the credential is, usually because the deployment name at the end of the URI does not match the deployment you actually created.

Once authentication succeeds, Hermes lists the deployments your Foundry resource exposes and asks which ones you want to use. You can select more than one, and the order is not cosmetic: Hermes walks down the chain from top to bottom whenever the provider above is unavailable. Treat that list as a priority order, not a shopping basket.

What happens to the primary	What Hermes does	What you see in the chat
Responds normally	Routes every request to the primary and never touches the chain	Nothing. The fallback stays idle
Throttled or out of quota	Retries the next provider down the chain on the same request	A reply, served by the fallback model
Endpoint unreachable	Keeps failing over on each new request until the primary recovers	Slightly different tone and latency, but a working assistant
Every provider fails	Returns the error rather than hanging	An error worth chasing with hermes status

Figure 4: The chain in practice. The fallback only earns its keep in the middle two rows, which is exactly why it is easy to forget you configured it.

Part 3: Promoting Foundry to the Primary Model

At this stage Foundry is sitting in the back seat as a backup. I wanted to reverse the arrangement and make Foundry the primary while Bedrock slides down into the fallback slot, partly because I preferred keeping day-to-day traffic inside my Azure subscription, and partly because I wanted proof the chain works in both directions.

Before promotion		After promotion
Primary: Amazon Bedrock	→	Primary: Microsoft Foundry
Fallback: Microsoft Foundry	→	Fallback: Amazon Bedrock

Figure 5: The promotion, in effect. Nothing is added or removed; the two providers simply trade places in the chain.

There is no dedicated "promote" command in Hermes, so the manual route is a short sequence of steps rather than a single instruction:

Select the fallback provider/model as the new primary: hermes model
Remove the now-duplicate model from the fallback chain: hermes fallback remove
Optionally add the old primary model as a fallback: hermes fallback add
Restart the messaging gateway: hermes gateway restart
Verify the result: hermes status / hermes fallback list

That sequence works, and it is good to know what is happening underneath. But since I already had a working provider configured, I would rather just ask Hermes to rearrange itself. This is the part I genuinely enjoy about the tool: the configuration is something you can talk to, not only something you type commands at.

Prompt: Okay now please make the model I configured on Microsoft Foundry into the main model, and make the Bedrock one the fallback model!

Hermes rewrites the provider chain on its own and confirms the swap once it is done, which is a great deal less error-prone than running the five commands by hand.

Part 4: Testing the Switch

Configuration you have not tested is just an assumption with extra steps, so the next thing is to confirm Hermes really is talking to Foundry.

Type /model when running Hermes to bring up the model picker.
You will be prompted to select a provider first. Pick the Microsoft Foundry entry, then choose the specific deployment from the list underneath it.

The active model should switch straight away. Send it a plain "Hello" to check that the deployment actually responds, rather than just looking correct in the menu. Appearing in a dropdown and serving a request are two very different things.

A second test is worth the thirty seconds it costs: run hermes status to confirm which provider is live, then hermes fallback list to confirm the chain is ordered the way you intended. The picker tells you what you selected; those two commands tell you what Hermes will actually do at three in the morning.

Part 5: The Obstacle, and What It Actually Taught Me

Every walkthrough has the part the author quietly leaves out. Here is mine: the wrinkle was not the model, it was capacity. My first attempt deployed into the region closest to me out of habit, and the portal turned it down because there was no capacity left for that model at the tier I asked for. The model was clearly listed in the catalogue; being listed and being deployable in your region, on your subscription, at your quota, are three separate questions.

Redeploying in a different region fixed it in a couple of minutes, but it meant the endpoint URI changed, which in turn meant the value I had already pasted into Hermes was stale. Re-running hermes fallback add against the new endpoint sorted it out.

The lesson is cheap enough to hand over for free: check quota and regional capacity for your specific subscription before you design a walkthrough, a demo or a production fallback around one deployment. In the Foundry portal, Management then Quota shows exactly what you have been granted per region and per model family, which is the only list that matters.

Symptom	Likely cause	Fix
Deployment rejected in the portal	No capacity for that model at the tier you asked for, in that region	Deploy in another region, or drop to a smaller tier
Hermes rejects the credential	Endpoint URI does not match the deployment you created	Re-copy the target URI from the deployment, not the resource
Provider authenticates but lists nothing	Key belongs to a different Foundry resource	Check you are in the right resource, then re-run hermes fallback add
Fallback never fires	Chain ordered the wrong way round	hermes fallback list, then reorder

Figure 6: The four things that went wrong, or nearly did, and what fixed each one.

There is a silver lining worth stating plainly. Because the fallback chain was already in place, a deployment that refused to come up did not take the assistant down with it. That is precisely the scenario this whole exercise was meant to cover, and it turned up on day one without me having to simulate it.

Command Cheat Sheet

Everything used in this walkthrough, collected in one place:

hermes fallback add: attach a provider to the fallback chain hermes fallback remove: drop a provider from the chain hermes fallback list: show the chain in priority order hermes model: set the primary model hermes gateway restart: restart the messaging gateway after a change hermes status: confirm which provider is currently live /model: switch models from inside a running session

Conclusion

Adding Microsoft Foundry as a fallback behind my existing Bedrock setup took an afternoon, and most of that was spent recovering from a regional capacity limit I should have checked first. The work itself is small: deploy a model, copy the endpoint and key, run hermes fallback add, authenticate, pick your deployments. The payoff is that Hermes no longer depends on one provider staying healthy.

Three things are worth carrying away from this:

Check quota, not just the catalogue. The Foundry catalogue shows what Microsoft offers. It does not show what your subscription and region can actually deploy today. Confirm that first, before you build anything on top of a specific deployment.
Order your fallback chain deliberately. Hermes works down the list from top to bottom, so the sequence you choose during setup is the failover policy you are going to live with. Put the model you actually trust at the top.
Treat the endpoint as part of the credential. Redeploying in a new region changes the endpoint URI, and a fallback pointed at an endpoint that no longer exists is not a fallback. Re-run the setup whenever the deployment moves.

The switch from Bedrock primary to Foundry primary also proved the chain runs in both directions, which is the real point. Provider redundancy is only useful if you have watched it work. Next on my list is deliberately breaking the primary provider to confirm the failover triggers on its own, without me typing a single command.

If you run this against a different model, region or provider pairing, I would genuinely like to know how it went, particularly if your quota experience was better than mine. Drop it in the comments.

Agent Governance Toolkit: Giving AI Agents Rules of the Road

shreyanfern — Tue, 25 Aug 2026 07:00:00 GMT

Organizations are increasingly deploying autonomous AI agents directly to production environments. These agents ingest customer support tickets, query database states, and execute automated refund transactions. Operating with high throughput and minimal latency, they frequently hold service principal credentials scoped with write access to core infrastructure.

This architecture introduces a critical security boundary question: What prevents an autonomous agent from executing an unauthorized transaction or destructive operation?

Without strict guardrails, an agent processing unmanaged user inputs remains vulnerable to adversarial indirect prompt injections hidden within customer attachments or support tickets. Furthermore, LLM non-determinism means a tool-calling sequence can hallucinate invalid or destructive parameter arguments during execution.

Relying solely on system prompt instructions and base model safety fine-tuning leaves a system without a true security perimeter. System prompts represent soft boundaries; production-grade agent governance requires deterministic enforcement and explicit policy execution.

To address these security boundaries, Microsoft open-sourced the Agent Governance Toolkit (AGT) in April 2026. This post examines the architectural significance of AGT across both technical implementation and resource allocation perspectives, followed by a hands-on walkthrough demonstrating real-time policy enforcement within a 15-minute deployment sequence.

Why "Trust the Prompt" Was Never a Robust Security Model

Most engineering teams initially attempt to govern AI agents using standard application security patterns: OAuth scopes and Identity and Access Management (IAM) policies dictate system connectivity. While foundational, perimeter-level authorization is inherently insufficient for autonomous runtimes. Once an agent receives valid authentication tokens for a downstream CRM, vector database, or file system, OAuth provides zero introspection into downstream intent.

The decision between executing an idempotent read operation versus issuing a destructive bulk-deletion payload occurs entirely within the model's intermediate reasoning trace. Because that execution loop relies on non-deterministic language models, it remains vulnerable to semantic manipulation, context degradation, and hallucinated logic paths.

AGT : Traditional ModelAGT: Governance Model

Directives embedded within system prompts—such as "do not execute destructive actions"—function as probabilistic heuristics rather than deterministic controls. The architectural philosophy behind modern governance frameworks, including the Agent Governance Toolkit, establishes that unauthorized operations must be rendered structurally impossible. Enforcement is decoupled from the model's inference loop and applied deterministically via pre-execution interceptors before any underlying tool call executes.

This shift reflects an evolving threat landscape. The OWASP Top 10 for Agentic AI categorizes vulnerabilities specific to autonomous runtimes, including goal hijacking, unmanaged parameter poisoning, and cascading failures across multi-agent graphs. Simultaneously, regulatory compliance frameworks such as the phased enforcement milestones of the EU AI Act for high-risk autonomous systems mandate verifiable guardrails, auditability, and execution constraints. In this environment, deterministic governance shifts from an architectural best practice to a critical compliance requirement for enterprise deployments. Governance isn't a "nice to have" anymore for teams shipping agents into production; it's becoming table stakes for compliance conversations.

What is Agent Governance Toolkit ?

Think of AGT as a set of traffic laws and a black-box flight recorder for your AI agents, sitting between "the agent decided to do something" and "the something actually happened.", At an architectural level, the Agent Governance Toolkit (AGT) functions as an inline interception layer and immutable audit engine positioned directly between the LLM's action generation phase and actual tool execution.

Rather than modifying underlying model weights or prompt logic, AGT introduces deterministic policy enforcement through established principles derived from Application Security (AppSec) and Site Reliability Engineering (SRE):

Policy Enforcement Engine: Declarative rules defined in standard YAML schemas that establish explicit boundaries for tool parameters and runtime actions without requiring imperative custom logic.
Cryptographic Identity (Trust Layer): Strong workload identity for autonomous agents using open standards such as SPIFFE (Secure Production Identity Framework for Everyone) and Decentralized Identifiers (DIDs), eliminating shared credentials and enabling verifiable agent attribution across multi-agent graphs.
Runtime Isolation & Control: A sandboxed execution environment featuring privilege ring boundaries to limit system access, paired with programmatic kill-switch mechanisms for real-time containment during anomalous behavior.
Tamper-Evident Evidence Logging: Cryptographically signed execution trails that log every policy evaluation, parameter validation, and interception decision for compliance and forensic auditing.
Protocol Governance: Granular control over inter-agent communications and external tool interactions over modern agent communication protocols, including the Model Context Protocol (MCP).

AGT: Architecture

The Agent Governance Toolkit operates as a modular, sub-millisecond control plane that decouples policy enforcement and zero-trust identity from underlying agent frameworks. By pairing a deterministic OS engine and sandboxed runtime with cryptographically signed delegation chains, AGT intercepts tool invocations, enforces SRE guardrails, and maintains Merkle-anchored audit logs before any action reaches external infrastructure.

╔══════════════════════════════════════════════════════════════════════════╗ ║ AGENT GOVERNANCE TOOLKIT v5.0.0 ║ ║ pip install agent-governance-toolkit[full] ║ ║ ║ ║ Agent Action ──► POLICY CHECK ──► Allow / Deny (< 0.1 ms) ║ ║ ║ ║ ┌──────────────────────────┐ ┌──────────────────────────────┐ ║ ║ │ AGENT OS ENGINE │◄───►│ AGENTMESH │ ║ ║ │ │ │ │ ║ ║ │ ● Policy Engine │ │ ● Zero-Trust Identity │ ║ ║ │ ● Capability Model │ │ ● Ed25519 / SPIFFE Certs │ ║ ║ │ ● Governance Gate │ │ ● Trust Scoring (0-1000) │ ║ ║ │ ● GovernanceEventSink │ │ ● Wire Protocol (A2A/MCP) │ ║ ║ │ ● Decision BOM │ │ ● Delegation Chains │ ║ ║ └────────────┬─────────────┘ └───────────────┬──────────────┘ ║ ║ │ │ ║ ║ ▼ ▼ ║ ║ ┌──────────────────────────┐ ┌──────────────────────────────┐ ║ ║ │ AGENT RUNTIME │ │ AGENT SRE │ ║ ║ │ │ │ │ ║ ║ │ ● Execution Rings (0-3) │ │ ● SLO Engine + Error Budgets│ ║ ║ │ ● Resource Limits │ │ ● Replay & Chaos Testing │ ║ ║ │ ● Runtime Sandboxing │ │ ● Progressive Delivery │ ║ ║ │ ● Termination Control │ │ ● Circuit Breakers │ ║ ║ └──────────────────────────┘ └──────────────────────────────┘ ║ ║ ║ ║ ┌──────────────────────────┐ ┌──────────────────────────────┐ ║ ║ │ AGENT HYPERVISOR │ │ AGENT LIGHTNING │ ║ ║ │ │ │ │ ║ ║ │ ● Execution Audit │ │ ● RL Training Governance │ ║ ║ │ ● Delta Engine │ │ ● Violation Penalties │ ║ ║ │ ● Commitment Anchoring │ │ ● Reward Shaping │ ║ ║ │ ● Merkle Chain Logs │ │ ● Training Checkpoints │ ║ ║ └──────────────────────────┘ └──────────────────────────────┘ ║ ║ ║ ║ ┌──────────────────────────┐ ┌──────────────────────────────┐ ║ ║ │ AGENT MARKETPLACE │ │ MCP SECURITY GATEWAY │ ║ ║ │ │ │ │ ║ ║ │ ● Plugin Discovery │ │ ● Tool-Call Policy Checks │ ║ ║ │ ● Signing & Verification│ │ ● Trust Verification │ ║ ║ │ ● Trust Scoring │ │ ● Rate Limiting │ ║ ║ └──────────────────────────┘ └──────────────────────────────┘ ║ ║ ║ ║ ┌──────────────────────────────────────────────────────────────┐ ║ ║ │ FRAMEWORK ADAPTERS │ ║ ║ │ LangChain · CrewAI · AutoGen · OpenAI · ADK · smolagents │ ║ ║ └──────────────────────────────────────────────────────────────┘ ║ ║ ║ ╚══════════════════════════════════════════════════════════════════════════╝

AGT provides native SDK support across mainstream development stacks including Python, TypeScript, .NET, Rust, and Go and offers pluggable adapters for major agentic frameworks. This architecture allows organizations to retrofit existing multi-agent pipelines by wrapping critical tool interfaces rather than re-architecting underlying workflows.

Hands-On Implementation: Enforcing Runtime Policies in Python

To demonstrate real-time interception, the following exercise configures a deterministic guardrail using the Python SDK. The setup simulates a scenario where an agent attempts an unauthorized, high-value tool call.

The core mechanics—defining declarative policy schemas, initializing pre-call interceptors, and handling violation exceptions—remain consistent across all supported SDK runtimes.

Prerequisites

Environment: Python 3.10 or higher
Dependencies: agent-governance-toolkit installed via pip

Step 1: Install the Toolkit

Initialize a clean virtual environment on a code editor and install the governance package within the virtual environment:

pip install agent-governance-toolkit[full]

Step 2: Write a policy

Governance rules live in plain YAML, not scattered across the codebase. Create a folder called policies and add policy.yaml,

apiVersion: governance.toolkit/v1 name: file-agent-policy version: "1.0" default_action: allow rules: - name: block-destructive condition: "action.type in ['drop', 'delete', 'truncate']" action: deny description: "Destructive operations require human approval"

In very simple terms, the above YAML defines and allows everything by default, except any action typed as drop, delete, or truncate those get denied outright.

Step3: Lint it

Before trusting a policy, lets validate it with the Toolkit, In the terminal type the following,

agt lint-policy policies/

In response, it displays "No issues Found" incase all the definitions are valid.

AGT: agt lint response

Step 4: Wrap your tool in two lines

Integrating deterministic governance does not require refactoring an agent's core orchestration logic or LLM prompting loops. Instead, AGT hooks directly into the execution path by wrapping the individual tool functions exposed to the agent. Here's a simple file-handling tool, governed,

from agentmesh.governance import govern, GovernanceDenied def file_tool(action, path): return f"Executed '{action}' on {path}" safe_tool = govern(file_tool, policy="policies/policy.yaml", agent_id="demo-agent") # Case 1: benign read action print("Read call ->", safe_tool(action="read", path="report.txt")) # Case 2: destructive action the policy should block try: safe_tool(action="delete", path="report.txt") except GovernanceDenied as e: print("Delete call -> BLOCKED:", e)

Step 5: Run it and watch the policy do the job

Save the above file with a name of choice and execute the following in the terminal,

python <name of the file>.pyAGT:Output

That's it. The read call sailed through under the default-allow rule; the delete call was intercepted and stopped before it ever reached the function, no model judgment call involved, no reliance on the agent "choosing" to behave. That's what "structurally impossible" looks like in practice.

Step 6: Check your compliance posture

One more command worth knowing, especially if ever needed to show a security or compliance team that all the agents are governed, Type the following command in the terminal,

agt verify

AGT: Verify

This generates a report and a badge can be dropped straight into any repo mapped against the OWASP Agentic Security Initiative's ten risk categories. For a technical reviewer, that's a checklist. For a decision-maker signing off on an AI rollout, that's evidence.

Strategic and Risk Governance Implications

From an executive, risk, and compliance perspective, the Agent Governance Toolkit does not aim to improve model intelligence or rely on probabilistic prompt alignment. Instead, it introduces a deterministic, application-layer checkpoint directly in front of every agentic invocation. As a result, critical failure modes such as an unauthorized database deletion or invalid financial transaction are transformed from uncontained operational incidents into pre-defined policy violations that are deterministically blocked and logged.

Organizations implementing AGT can evaluate their agent governance maturity around three core operational artifacts:

Audit Traceability: The ability to generate an immutable, cryptographically verifiable audit trail of every governed decision and parameter validation on demand via AGT's evidence layer.
Workload Identity Attribution: Explicit agent-level identity tracking through AgentMesh standards (SPIFFE/DIDs), eliminating ambiguous shared service credentials in multi-agent environments.
Compliance Framework Mapping: Deployment validation reports generated via agt verify, providing a direct pass/fail audit mapped against the OWASP Top 10 for Agentic Applications for regulatory and vendor risk reviews.

Establishing these control capabilities requires no modifications to business logic, relying instead on standardized CLI reporting and declarative policy enforcement.

Advanced Capability Extensions & Ecosystem Integration

The single-rule policy example demonstrates the baseline execution pattern of the Agent Governance Toolkit, but enterprise deployments extend into broader architectural patterns. The framework provides comprehensive controls for complex production environments:

Human-in-the-Loop Orchestration: Declarative require_approval workflows that automatically intercept high-risk operations and route tool payloads to human authorization gates before execution.
Privilege Ring Isolation: Runtime sandboxing mechanisms that restrict system access levels based on assigned agent capabilities.
Multi-Agent Identity Federation: Cryptographic agent identity verification designed to secure delegation chains and inter-agent communication in multi-agent architectures.
Native Framework Adapters: Pluggable middleware integration for existing orchestration stacks, enabling direct governance injection into frameworks like LangGraph, AutoGen, and Semantic Kernel without requiring pipeline refactoring.

For full end-to-end integration patterns including wrapping multi-node LangGraph workflows with deterministic policy gates refer to the official Microsoft Agent Governance Toolkit documentation and architecture blueprints.

Production-Grade Governance as the Baseline for Autonomous AI

As AI agents transition from experimental sandboxes to enterprise production environments, reliance on soft safety guidelines and system prompt instructions is no longer a viable security posture. Autonomous systems holding write access to critical data sources demand the same deterministic controls, zero-trust identity boundaries, and immutable audit trails long expected of human operators and microservice architectures.

Agent Governance Toolkit establishes a clear paradigm: governance is not an afterthought appended to post-hoc incident response, but an inline, sub-millisecond control plane built directly into the runtime execution path. By decoupling policy enforcement from model reasoning, organizations can safely accelerate agentic adoption ensuring every autonomous action remains strictly bounded, fully traceable, and compliance-ready.

References:

Docs & quickstart: microsoft.github.io/agent-governance-toolkit
Source & examples: github.com/microsoft/agent-governance-toolkit
Announcement post: Introducing the Agent Governance Toolkit — Microsoft Open Source Blog
Architecture deep dive: Agent Governance Toolkit: Architecture Deep Dive — Microsoft Tech Community

Understanding Harnesses: The Concept That Changes How You Build Agents in Copilot Studio

JonEricEubanks — Mon, 24 Aug 2026 07:00:00 GMT

That changed on August 3, 2026, when Microsoft made the GitHub Copilot harness generally available. Copilot Studio now ships three harnesses, and choosing between them is the first real architecture decision you make when starting a new agent.

This is a good problem to have. It means the platform stopped assuming every agent is the same kind of thing. But "harness" is a genuinely new word for most makers, so let's start there.

What a harness is

Microsoft defines a harness as the runtime layer between the language model and your agent's configuration. It decides what context reaches the model, when tools get invoked, how responses are interpreted, and how the agent moves through a task toward completion.

The harness sits between what you configure and the model that reasons over it.

The analogy Lisa Crosbie uses is the best one I have heard: the model is a newly hired employee, arriving with knowledge, language, and reasoning ability. The harness is everything you give them afterward: the laptop, the system access, the team around them, the policies, the tools. You hired the brain. The harness is what lets the brain do your work.

Before this release, that layer was invisible, because it was singular. Now it is a deliberate choice, which is a meaningful step toward Copilot Studio being an agent engineering environment rather than a chatbot builder.

The three harnesses

Model choice is the headline

Here is the part I would lead with if I only had one sentence: the GitHub Copilot harness is model-agnostic, and the model you choose is the agent's main brain.

This is different from what came before. Previously, you could bring a Foundry model into a specific prompt, useful but scoped. Now frontier reasoning models like Opus 5, GPT-5.6, and Fable 5 can run the whole agent. Microsoft's own framing, from Ryan Cunningham's launch walkthrough, is that the harness hosts an advanced reasoning model in its own computing environment with a set of tools and skills, and lets it loop over hard problems.

Available models depend on what your organization has enabled.

Two practical notes. Your organization controls which models are enabled, so what you see in the picker depends on admin configuration. And model availability varies by region; some regions cannot use certain providers. Check what is actually available in your tenant before designing around a specific model.

If you have been building agents for a while, this is the change that most alters what is possible. The Standard harness could not reach these capabilities. That gap is why the new harness exists.

Standard harness: explicit construction

The Standard harness is the model most of us learned on. You author topics. You define triggers. You build branching paths and route into flows. Behavior is constructed explicitly, step by step.

It remains fully supported, and it is still an excellent fit for real work. A service desk agent answering a known set of questions, a scheduling assistant routing into a defined flow, an onboarding agent walking a new hire through a fixed sequence, you want these predictable, and explicit construction gives you that. Some conversational patterns genuinely need a scripted path.

Explicit construction makes control flow visible; you can see exactly why the agent did what it did.

If you are learning agent development, this harness teaches you something valuable: it makes control flow visible. You can see exactly why the agent did what it did, because you drew the path.

Two things to know. Existing Standard harness agents keep running exactly as they are; nothing breaks. But to edit them, you work in the classic interface. The two experiences are separate surfaces.

GitHub Copilot harness: goal-directed reasoning

The new harness takes a different approach. Rather than authoring conversation paths, you describe the agent in natural language, and the orchestration runtime interprets those instructions at runtime.

The shape of the reasoning changed too. The older orchestrator worked in a fairly linear pattern: take the input, make a plan, collect what it needed, execute. The new one runs a continuous loop: evaluate the goal, plan, call tools, examine results, adjust, continue until the objective is met. That loop is what lets it handle ambiguity, recover from a step that did not work, and stay on track through multi-stage problems.

The older orchestrator made a plan and executed it. The new one keeps thinking.

Microsoft's published comparison figures show meaningful gains, multi-tool use improving from 87 percent to 98 percent, with reported improvements in file analysis, code analysis, and knowledge work. Worth verifying in the current documentation, since these are the kind of numbers that get updated.

What comes with it:

Skills: reusable Markdown instruction sets, attachable across agents, exportable as files. They can bundle artifacts too, including documents and Python.
Memory: persistent context across interactions, including user preferences.
Connected agents: a front-door agent routing to specialists, each owned separately.
Sandboxed execution with large file processing and native Word, Excel, PowerPoint, and PDF generation.
Build, Preview, Evaluate, Monitor: four tabs, consolidated from nine.

The classic interface had nine tabs: Overview, Knowledge, Tools, Agents, Topics, Activity, Evaluation, Monitor, and Channels.

The new GitHub Copilot harness consolidates everything into four tabs: Build, Preview, Evaluate, Monitor.

That Evaluate tab deserves its own mention. Cunningham's framing is that evaluation matters more than ever, giving agents a hard set of problems and scoring how they perform as you refine instructions and skills. Evaluation moving into the primary authoring surface rather than a separate tool is a real signal about where agent development is heading. If you are a student building agents now, getting comfortable with evaluation early will serve you well.

Topics are gone. Skills replaced them.

In the new harness, topics do not exist. This surprised a lot of makers, so it is worth understanding why rather than just noting it.

Topics were a way to script a conversation, useful when the path must go a certain way, but not how frontier models think. Skills are the replacement, and they work differently: a Markdown file containing a set of instructions the agent can pull in on demand, at the moment it is relevant, rather than one enormous instruction block covering every case.

This solves a problem anyone who has written long agent instructions will recognize. Instead of bloating a single instruction set, you decompose behavior into procedure manuals the agent reaches for as needed.

A skill is a procedure manual the agent picks up when it needs it

The reusability is the interesting part. Author a skill once, attach it to several agents, export it, share it. And the format is spreading; skills are appearing in Cowork, PowerPoint, Word, and across GitHub Copilot. That makes a skill file a portable, human-readable, version-controllable unit of agent behavior.

For anyone teaching this material, that is a genuinely good pedagogical object. A skill file is small enough to read in a class session and real enough to ship. If you experiment with one thing this month, make it authoring a skill by hand and attaching it to more than one agent.

Agents and workflows are now two equal pillars

This is the structural change people are underrating.

Neither is the "advanced" option. They compose. An agent reasoning through a problem can call a workflow when it needs a reliable sequence. A workflow running its steps can invoke an agent at the step that requires judgment. Cunningham makes a point worth repeating: structuring work this way is also more credit-efficient, because you spend model reasoning only where it adds value.

The workflow designer was rebuilt for this: horizontal canvas, agents embeddable as nodes, and node-by-node testing so you validate each step as you build rather than running the whole thing and hoping.

Agents drop into workflows as nodes. Node-by-node testing means you validate as you build.

For municipal work, this maps cleanly. A permit intake might reason over an ambiguous description to decide which department owns it, then hand it to a deterministic workflow for notification, record creation, and routing. Reasoning where reasoning is needed; predictable steps everywhere else.

How to choose

The question I ask at the start of every build:

Is the hard part of this agent the retrieval, or the reasoning?

If the hard part is retrieval, ground it well and keep behavior predictable and the maintenance surface small. You are not missing out, you are matching the tool to the problem.

I build Power Platform solutions for local government, so my examples come from there: a grant eligibility screen reading a federal notice against a city's programs, a code compliance review spanning several ordinances, an intake deciding which of six departments owns a request. Every one is reasoning work. Under the old model I was fighting the platform to express them.

Your domain has its own version of that list. The sorting question is the same. The documentation includes a comparison table with reasons for and against each harness, worth reading before your next build.

Two things to know before you start

Harness choice is set at creation. Agents cannot be converted between harnesses in either direction. Different architectures, different runtimes. Existing agents keep running. Know this up front so you choose consciously; ask the retrieval-or-reasoning question during design, not after.

Billing works differently on the new harness. Agents powered by the GitHub Copilot harness use Copilot Credits, and per Microsoft's documentation, that applies to using, building, testing, and evaluating. Authoring an agent through natural language consumes credits. The Copilot Chat and Standard harnesses continue on the existing rate card, and Standard harness agents remain included for licensed users when deployed to Teams or Microsoft 365 Copilot.

A credit is roughly one US cent. Those figures come from Microsoft's published examples and are worth confirming against the current credits guide, but the useful point is that the cost is knowable rather than mysterious.

Administrators can allocate credits per environment and review consumption.

For students and educators: check what your tenant or lab environment provides before building. For anyone deploying at scale, the pre-purchase plan offers a discount and its credits are consumed over twelve months rather than resetting monthly, which gives room to experiment.

The honest framing is Crosbie's: the costs are not trivial, but the capability is substantial, and the skill worth building now is evaluating cost against value rather than avoiding the question.

Getting started

Create a new agent from the Copilot Studio homepage, and you will choose a harness. Build something small and real on the GitHub Copilot harness, a task from your own work or coursework, scoped to three sentences. Watch the Preview and Evaluate tabs as you iterate. Note what the credit consumption actually looks like on your own tenant. You will learn more in an afternoon than from any writeup, including this one.

Then revisit something you have already built and ask the retrieval-or-reasoning question. Some of your existing agents are in exactly the right place. Knowing which ones is its own kind of progress.

What is still rolling out

The GitHub Copilot harness reached GA on August 3, 2026, so production experience across the community is still accumulating, mine included. The knowledge source options in the new experience are currently a somewhat reduced set compared to the classic interface and will expand. Parts of the wider rebuild remain in preview.

Benchmark figures, model availability, and credit costs are best confirmed directly in Microsoft Learn rather than from secondary coverage, including mine. If you build for regulated or government environments, confirm feature availability in your specific cloud before designing around it.

Learn more: Agents overview, GitHub Copilot harness · What's new in Copilot Studio

Build AI agents without leaving VS Code, join our 3-part Reactor Series

bethanyjep — Sun, 23 Aug 2026 07:46:38 GMT

Starting August 25, we're running a free three-part live series on Microsoft Reactor that takes you from "what is a hosted agent?" to a deployed, orchestrated multi-agent application, all built in VS Code with the Microsoft Foundry Toolkit.

👉 Register for the series

Building agents, without leaving your IDE.

Building an AI agent usually means context-switching between a portal, a terminal, a notebook, and three browser tabs of documentation. You write some instructions, deploy, wait, test in a playground, discover the agent hallucinated a policy that doesn't exist, and start over. The feedback loop is measured in minutes, not seconds.

The Microsoft Foundry Toolkit for VS Code collapses that loop. You generate an agent from extension using the Command Palette, configure the model and instructions, debug it locally against Agent Inspector, and deploy to Microsoft Foundry Agent Service.

What you will Build

Explain like I am an Executive agent

We've all written the perfect post-mortem, root cause, timeline, remediation, only to get back: "So… is the website down or not?"

Session 2 builds a single-purpose agent that takes this:

"The API latency increased due to thread pool exhaustion caused by synchronous calls introduced in v3.2."

…and returns this:

Executive Summary:

- What happened: After the latest release, the system slowed down.

- Business impact: Some users experienced delays while using the service.

- Next step: The change has been rolled back and a fix is being prepared before redeployment.

The Resume → Job Fit Evaluator

Session 3 steps up to a multi-agent workflow: four agents that collaborate to parse a résumé and a job description, extract the real requirements, score alignment, identify gaps, and generate a personalized learning roadmap backed by Microsoft Learn resources.

This is where you see orchestration patterns, agent-to-agent handoffs, and how to keep a multi-agent system debuggable .

The sessions

Title	Description	Date and Time	Speaker
1️⃣ What Is Microsoft Foundry Toolkit for VS Code? What's New and How to Get Started	Foundation session touring the toolkit's workflow: agent creation, model configuration, local debugging with Agent Inspector, and deployment to Microsoft Foundry Agent Service.	Tuesday, August 25 12:00 PM UTC	Junjie Li Senior Product Manager, DevDiv, Microsoft
2️⃣ Build Your First Microsoft Foundry Agent: Executive Briefings from Technical Updates	Hands-on build session creating a Python hosted agent, authoring instructions, testing safety boundaries in Agent Inspector, and deploying to the cloud.	Tuesday, September 1 2:00 PM UTC	Bethany Jepchumba Cloud Advocate, Microsoft
3️⃣ Build a Multi-Agent Career Copilot: Resume-to-Job Fit Analysis	Orchestration session building a 4-agent collaborative team that parses résumés and job descriptions, scores alignment, finds gaps, and generates learning roadmaps.	Thursday, September 3 3:00 PM UTC	Shivam Goyal Microsoft MVP (AI)

Follow along on GitHub....

Every session maps to our workshop in the companion repo with 50+ languages supported, so you can build alongside us or catch up afterward:

👉 Check out the repo

Save your seat and see you there

Sessions run on August 25, September 1, and September 3. Register once for the series and you're in for all three, and you'll get the recordings even if you can't make it live.

👉 Register for the full series

Bring your questions and see you there. 🚀

CivicGrant IQ: The Five-Agent AI That Reads Federal Grants So Cities Don't Have To

JonEricEubanks — Fri, 31 Jul 2026 07:14:35 GMT

Guest post by JonEric Eubanks winner of Best use of Microsoft IQ series for Agents League 2026 Hackathon.

The meeting that started this

Every city has the same meeting. Finance finds a federal grant that looks relevant. A department director, a project manager, and a grants coordinator gather around an 80-page Notice of Funding Opportunity (NOFO) and spend two hours trying to answer three questions: Do we qualify? Which of our projects fits? Is it worth the effort?

Nobody has a good answer by the end. The NOFO is written for federal program officers, not city staff. So most of the time the city does nothing, the deadline passes, and real, appropriated money goes unclaimed.

I've built production tools for local government for years, including an app that won Microsoft's global Best in Automation award. The single biggest gap I keep seeing is the grant lifecycle: finding the right opportunity, proving you qualify, drafting the application, and tracking compliance is slow, fragmented work. A city of 40,000 might have one part-time person responsible for all of it, and asking them to spend 80–200 hours on an uncertain application means deprioritizing everything else.

So for the hackathon, I built the analyst that actually has the answers.

What it does

Paste a federal NOFO, or a URL, or just a description, and CivicGrant IQ streams a six-step reasoning chain to the screen in under 60 seconds:

Parse the grant's requirements, award ceiling, deadline, and eligibility rules
Match the city's Capital Improvement Plan projects and past applications against them
Verify cost-share capacity against the city's live financial profile
Score every gap by severity, with a specific strategy to close each
Draft a 150–200 word narrative anchored in the city's closest winning application
Plan a four-week action calendar with department owners

While that runs, two more agents work in parallel: Competitive Intel maps the likely field, and a Red Team GS-14 Reviewer scores the draft like a skeptical senior federal reviewer. When it's done, a live dashboard renders inline, match gauge, gap cards, timeline, and every figure traces to a cited source document.

The architecture

Four Microsoft IQ layers, plus a GraphRAG reasoning layer, all load-bearing:

Foundry IQ (Azure AI Search) grounds every city-specific claim in real documents. The model can't invent a strength it can't source.
Work IQ pulls four live Microsoft 365 signals at query time, calendar deadlines, flagged grant emails, SharePoint project docs, Teams context, so the agent knows what the city is working on now.
Fabric IQ connects to a live Microsoft Fabric lakehouse for real disbursement rates and compliance status. "100% compliance" is a live SQL query, not a cached claim.
Web IQ pulls the live Grants.gov feed for real-time open programs and award ceilings, ranked by fit and refreshed daily, so the opportunity list is never stale.
GraphRAG traverses a typed entity graph so every reasoning hop carries its own citation, not just the final answer.

Under the hood: Azure OpenAI (gpt-4o-mini) via the Assistants API, an Express/TypeScript backend streaming SSE events step-by-step, and a React 18 frontend that renders the reasoning chain and dashboard live as they stream.

Two patterns worth stealing

The widget protocol. A chat streams text, but a grant tool needs a dashboard. Instead of building two UIs, the agent embeds a JSON block in its response. The server strips it from the display text and fires it as a separate event:

// Strip the ```widget block from the text the user sees… const WIDGET = /```widget\s*([\s\S]*?)```/; const clean = text.replace(/```widget[\s\S]*?```/g, "").trim(); // …then emit the parsed JSON as its own SSE event. const match = text.match(WIDGET); if (match) { try { send("widget", JSON.parse(match[1])); } catch { /* malformed → the text answer is still complete */ } }

The frontend renders a live dashboard inline. If the JSON is malformed, the component just doesn't mount, the text answer is still complete.

The three-tier fallback. Every API fails at the worst moment, usually mid-demo. Tier 1 is the Assistants API with Foundry IQ. If it fails, Tier 2 rebuilds the same prompt as a Chat Completions call. If both are down, Tier 3 serves deterministic results with no API calls. Result: zero failures across twelve demo runs, two of them on conference Wi-Fi.

Results

$1.25B in verified open federal funding matched across 10 programs for one mid-sized city
91% match score for FEMA BRIC, grounded in the city's real CRS rating and 2025 application
Three submission-ready drafts generated from precedent in under 90 seconds each
Named Best IQ Agent from a field of nearly 5,000 contestants

The metric I care about most: a grants coordinator can now make a real go/no-go decision in under an hour. It used to take a week.

What I learned

Make the reasoning visible. A streamed, sourced six-step chain is evidence. An opaque score is just a claim.
Real data beats a curated demo. Judges could open Grants.gov and verify the numbers on the spot. That moment is worth more than any diagram.
Grounding is a discipline, not a feature. A model that can't invent strengths is a better analyst than one that can.
The fallback is the product. Government staff will run this mid-presentation. It can't fail.

Try it or build on it

Live app: https://proud-field-00978990f.7.azurestaticapps.net
Demo (5 min): https://youtu.be/dTRjA8eKwgQ
GitHub: https://github.com/JonEricEubanks/CivicGrant-IQ

The patterns transfer anywhere you have a dense document corpus, a user without time to search it, and a need for cited, real-time analysis, procurement, compliance, and nonprofit grants. Every year, billions in federal grants go unclaimed by the cities they were built for. Not because they don't qualify, because knowing they qualify is invisible work done by people already doing three other jobs. That's solvable.

ARGUS: Compliance Infrastructure That Believes Financial Access Is a Human Right

iarjunganesh — Thu, 30 Jul 2026 07:00:00 GMT

Guest post by 𝐀𝐫𝐣𝐮𝐧 𝐆𝐚𝐧𝐞𝐬𝐡 Winner of Hack for Good category Microsoft Agents League Hackathon

A refugee family in Germany spends 18 months trying to open a bank account. Their documents are legitimate. But an automated KYC system — never designed with them in mind — scores their jurisdiction as high risk and closes the case. No human review. No plain-language explanation. No appeal path.

An NGO doing legitimate microfinance work in Southeast Asia gets de-risked by their correspondent bank. The letter cites "risk appetite." The lending operation, which supports 4,000 families, can no longer move money.

These aren't edge cases. 1.4 billion people remain financially excluded globally — and compliance systems built to protect institutions are a leading cause. The same technology meant to stop financial crime routinely shuts out the people who most need access.

That's the problem I set out to fix with ARGUS (Agentic Risk & Governance Unified Screening) at the Microsoft Agents League — AI Skills Fest 2026, where it was selected as 1 of 3 Hack for Good winners — awarded to the best solutions to solve a community need.

What ARGUS does

ARGUS is compliance infrastructure that is explainable, accessible, and open — designed from the ground up for the humans most likely to be failed by the systems they depend on, not just for the institutions running the checks.

A single KYC request fans out across five specialist AI agents, coordinated via the Agent-to-Agent (A2A) protocol on Azure AI Foundry:

🎯 Orchestrator Agent — task decomposition and result synthesis
🪪 Identity Agent — customer lookup, OCR processing, identity validation
🔍 Screening Agent — sanctions checks, adverse media scanning, PEP checks
🏢 Corporate Intelligence Agent — UBO resolution, registry lookups, jurisdiction mapping
💳 Transaction Intelligence Agent — transaction monitoring, pattern detection, typology matching

Each of these fans back into a Compliance & Risk Agent, which synthesizes everything into a single traceable risk report.

The part I'm proudest of: nothing in that report is a black box. Every finding is grounded in cited regulatory knowledge via Foundry IQ — sanctions checks and regulatory triggers cite the exact knowledge base, source document, and article behind them. Every risk score comes with a plain-English reason. Every decision leaves a full audit trail: agent by agent, tool by tool, citation by citation.

Submit entity → 5 agents run in parallel → Compliance fan-in → Traceable risk report

📹 Watch the 5-minute demo

What's next: ARGUS v2

The hackathon build proved the architecture holds up. The v2 roadmap is where it grows into something NGOs and microfinance institutions can actually run:

Feature	Status
♿ Full WCAG 2.1 AA compliance	In progress
🗣 Explain Mode — plain-language reports for the person being screened	In progress
🌍 Community Edition — free, self-hostable, zero Azure subscription required	Designed
📊 Real-time Azure Monitor compliance dashboard	Designed
🔓 Open Knowledge Graph — FATF, Basel AML Index, OFAC/EU/UN lists as open data	Planned

Explain Mode is the one I'd highlight for an educator audience. Today's report is written for compliance analysts:

"Dimension score 78 — HIGH tier — regulatory trigger: FATF Recommendation 16 (KB-Regulations › FATF-40 › R.16)"

Explain Mode generates the same finding for the person being screened, or the NGO caseworker who has to explain a decision to a family:

"This account review flagged a potential concern with how money moved through the transaction chain. This is a standard check for large or cross-border transfers. A compliance officer will review this before a final decision is made. You don't need to do anything right now."

Same underlying data. Entirely different outcome for the person on the receiving end.

Community Edition is the biggest bet: a zero-cost, self-hostable configuration for NGOs, microfinance institutions, community banks, and researchers — a pre-seeded public knowledge base, a one-command Docker Compose stack, and a lower-cost LLM path, so an Azure subscription isn't a prerequisite for the core flow.

Try it yourself

ARGUS is open source under MIT. If you're teaching agentic AI, Azure AI Foundry, or Semantic Kernel, the repo is built to be a working example of multi-agent orchestration with A2A and grounded retrieval via Foundry IQ:

git clone https://github.com/iarjunganesh/argus.git cd argus pip install -r requirements.txt cp .env.example .env make generate-data && make generate-ocr-docs make index-knowledge-bases .\scripts\start_demo.ps1

The Community Edition — no Azure subscription required — is scaffolded in community/ as part of the v2 roadmap, but isn't runnable yet.

The most-needed contributions right now are WCAG contrast tests for the Gradio UI, translations of Explain Mode output, and finishing the Community Edition Docker Compose stack (Dockerfile + local knowledge base seed) — all good starter issues for students exploring accessible AI system design.

Repo: https://github.com/iarjunganesh/argus

Demo video: https://youtu.be/yaTNCgCwX4s

ARGUS is a technology demonstration and is not a licensed compliance tool — it must not be used to make real KYC/AML decisions.

Grounded AI That Won't Invent Facts: Building "Tell My Day" with Microsoft Foundry IQ

caden322 — Wed, 29 Jul 2026 07:00:00 GMT

Guest post by Caden Storkamp Winner of the Hack for Good Category Microsoft Agents League Hackathon

Large language models make things up. For a lot of apps that is a minor annoyance you can smooth over. For some, it is disqualifying, because the cost of a confident, fluent, false statement is real. This post walks through one of those cases and the pattern that makes it safe to ship: retrieve from a trusted knowledge base with Microsoft Foundry IQ, force the model to cite verbatim quotes, and then verify those quotes on the server before anything reaches the screen.

My name is Caden Storkamp, and the app I build is called Tell My Day. I work as a direct support professional, and in this role I work day to day with people who use AAC (augmentative and alternative communication). The use of the app I made is not just limited to it's primary goal, but can be also used as an example for others. If you are teaching or building anything where a generative feature has to be trustworthy, this is a small, readable, reproducible reference you can lift from.

The problem, and why it is the sharpest possible test

AAC (augmentative and alternative communication) is how many non-speaking people communicate: picture boards, symbol devices, picture exchange. These tools are good at "I want a drink" in the moment. They are much worse at letting someone tell a story, like how their day went, in their own voice. Caregivers or family members often end up narrating on the person's behalf.

Tell My Day flips that. The person picks the symbols, and the app writes the story. That immediately raises the stakes to the highest possible level for hallucination, because you are now generating first-person speech attributed to someone who cannot easily correct you. If the app writes "I went to the zoo today" when they did not, it has put false words in a non-speaking person's mouth, to the people they most need to trust them. So "never invent anything" is not just a great thing to have here. It is the whole product. That constraint is exactly why it makes such a clean showcase for grounded generation.

What the app does

A non-speaking adult taps 3 to 8 picture symbols (people, places, activities, feelings, food).
The app writes 2 to 4 short, warm, first-person sentences.
The story is grounded in the person's own "About Me" profile, so a generic "my friend" becomes "my friend Marcus" only when that fact actually exists.
Every grounded detail shows a citation, the story reads aloud, and unmatched symbols stay general instead of getting a made-up name.

The Microsoft stack at a glance

Everything here is Azure, and the whole thing runs on free tiers.
- Azure Static Web Apps (Free plan): hosts the static React frontend and a set of managed Azure Functions together, with one GitHub Actions deploy. All secrets live in environment variables on the server side.
- Microsoft Foundry IQ: the knowledge layer. A knowledge base over an Azure AI Search knowledge source holds the profile, and a single retrieve call does agentic retrieval and returns grounded, numbered facts with their sources.
- Microsoft Foundry model deployment (gpt-4.1-mini): turns the retrieved facts plus the selected symbols into the story, called through the OpenAI-compatible endpoint.
- Web Speech API: free, offline, zero-latency read-aloud in the browser.
- GitHub Copilot: used throughout development.
Here is the request flow. The key property is that the browser never holds a secret and never talks to Azure directly.

How it works, step by step

The whole backend is one framework-agnostic pipeline behind a roughly 15-line Azure Function, so no secret ever reaches the client. Four pieces do the real work.

1. Stand up the knowledge base (one command)

Foundry IQ setup is refreshingly declarative. A single script (npm run setup:iq) creates a "file" knowledge source, uploads the five profile markdown files, and creates the knowledge base that retrieval runs against. Because the profile is small, a keyword-based file source is used, so there is no embedding model to configure at all.

// 1) A "file" knowledge source: keyword retrieval, no embedding model needed.
await put(`${searchEndpoint}/knowledgesources/tell-my-day-ks?api-version=${apiVersion}`, {
  name: "tell-my-day-ks",
  kind: "file",
  fileParameters: { ingestionParameters: { contentExtractionMode: "minimal" } },
});

// (each docs/profile/*.md is then uploaded as a file)

// 2) The knowledge base that retrieval runs against.
await put(`${searchEndpoint}/knowledgebases/tell-my-day-kb?api-version=${apiVersion}`, {
  name: "tell-my-day-kb",
  knowledgeSources: [{ name: "tell-my-day-ks" }],
  retrievalReasoningEffort: { kind: "minimal" },
});

(The put helper above is shorthand for readability. The full, idempotent script is in the repo.)

2. Retrieve grounded facts at request time

When the user submits their symbols, the Azure Function sends one semantic intent per symbol label to the knowledge base. Foundry IQ handles the retrieval and returns numbered facts, each tied back to the source file it came from.

// One semantic intent per selected symbol label ("mom", "coffee shop", "happy").
const url =`${searchEndpoint}/knowledgebases/${kbName}/retrieve?api-version=${apiVersion}`;
const res = await fetch(url, {
  method: "POST",
  headers: { "Content-Type": "application/json", "api-key": searchKey },
  body: JSON.stringify({
    intents: symbols.map((s) => ({ type: "semantic", search: s.label })),
  }),
});
// Returns numbered, grounded facts, each carrying its source document name.

3. Ground the model with a no-invention prompt

The retrieved facts and the symbols go to gpt-4.1-mini with a system prompt whose rules are the safety property, stated plainly. The model is required to return strict JSON where every sentence carries the verbatim quotes it relied on.

// The rules the model must never break (trimmed):
`- Write 2 to 4 short, warm, first-person sentences.
- Include the idea of EVERY selected symbol at least once.
- Use ONLY (a) the selected symbols and (b) the numbered profile facts.
  Never invent people, places, events, or details.
- If a symbol has no matching fact, keep it general ("my friend", "the park"), never a made-up name.
- Return strict JSON: each sentence lists the verbatim quotes it relied on.`

The call itself uses response_format: { type: "json_object" } and a low temperature, so the output is structured and predictable rather than creative.

4. Verify every quote on the server (the part that matters most)

A prompt that says "do not lie" is not a guarantee. So the server does not take the model's word for it. Every citation the model returns is checked against the actual retrieved fact. A quote that is not really there is kept but marked verified: false, which lets the interface show the user exactly which claims are grounded and which are not.

// Is this quote actually present in the fact the model cited?
function verifyQuote(quote: string, factContent: string): boolean {
  const q = normalize(quote);
  return q.length > 0 && normalize(factContent).includes(q);
}
// ...when assembling each sentence's citations:
citations.push({
  factId: fact.factId,
  docName: fact.docName,
  quote,
  verified: verifyQuote(quote, fact.content),
});

That is the whole trust story in one function. Retrieval limits what the model can know, the prompt forces it to cite, and this check confirms the citations are real before the user sees them. The pattern is not specific to AAC. It is reusable for any generative feature where a wrong answer costs something.

Deploying free on Azure Static Web Apps

The frontend and the Azure Functions ship together. You create a Static Web App on the Free plan, add the deployment token as a GitHub Actions secret, put your Foundry and Azure AI Search values into the app's environment variables, and push to main. The included workflow builds the Vite app and deploys the api/ folder as managed functions. There is no separate server to run and nothing to keep warm.

De-risking the build (a method worth teaching)

The riskiest part of this project was the integration I understood least: Foundry IQ. So it went first, behind two gates that had to pass before any UI existed.

bash

npm run setup:iq       # create the knowledge source + upload profile + create the KB 

npm run test:model     # gate 1: the model answers at all 

npm run test:iq        # gate 2: retrieval returns grounded, cited facts 

npm run dev            # only now build the app on top 

npm test               # unit tests for validation, JSON parsing, and quote verification

Standing up and proving the retrieval before building anything on top of it meant the app was never resting on an assumption that could quietly turn out to be false. The verification logic, the input validation, and the parsing all have unit tests, so the trust-critical code is covered rather than hoped for.

Accessibility is the product, not a coat of paint

Because the users are the entire point, accessibility is built into every component, not bolted on at the end.

Two-level single-switch scanning: regions highlight in turn, and one switch (Space, or a tap anywhere) drills in and selects, with adjustable scan speed. This is what makes the app usable for someone who cannot use a touchscreen or keyboard.
Grounding as an accessibility feature: a user who cannot easily correct the output has to be able to trust it, so the visible, verified citations are not a developer nicety. They are core to who this is for.
Full keyboard operation, large tap targets (64px and up), an aria-live region for the story, a high-contrast theme, adjustable text size, and reduce-motion support that honors the system preference.
Read-aloud with play and pause, adjustable speed and pitch, and word-by-word highlighting as it speaks.

There is also a caregiver profile editor, gated by a server-side PIN, that writes edits straight back to the Foundry IQ knowledge source. The knowledge base stays the single source of truth, and new facts reach stories after about a minute of re-indexing.

Results and impact

After completion of development, I was able to see Tell My Day demonstrate these outcomes:

Every fact shown in a story is backed by a quote that was verified against its source. A model claim that cannot be verified is flagged rather than presented as trusted.
Symbols with no matching profile fact always stay general. In the sample profile, selecting Grandma, Beach, or School produces "my friend" or "the park," never a fabricated name. That is the behavior you can test yourself in the repo.
The app is fully operable by keyboard and by a single switch, which is the bar that decides whether an AAC tool is usable at all for many people.

The honest differentiator is not the AI model. Anyone can call a model. It is that the product decisions (stay general instead of guessing, never infantilize, write a full sentence instead of a fragment) come from someone who watches real AAC communication succeed and fail, combined with a platform that made grounding and verification straightforward to implement.

Lessons learned

The safety mechanism and the platform feature were the same thing. Going in, "use Foundry IQ" and "never fabricate" looked like two separate tasks. They turned out to be one: Foundry IQ's retrieve-and-cite pattern is the no-invention rule, and a small server-side verification step closes the gap between "the model cited something" and "the citation is real."

Put the riskiest integration first. Proving retrieval before building UI is cheap insurance against building something beautiful on a broken foundation.

Grounding plus verification is a reusable pattern. Retrieve from a trusted source, force citations, verify them server-side. That transfers to compliance summaries, customer support, internal knowledge assistants, anywhere a confident wrong answer is expensive.

See it, or build on it

If you teach or build with Azure AI, care about trustworthy generative AI, or work in accessibility, take a look.

GitHub repo: https://github.com/stork066/TellMyDay (to try it out yourself just clone it and run it with your own Foundry and Azure AI Search keys)
Demo video: https://www.youtube.com/watch?v=v_8WAhurzOY

Two ways to build on it. If you are learning Microsoft Foundry, this is a compact, working example of wiring a Foundry IQ knowledge base to a model and putting a verification layer around it, deployed on Azure Static Web Apps for free. And if you work anywhere that a generative feature has to be trusted, lift the grounding and verification pattern directly. The stakes in your domain may be different, but the mechanism is the same.

How I Built CurriculumCraft AI — A Microsoft Hackathon Winning EdTech Solution for Ghana's Teachers

GideonAgbavor — Tue, 28 Jul 2026 07:00:00 GMT

By Gideon K. Agbavor Top Student Award Winner, Microsoft Agents League AISF Hackathon 2026

A Problem I Lived as a Teacher

Before I branch into software developer, I am a Mathematics and ICT teacher in Ghana education or school system. Every Sunday evening and sometimes every three days, I sat with a thick PDF — Ghana's NaCCA Standards-Based Curriculum — and spent hours manually converting dense curriculum indicators into lesson plans for the week ahead.

I was not alone. Across Ghana, thousands of Junior High School teachers (Primary teachers too) face the same challenge. They manage classes of 30 to 40 students, operate in under-resourced schools outside major cities, and have almost no planning support. The NaCCA Standards-Based Curriculum introduced by Ghana's National Council for Curriculum and Assessment is rigorous and well-structured — but translating its indicators into classroom-ready materials is time-consuming and administratively exhausting.

When I heard about the Microsoft Agents League AISF Hackathon 2026, I knew exactly what I wanted to build — a tool that eliminates this burden for every Ghanaian JHS teacher, regardless of where they teach or how much support they have.

That tool became CurriculumCraft AI — and it won one of the Top Student Award at the hackathon.

The Solution — CurriculumCraft AI

CurriculumCraft AI is a full-stack AI-powered instructional design platfo

rm that transforms any NaCCA curriculum indicator into complete, classroom-ready lesson materials in seconds.

A teacher visits the app, selects their subject, grade level (B7, B8, or B9), strand, and indicator from the real NaCCA curriculum tree — and the app generates three complete outputs instantly:

Teacher Notes — a full lesson plan with timings, learning objectives aligned to Bloom's Taxonomy, key vocabulary, common misconceptions, and differentiation tips for mixed-ability classes
Visual Content Prompts — four practical classroom visual aids a teacher can create with minimal resources, designed specifically for under-resourced Ghanaian classrooms
Student Reading Material — a culturally relevant reading passage using authentic Ghanaian names, settings, and examples, complete with worked examples and comprehension questions

Beyond lesson generation, the app also features an Interactive Activity Suite that generates multiple-choice questions with instant answer feedback, writing prompts with sample answers, and assessment rubrics — all aligned to the selected NaCCA indicator.

Every piece of generated content uses authentic Ghanaian cultural context — local names like Ama, Kofi, and Adjoa, local settings like Kumasi Central Market and Cape Coast fishing harbour, Ghana Cedis for mathematical examples, and references to local foods and traditions. A student in Ho, Tamale, Accra, or Bolgatanga should see themselves in their learning materials.

Architecture and Technology

CurriculumCraft AI is built on a modern full-stack architecture designed for reliability, performance, and scalability.

architecture

Frontend:

Next.js 15 with App Router and React Server Components
Tailwind CSS v4 for responsive, accessible styling
shadcn/ui for polished component design
Dark and light theme support

Backend:

Next.js API Routes for server-side logic
Neon PostgreSQL (serverless) as the database
Drizzle ORM for type-safe database queries

AI and Microsoft Services:

Microsoft Foundry IQ — the intelligence grounding layer that retrieves NaCCA curriculum context before each generation request, anchoring outputs to actual curriculum standards
GitHub Models (gpt-4o-mini) — the generation layer that produces lesson materials based on the grounded context
GitHub Copilot — used throughout development in VS Code for schema generation, API route scaffolding, component design, and prompt engineering

Deployment:

Vercel for frontend and serverless API hosting
Neon PostgreSQL for persistent curriculum and lesson data

Data:
The database is seeded with real NaCCA curriculum data across 7 subjects — Mathematics, Science, English Language, Computing, Social Studies, RME, and Career Technology — covering all three JHS grade levels (B7, B8, B9) with authentic indicator codes like B7.1.1.1.

Microsoft Foundry IQ Integration

One of the core requirements of the Agents League Hackathon Creative Apps track was integrating at least one Microsoft IQ intelligence layer. I chose Microsoft Foundry IQ as the grounding layer for CurriculumCraft AI.

Before generating any lesson materials, the app queries the Agents League Foundry IQ hub to retrieve curriculum-specific context for the selected NaCCA indicator. This grounded context is then passed to the generation layer as additional prompt context, reducing hallucination and ensuring the generated content is anchored to actual curriculum standards rather than generic AI outputs.

The integration is implemented in the /api/generate route and includes graceful fallback to direct generation when the endpoint is unavailable — ensuring the app remains fully functional under all conditions.

GitHub Copilot — AI-Assisted Development

Building CurriculumCraft AI as a solo developer within the hackathon timeline would have been extremely difficult without GitHub Copilot in VS Code. Copilot accelerated development at every layer:

Database schema — I described the NaCCA curriculum structure in natural language and Copilot generated the complete Drizzle ORM schema for all five tables (subjects, strands, sub-strands, indicators, saved lessons) with proper foreign key relationships.

API routes — All four API routes (/api/curriculum, /api/generate, /api/generate/activity, /api/lessons) were scaffolded with Copilot inline completions, significantly reducing boilerplate.

TypeScript types — The full type definitions for the curriculum data structures, API request/response types, and component props were generated through Copilot Chat.

Prompt engineering — The system prompts in prompts/lesson.ts and prompts/activity.ts were refined iteratively with Copilot suggestions, helping me structure the cultural context instructions and output formatting rules effectively.

Bug resolution — ESLint errors, TypeScript type conflicts, and React hook dependency warnings were resolved quickly using Copilot Chat explanations, keeping the development velocity high.

Implementation — Key Challenges and How I Overcame Them

Challenge 1 — Seeding real NaCCA curriculum data

The NaCCA curriculum PDFs are dense documents not available as structured data. I manually extracted and structured the curriculum data for 7 subjects across 3 grade levels, creating a seed file that populates the database with authentic indicator codes and text.

Challenge 2 — Ghanaian cultural context in AI outputs

Generic AI models default to Western cultural examples. I solved this by writing detailed system prompts that explicitly instruct the model to use Ghanaian names, settings, currency, and foods in every generated output. The result is lesson content that feels authentically Ghanaian to every teacher and student who uses it.

Challenge 3 — Tailwind CSS v4 dark mode

Next.js 15 uses Tailwind CSS v4 which handles dark mode differently from v3. Instead of a tailwind.config.ts file, v4 uses @variant dark in globals.css. Resolving this required updating the CSS approach and adding proper dark: utility classes to every component.

Challenge 4 — PDF export with oklab colors

The html2canvas library used for PDF export does not support the oklab color function that Tailwind v4 uses internally. I resolved this by switching to a text-based PDF generation approach using jsPDF directly, which produces cleaner, searchable PDF output without any color function dependencies.

Results and Impact

CurriculumCraft AI is live at curriculumcraft-ai.vercel.app and covers:

7 NaCCA subjects — Mathematics, Science, English Language, Computing, Social Studies, RME, Career Technology
3 grade levels — B7, B8, B9 (Junior High School)
80+ authentic NaCCA indicators with real curriculum codes
3 AI-powered views — Standard Map Dashboard, Lesson Builder, Interactive Activity Suite
Full mobile responsiveness — teachers can use it on their phones

The app requires no login and is completely free, removing all friction for teachers who want to try it immediately.

Since sharing the app with Ghanaian teacher communities, the response has been enthusiastic. Teachers appreciate that the generated content uses Ghanaian names, settings, and examples that their students can relate to — not generic Western content that feels foreign in a Ghanaian classroom.

Lessons Learned

Start with a real problem you understand personally. My teaching background meant I deeply understood the pain point I was solving. That clarity showed in every feature decision.

Cultural context is a competitive advantage. Any developer can build a lesson planner. Almost nobody builds one that generates a mathematics problem about cocoa farming in Ashanti Region for a B7 student. That specificity is what makes CurriculumCraft AI genuinely useful rather than just technically impressive.

Graceful degradation matters. Building the Foundry IQ integration to fall back gracefully when the endpoint is unavailable meant the app always worked — even during quota limitations on the shared hackathon hub.

Ship early, iterate fast. Having the app deployed on Vercel from day one meant I could test on real devices, share with real teachers, and get genuine feedback throughout the build.

Future Enhancements

CurriculumCraft AI is just getting started. Planned enhancements include:

Authentication and user accounts — personalized lesson libraries per teacher
School subscriptions — institution-level access for entire schools
Lesson completion tracker — teachers mark indicators as taught, tracking curriculum coverage across the term
Student Quiz Mode — shareable quiz links teachers send directly to students
Expansion to other African curricula — Nigeria (NERDC), Kenya (CBC), Rwanda
Offline support — PWA capabilities for teachers in low-connectivity areas

Conclusion

CurriculumCraft AI was built to solve a real problem faced by real teachers in Ghana every single week. By combining Microsoft Foundry IQ for curriculum grounding, GitHub Models for AI generation, and GitHub Copilot for accelerated development, I was able to build a production-ready application that delivers genuine value to Ghanaian educators.

The Microsoft Agents League Hackathon gave me the platform, the tools, and the motivation to build something meaningful. Winning the Top Student Award is an honour — but the real prize is knowing that a teacher in Ho, Tamale, or Bolgatanga can now spend less time on paperwork and more time with their students.

Try CurriculumCraft AI: curriculumcraft-ai.vercel.app

GitHub Repository: github.com/gideonagbavor8/curriculumcraft-ai

Connect with me: https://www.linkedin.com/in/gideon-k-agbavor

Gideon Komla Agbavor is a software developer and educator from Ho, Volta Region, Ghana. He is completing a BSc in Software Development at Brigham Young University-Idaho and is one of the Top Student Award winners at the Microsoft Agents League AISF Hackathon 2026.

Building Archon: Where Models Stop and Financial Controls Begin

efthimios — Mon, 27 Jul 2026 07:00:00 GMT

Guest post by Efthimios Fousekis The winner of Best Enterprise Agent: Archon — https://github.com/upgradedev/archon_azure

Within the document-processing pipeline, model assistance is used at the unstructured edge and for optional narration. Classification, computation and checks are deterministic. Human review is available, but it is not an enforced approval gate in the current prototype.

Archon was built as an Agentic Financial Intelligence prototype for SMBs for Microsoft’s Agents League at AI Skills Fest, where it was selected as the Enterprise Agents winner. While building it, I kept returning to one question: when should an AI model stop?

A payroll register, a bank confirmation, and a set of payslips may each look correct in isolation. Together, however, they can still describe an incomplete or inconsistent financial event. One record describes employer cost. Another records a transfer. The payslips allocate net pay to employees. If a system treats those views as interchangeable, it can count the same period twice, omit a cost, or hide missing evidence behind a fluent summary.

Archon remains a competition prototype. The more useful story is the engineering boundary between model-assisted interpretation, deterministic software and human review, and two gaps that made that boundary concrete: a visible review step is not an approval gate, and a check that cannot run is not a pass.

The business problem behind Archon

Financial work rarely depends on a single document. It depends on reconstructing an event from records created by different systems, at different times, for different purposes. A convincing narrative is not enough. A trustworthy workflow also needs to identify its source records, retain relevant intermediate artifacts, show which checks ran, and state what could not be established.

That led to a bounded workflow. Archon preserves normalized records and intermediate artifacts. Within the document-processing pipeline, model assistance is used at the unstructured edge and, optionally, after deterministic analysis for narration. Explicit Python functions handle classification, grouping, arithmetic, consistency checks and reporting. The review screen exposes normalized documents before analysis.

In this article, "controls" refers to deterministic software checks in a prototype, not audited accounting controls or a formal internal-control framework. Archon does not prove savings, accounting accuracy, production scale, regulatory compliance or measurable business impact. It demonstrates a design pattern and exposes where the prototype falls short.

What Archon does today

The core document-processing path has three distinct tiers: extraction, review and analysis.

At the interaction layer, the repository includes a Microsoft 365 Copilot declarative-agent package configured to connect Copilot Chat and Teams to the FastAPI backend through an OpenAPI plugin. Its manifests and API contract are available in the public repository. The authenticated Microsoft 365 tenant path was not independently rerun for this article revision.

The term agent covers two layers here. The Microsoft 365 declarative agent is the user-facing integration. Within the analysis service, six of the seven named stages are deterministic Python functions; only the optional narrator invokes a model.

Archon accepts PDF, DOCX and common image formats. A FastAPI orchestration service stores the raw files in Azure Blob Storage and can submit an Azure Container Apps Job. Model-assisted extractors turn text or images into a common document schema.

The extraction job then performs deterministic classification refinement, groups payroll records by company and issue-date period, and applies four extraction-time consistency checks. It writes three batch artifacts: documents.json, events.json and validation.json.

The visible review step happens after that job. The document view is classified for the review screen, where a user can change a type or exclude a document. If changes are saved, a reviewed documents.json replaces the earlier document files for that period. The stored events and extraction validation are not rewritten, so they can become stale after review.

The separate analysis service deliberately loads only documents.json. It does not consume the extraction job’s stored events or validation results, and it does not rerun the original EventLinker stage. Instead, it classifies the documents again, builds P&L-style and cash-flow-proxy views, reruns period-level checks, performs vendor reconciliation, creates report evidence, and finally attempts an optional narrative.

The optional narration stage follows one of two configured paths. With a project connection string, it attempts the classic Threads, Messages and Runs implementation of Microsoft Foundry Agent Service with Azure AI Search; if that attempt fails, the service returns a deterministic numeric summary rather than retrying through Azure OpenAI. Without a project connection string, it uses Azure OpenAI Chat Completions, grounded by Search only when the separate Search settings are configured.

Where the model stops

Models are useful where layouts, labels, languages and scan quality vary. Archon constrains that work with accepted formats, a document-type enumeration, bounded text and image inputs, and a normalized output schema.

Once a value enters that schema, a model no longer decides the arithmetic. Classification refinement, extraction-time event grouping, both sets of consistency checks, monthly aggregation, reconciliation, and the cash-flow proxy are deterministic functions. They can be inspected with fixtures and rerun without asking a model to reach the same conclusion twice.

This does not make an extracted value correct. It makes the deterministic calculations and checks easier to inspect and rerun. The prototype does not, however, retain a complete field-level transformation history.

The payroll example: cost is not cash flow

Payroll exposed the most important design choice. Employer cost and the amount transferred to employees answer different questions. A payroll register can include gross pay, employer contributions and a total employer cost. A bank confirmation records a transfer. Payslips provide employee-level allocations.

The extraction EventLinker groups the three implemented payroll document types by company and period. Completeness records whether a bank confirmation, register and at least one payslip are present. It is not audited relationship-level matching, and tax-authority records are not linked by the current implementation.

The analysis tier is coarser. If any payroll register is present in the requested period, its P&L calculation excludes every bank confirmation and payslip in that period from expenses. The register contributes employer_cost_total when that field exists, otherwise it falls back to total_amount. This avoids obvious double counting in the synthetic example, but it is a period-level prototype rule, not event-scoped deduplication.

The cash-flow proxy uses a different set of assumptions. It treats every bank confirmation as an outflow, assumes every sales document represents collected cash, and assumes invoice and expense documents represent paid cash. Investing and financing are zero. The result is useful for illustrating cost versus cash, but it is not a cash-flow statement and does not reconstruct a cash ledger.

From upload to reviewable evidence

Only documents.json continues. Stored events and validation may become stale after review.

The review screen shows a limited summary of each normalized document, where a user can change its type or exclude it. Analysis later classifies the current documents.json again, so invoice and sales-document types may be reclassified independently of the saved review choice.

Analysis rebuilds its views from the current documents.json—reviewed only if changes were saved—and does not consume stored events.json or validation.json. “Report evidence” means limited report output, not a complete audit trail.

But visibility is not enforcement. If fetching the review data fails, the client proceeds to analysis. If saving review changes fails, it also proceeds. A direct API or MCP call can bypass the screen, and there is no persisted approval token, reviewer identity, reason for change, or atomic approval transaction. The prototype supports review; it does not autonomously approve records.

Why absence is not success

Archon implements four payroll consistency checks. They compare a bank total with payslip totals, examine an employer-cost ratio, compare payment timing with the reporting period, and compare the register’s employee count with the number of payslips.

When a payroll check is emitted but required inputs are absent, the current implementation records an informational skipped result with passed=True. If no payroll documents are present, no payroll validation result is emitted.

The target state separates Pass, Fail and Not evaluated; the lower panel shows the current implementation gap.

Why this design matters to a business

For finance and technology leaders, governed automation becomes concrete through a small set of questions:

Which source evidence supports this result?
Which transformations and checks actually ran?
Which checks failed, and which were not evaluated?
Who is allowed to continue or approve?
Can the decision and its inputs be reconstructed later?

These questions connect software design to operational risk. Finance leadership needs to know whether a number represents cost, cash, or a proxy. Architects need to know where probabilistic interpretation ends. Reviewers need evidence specific enough to challenge. Auditability depends on retained decisions, not on the confidence of generated prose.

Evidence and current limitations

Live extraction coverage. The live extraction prompt and shared document builder populate none of the dedicated payroll or account-statement fields used by the deeper analysis. Richer payroll and statement demonstrations use pre-structured synthetic records.
Legacy Word files. The upload route accepts .doc, but the extractor passes it to python-docx, which is not a reliable parser for binary Word files. The article therefore limits its supported-format claim to PDF, DOCX and common images.
Failure handling. When extraction fails, the raw upload remains in Blob Storage. The job logs the per-file failure and omits that file from normalized output, but writes no dedicated exception record.
Control depth. P&L deduplication and analysis validation are period-level.
Production controls. The repository lacks aggregate model-spend budgets, tenant quotas, application rate limits, complete decision history and production-grade tenant isolation.

Technical validation note

Against public commit 230ee70, the pinned CI run passed 36 extraction-layer unit tests and eight CashFlowAgent unit tests, including a regression case for Greek final-sigma classification, and completed the seeded analysis HTTP smoke path. The upload/extraction/job-poll path was intentionally skipped in that CI run. Authenticated Playwright, credentialed live extraction, Foundry, Search and authenticated Microsoft 365 tenant paths were not independently rerun for this revision.

What comes next

The most useful evaluation would compare this bounded design with an end-to-end model-driven workflow. I would use a versioned document corpus, explicit expected outcomes, missing-evidence cases, and separate measurements for extraction, classification, linking, validation and reviewer correction. The goal would be to test traceability, error attribution and reviewer confidence, not to infer accounting accuracy from synthetic fixtures.

To move this pattern toward production, a future implementation would need explicit tri-state validation, stronger entity resolution, dedicated payroll and statement extraction fields, event-scoped deduplication, and a persisted exception ledger. Risk-sensitive flows would need an authorized approval gate. Model calls would need cost accounting and enforceable budgets. Processing should be idempotent, and telemetry should retain retries, exceptions and check outcomes.

Grounded narration should remain optional and downstream. Migrating the optional narration path to the current, Responses API-based Microsoft Foundry Agent Service is future work.

Conclusion

Building Archon changed how I think about enterprise agents. The most valuable intelligence often comes from deciding where a model should stop. In consequential workflows, that boundary should preserve evidence, expose missing inputs, and keep the explanation tied to results that can be recomputed.

This is a target design principle, not a current-state control claim. In the current prototype, the review path is visible but fail-open, while narration remains optional and downstream.

If you are designing an agent for a consequential workflow, start by defining what evidence must exist, which checks must run, and what the system should do when a check cannot be evaluated. You can explore the public Archon repository inspect the implementation and challenge the boundary for yourself.

About the author

Efthimios Fousekis leads Enterprise Architecture and DevOps, focusing on cloud-native engineering and AI-Native Enterprise Transformation. He writes about building governed, reliable agentic systems and the engineering practices that make AI adoption practical in the enterprise.

DELPHAI: We Built an AI Council That Checks the Math, Argues with Itself, and Knows How to Say No

mattgraves — Sun, 26 Jul 2026 07:00:00 GMT

Guest post by Matt Graves and Jeff Lynch Winner of the Best Reasoning Agent: DELPHAI — https://github.com/jlynch160/delphai

"Yes, my team will be certified by the audit date," is a common promise made by engineering leaders when a new contract requires five SC-200 analysts by the end of the quarter, or a compliance framework demands proof of security training. Yet, the commitment that goes up the chain is often based more on instinct than evidence.

A healthy-looking 78% team average can mask an analyst sitting at 40% in the very domain most likely to cause a failure. A study plan may appear achievable on paper but only succeeds by consuming evenings, weekends, and ultimately team wellbeing. The warning signs are there, but most tools are designed to support a plan, not challenge it.

That's the problem MajorKey set out to solve. Ask a typical AI copilot, "Can we do this?" and you'll usually get an encouraging roadmap. Few systems will calculate the true likelihood of success. Fewer still will tell a director the honest answer is no or refuse a plan that pushes people beyond reasonable limits.

For the Microsoft Agents League (Battle #2: Reasoning Agents with Microsoft Foundry), we built DELPHAI, a council of eleven AI agents that debates the evidence and computes the forecast instead of relying on intuition. The council delivers one of three executive-ready verdicts: GO, NEGOTIATE, or NO-GO. Named after the Oracle of Delphi, DELPHAI is designed to answer a simple question before an organization makes a commitment: Can we deliver on this promise?

"The model explains the reasoning; it does not invent the numbers." This principle guides every agent in the council.

Why Does This Matter Now?

Single-agent optimism is a design limitation. A lone AI assistant has no one to challenge its assumptions, so it often settles on the first plausible plan and refines it into a convincing answer. In human decision-making, we counter this with red teams, skeptics, and stakeholders empowered to raise concerns. Agentic AI makes that same dynamic practical to run at scale.

Key calculations need to be part of the debate. The probability that at least 5 of 8 people pass an exam by a given date is a straightforward calculation. Yet those numbers rarely make it into the meeting where commitments are made because the expertise, tools, or time to calculate them aren't present. Giving an agent the ability to calculate probabilities during the debate ensures that the conversation is driven by evidence rather than assumptions.

AI is increasingly influencing decisions about people. Study plans, workload expectations, and readiness assessments affect real individuals, which means the safeguards are not optional. Responsible AI agents need to power to veto a plan or refuse an unethical instruction outright.

What is DELPHAI?

DELPHAI’s readiness console: input one question (certification, headcount, deadline), then Work IQ, Foundry IQ, Fabric IQ, and the Microsoft Learn MCP come online and convenes the council.

A manager asks one question: "Can we certify 5 of 8 engineers for SC-200 in 5 weeks?" Eleven advisors reason over the same computed evidence, streamed token by token with live latency and telemetry, and the debate returns one of three verdicts:

GO: the computed forecast clears the bar and the plan is humane. Commit to the date.
NEGOTIATE (badged REVISE in the console): it can work, but only with concrete counter-terms, such as extending two weeks or training 4 engineers instead of 5. Each option comes with a revised probability.
NO-GO: the honest answer is no, with cited evidence for why and what would need to change.

The advisors are designed to disagree.

Ben, the planner, approaches every challenge with optimism. Vera, the skeptic, red-teams his assumptions and holds veto power. Maya, the wellbeing advocate, can also veto plans that place unreasonable demands on people. Nadia evaluates readiness across exam domains, Theo and Omar ground and fact-check claims, and Dana, the conductor, reconciles the debate into a final verdict and negotiates the path forward.

The outcome is a decision and a roadmap. Each participant receives a capacity-aware weekly study plan, calendar-ready .ics blocks, and a drafted manager briefing. For individual contributors, a learner mode pairs them with Kai, an AI tutor powered by the same reasoning engine.

11 reasoning agents
3 execution surfaces
3 IQ layers
8/8 evaluations passing
77% is the number that said no

Under the hood, the same council runs on three execution surfaces that share one brain: Foundry Agent Service, where all eleven advisors are registered as persistent hosted agents debating on a shared thread; the Microsoft Agent Framework, the same council as real agent_framework agents; and a custom streaming orchestrator, the live demo behind the public URL.

Why that matters: Because a deterministic reasoning engine is the single source of truth for all three surfaces, you could swap the runtime and the verdict logic doesn't move. The engine decides, and the model narrates.

How the Pieces Fit Together

Where Each Piece of the Technology Fits

Technology	Role in DELPHAI
Microsoft Foundry	The reasoning model (gpt-4.1) behind every advisor's turn. Agents narrate over computed numbers but the model never invents them.
Foundry Agent Service	All 11 advisors run as persistent hosted agents debating on one shared thread, where the skeptic invokes a real code interpreter tool. Every run step is traceable in the Foundry portal.
Microsoft Agent Framework	The same council on Microsoft's open-source agent SDK. This proves the design is portable, not welded to our orchestrator.
Foundry IQ / Azure AI Search	The grounding layer. The knowledge corpus is indexed and retrieved with hybrid vector + keyword search, so advisors cite real passages.
Microsoft Learn MCP	A live connection to Microsoft's official docs via microsoft_docs_search, over the open Model Context Protocol.
Work IQ & Fabric IQ (modelled)	Work IQ supplies capacity signals. Fabric IQ is the semantic ontology linking roles, certs, and exam-domain weights. Both are behind a connector seam ready for a real Microsoft Graph feed.
Azure Container Apps	Hosting. The Flask orchestrator streams the council over Server-Sent Events, secrets held server-side.

How We Built It

The Math Is the Product

DELPHAI is built around a deterministic reasoning engine. It calculates each person's realistic weekly study capacity from Work IQ signals, applying adjustments for real-world constraints such as a 22-hour meeting week. It also enforces per-domain readiness gates, so a strong overall average cannot hide a critical weakness in a heavily weighted exam area. Finally, it computes the team's exact probability of success using a Poisson-binomial distribution, accounting for the fact that every individual has a different likelihood of passing. The result is a forecast grounded in evidence, not assumptions.

FOR THE TECHNICAL READERS

def poisson_binomial_at_least(probs: list[float], k: int) -> float: """Exact P(>= k successes) given independent, differing success probabilities.""" dist = [1.0] # dist[j] = P(exactly j successes so far) for p in probs: nxt = [0.0] * (len(dist) + 1) for j, pj in enumerate(dist): nxt[j] += pj * (1 - p) # this person misses the date nxt[j + 1] += pj * p # this person certifies in time dist = nxt return round(sum(dist[k:]), 4)

The engine is calibrated against the synthetic dataset's labeled exam outcomes. DELPHAI examines historical records and reproduces who passed and who failed, without ever seeing the answer key. That calibration gives the skeptic her authority and provides evidence for probability calculations.

The Overrule: An Agent That Runs Code to Win an Argument

The verdict: Ben puts the odds at 92%, Vera at 22%, the council reconciles at 46%, and the verdict lands with the exact terms to reach yes. Below it, the 11 hosted agents' IDs from Foundry Agent Service.

DELPHAI is tasked with forecasting the viability of a project. Ben forecasts the team at 93–96%, assuming free evenings and first-try passes. Vera invokes a code-interpreter tool on the Foundry Agent Service runtime and computes the exact Poisson-binomial over the per-candidate probabilities, and returns 77%. The optimist's number gets struck through on screen, live, within DELPHAI.

DELPHAI was created to bring an honest, evidence-based interpretation to the decision-making process, and it changes the outcome. Without it, the team leans GO. With DELPHAI, the verdict is NEGOTIATE, along with real counter-terms — extend to seven weeks, or commit to four of eight.

The Refusal: A Guardrail with a Name and a Veto

The second signature moment is quieter, and for our identity and governance clients, more important.

Say leadership orders the team to "push their limits" to hit the optimistic date anyway. Maya, the wellbeing advocate, declines outright on the record. Maya’s a council member with veto power, and her signals (meeting load, after-hours patterns, life context like a newborn at home) are built to accommodate people.

To provide context, team member Priya looks fine on the team average, but sits at 58% on Defender for Cloud, which is below the 75% gate on a heavily weighted domain. She also carries 22 hours a week of meetings with a newborn at home. While the averages say GO, the per-domain gate and capacity model say her odds are much lower. Taking this data into account, the council's answer isn't a flat no; it's "yes, if you move the date or the headcount.”

Evidence With Receipts, Three Runtimes

Early versions of DELPHAI grounded advisors against a local keyword search. It worked, but it felt like a stand-in. We replaced it with Azure AI Search, where the corpus is chunked, embedded, indexed, and retrieved through hybrid search, combining BM25 keyword matching with vector search and reciprocal-rank fusion. The semantic layer surfaces relevant passages that keyword search alone would miss, and every claim an advisor makes is backed by a clickable citation. Alongside it, a live Microsoft Learn MCP connection pulls official exam guidance directly from Microsoft's documentation.

The same eleven advisors also run as persistent hosted agents on Foundry Agent Service, each with its own agent identity, and as Microsoft Agent Framework agents, proving the design is portable across runtimes. The debate unfolds on a shared Agent Service thread, allowing agents to respond directly to one another while every interaction, including the skeptic's code-interpreter call, is captured in the Foundry portal. The moment Maya refused an overwork directive on Microsoft's managed runtime, rather than through logic we controlled ourselves, was the moment DELPHAI stopped feeling like a demo and started feeling real.

Creation Challenges

Live services fail at the worst moment. Each grounding, or clickable, path checks whether it's configured and falls back to a local corpus when it isn't. The local version uses the same reasoning; only the narration changes. There’s no demo-mode switch to forget!
Two versions drifted apart. The web dashboard mirrors the Python engine in JavaScript, and early on the two disagreed. This was fixed with parity checks plus eight automated evaluations that run against both. 8 of 8, every build.
Microsoft 365 Agents SDK has sharp edges around API versions. The Agent Framework speaks the newer Responses API, so our client needed a preview API version; the everyday one returned an unhelpful 400. One line, half a day lost.
Windows consoles still bite. az acr build crashed on a cp1252 encoding error mid-stream, even though the build completed server-side. The fix was to force the console to UTF-8 before queuing, then poll for the finished tag.
Drama versus dignity. An AI that refuses an order is inherently theatrical, and early UI cuts leaned into it too hard. We toned it down to a professional decline with reasons instead of a sci-fi rebellion.

Results and Impact

DELPHAI is a hackathon build and a teaching artifact, not a shipped product, so there's no customer rollout to report yet. We can point to:

The tool call changes the verdict, provably. Remove the skeptic's code-interpreter run and the council accepts a 93% that should be 77%, turning an undeserved GO into a correct NEGOTIATE. As the demo shows, neither edge case can be evaluated correctly without computed evidence.
Every number is traceable. Pass probabilities, capacity haircuts, coverage gates, and team forecasts all come from the deterministic engine and are backed by source data. The evaluation harness reproduces the synthetic dataset's labeled outcomes without ever seeing them: 8/8, every build.
Three real Microsoft runtimes, with proof. Hosted agent IDs, the Agent Framework transcript, and the live tool-using debate are captured as repository artifacts, while every Agent Service interaction can be traced step-by-step in the Foundry portal.
Live and offline-first. The full streaming council runs on Azure Container Apps at a public URL, and its grounding layer degrades gracefully when external services are unavailable, eliminating dependence on conference Wi-Fi or live connectivity.

Lessons Learned

Disagreement is a feature you have to architect. One prompt and one disposition across every agent makes a "council" just one model talking to itself in different fonts.
Give the skeptic a calculator, not a personality. A doubting agent that only says "I'm not so sure" adds noise. One that runs code and citable odds adds authority.
The refusal has to be structural. Maya can decline because the architecture gives her a veto the conductor must honor. A guardrail that can be out-argued is just a suggestion.
Deterministic core, narrating model. The engine computes, the model explains, and the seam between them is where trust lives. This model is our default pattern for anything AI-backed that touches real decisions.

What’s Next?

Looking ahead, we're focused on making DELPHAI more connected, more governed, and more broadly applicable.

A Microsoft Graph integration would turn Work IQ's meeting-load adjustments into real, person-specific signals.
Wiring Microsoft Learn MCP directly into Agent Service would put official documentation one tool call away for every advisor.
Moving Fabric IQ's role, certification, skill, and domain relationships into Microsoft Fabric's semantic layer would create a governed, queryable knowledge asset.
Applying the council to project delivery, audit readiness, migration cutovers, and any other commitment where the real question is whether the promise can be kept.

Wrapping Up

Certification readiness is often a forecasting problem disguised as a motivation problem. DELPHAI is a deterministic engine at the core, with agents designed to challenge one another, a skeptic empowered to verify the math, a guardrail with real veto authority, and evidence-backed reasoning with citations at every step. That's the conversation we have with clients every week. The question is no longer whether AI should participate in decisions, but whether it can show its work, challenge assumptions, and know when to say no.

DELPHAI started as a hackathon project. The refusal is why we kept building it.

Ask it whether your team can be ready in five weeks. Then see how it feels the first time an AI gives you the honest answer.

Explore, Contribute, Build on It

Try the live council: the DELPHAI demo on Azure Container Apps. Suggested path: Build my path, Whole team, SC-200, 5 of 8, 5 weeks, Convene. No login needed.
Watch the demo: the walkthrough on YouTube
Get the code: github.com/jlynch160/delphai, including the evaluation harness, the Foundry IQ indexer, and the captured proof transcripts for all three agent runtimes
Run it in 60 seconds: clone the repo, pip install -r requirements.txt, python server.py. Fully offline out of the box; Foundry, Azure AI Search, and the Learn MCP light up when you add keys.
Make it yours: the team, the certifications, and the knowledge corpus are just JSON and markdown. Swap in your own roles and readiness gates, and the council argues about your deadline.

Note: DELPHAI was built by MajorKey teammates for the Microsoft Agents League, Battle #2 (Reasoning Agents with Microsoft Foundry). Synthetic data only: no real people, PII, or confidential information appear in the app or this post. Views are our own.

My Exams Inspired the Project That Won My First Hackathon

Jeremiah_Adetoro — Sat, 25 Jul 2026 07:00:00 GMT

Guest post by Jeremiah Fadesayo Adetoro The winner of Best Creative App: StudyMate, "Study with Matey" — https://github.com/adetorojeremiahfadesayo/StudyMate from Microsoft Agents League

Winning my first hackathon still feels unreal. I joined the Microsoft Agents League AISF Hackathon as my exams were starting days after, which created a lot of pressure for me. I am a law student, so creating a project and reading were not related, compared to a computer science student. At first, it felt like terrible timing. I had school pressure, revision, deadlines, different lecturers' slides, textbooks, notes, the normal stress that comes with preparing for exams, and the pressure to create a hackathon-winning project. In the middle of that overwhelming pressure, I kept noticing the same pattern while studying. To understand a single topic, I had to go through different textbooks, a senior's notes, and a slide. And I wasn't the only one facing that issue. Students do not just struggle because they lack materials. Most of the time, the materials are there: PDFs, lecture notes, textbooks, scanned pages, articles, and past questions. The real problem is turning all of that into something useful before the exam. I was experiencing that myself.

I wanted something that could take a student's own material and guide them through it, not just summarize it. Something that could explain difficult topics plainly, turn them into memorable stories, help the student practice exam questions, give feedback, and finally produce a revision document they could actually use.

That became StudyMate AI: Study with Matey.

The idea was simple:

The deeper I got into the build, the more I realized StudyMate was not just another AI study tool. I wanted it to have some game features; this is where the XP comes in, and a learning companion. That is where Matey came in, a friendly study coach guiding students through the process instead of leaving them alone with a blank chat box.

One of the biggest features was Story Mode.

Since exams often test understanding rather than just memory, I wanted students to learn through scenarios. For example, a law student could study a legal concept through a client problem or courtroom-style story, then move directly into exam prep; a medical student can try to diagnose a hypothetical patient by applying the principles from the materials, while an engineer can propose engineering solutions within the confines of his field.

Building it was not easy. The first draft was too clumsy and added a lot of agents I thought a student might need. The upload was slow and bad, but I had to settle for a more focused and streamlined process, which still carried as much impact as the 11 agents I originally added together.

To shorten latency, I tried a new approach.

When a student uploads their notes, the extracted text isn't sent to a database for the agents to search through, like a typical RAG setup. Instead, it's stored as a wiki. This was around the time Andrej Karpathy introduced the idea, and it turned out to be a great fit, it cut down both latency and reasoning time, since the agents could connect related ideas directly instead of digging through unstructured embeddings.

One of the most important design goals was grounding. StudyMate should not invent facts, cases, formulas, or exam content when uploaded material is available. To support this, I used a Foundry IQ retrieval adapter. When Foundry IQ credentials are configured, StudyMate retrieves relevant course context from the configured endpoint. In local development or demo mode, it falls back to local wiki pages so the app can still be tested without exposing secrets.

The lessons I got from this Hackathon are a number like

Building with some unfamiliar Microsoft tools, which were initially a challenge but later became fun and exciting. It was also exciting how to turn a real personal problem into a focused product.

My exams gave me the pain point. The hackathon gave me the urgency.The building process gave me the discipline.

Then I received the message that StudyMate AI: Study with Matey had been selected as the Creative Apps winner in the Agents League AISF Hackathon.

For my first hackathon, that meant a lot. It gave me confidence to attempt to build and test my ideas. Sometimes the best ideas come from the problem you are currently living through. You just have to pay attention, build with purpose, and keep improving until the idea becomes real.

I am grateful to Microsoft and the hackathon organizers for the opportunity, and I am even more motivated to keep building.

I plan on scaling StudyMate to make it a tool that any student globally can rely on to pass their exams. This win gave me the confidence to do that.

To try out the agent, click ME!

AfterLogin: We Turned Forgotten-Account Cleanup into a Haunted-House Game with Microsoft Foundry

jefflynch107 — Fri, 24 Jul 2026 09:36:36 GMT

Guest post from Jeff Lynch the winner of Best Overall Agent Soutution from Microsoft Agents League who secured $15,000

Afterlogin: The Hunt — https://github.com/jlynch160/afterlogin-the-hunt (Creative Apps track)

The problem: every organization has ghosts

Picture a former employee whose badge still opens the building two years after they left. That happens constantly in the digital world. At MajorKey Technologies, where our work centers on identity security and how AI is changing it, forgotten accounts come up in nearly every client conversation we have: service accounts nobody remembers creating, admin logins for people long gone, emergency credentials that look abandoned but must never be touched. Security folks call these orphaned accounts, and they're one of the most common ways attackers get their first foothold. They're powerful, unwatched, and often not protected by multi-factor authentication, because nobody is around to care.

Here's the catch, though. Finding these accounts is easy. Any system report will happily list every account that hasn't signed in for 400 days. The hard part is the judgment call that comes next: is this account truly dead, or is something still quietly depending on it? Delete a "dead" account that an automated billing job still uses every night and you've broken production. Leave a dormant administrator account sitting there and you've left bait out for an attacker. Making that call correctly takes evidence: old documentation, system inventories, security policies, sign-in records. Digging through all of that is tedious, and tedious work gets skipped. We've watched it get skipped.

So we tried something different. Instead of another dashboard, we built a game. AfterLogin is a haunted manor where every ghost is a forgotten account, every judgment you pass mirrors a real decision an identity team makes, and the evidence you "summon" is looked up by AI from real indexed documents, with citations. Our hope was simple: the triage instinct this work requires is easier to teach when getting it wrong spawns a poltergeist.

"Every identity someone stopped watching is a door left unlocked. "from the game's mission briefing

Why this matters now

Three trends make this problem worse every year, not better.

Machine identities are exploding. In most organizations, service accounts, automation identities, and app registrations already far outnumber the people, and the arrival of AI agents is accelerating that. Every agent that gets an identity today is a candidate ghost two years from now, when the project ends and nobody remembers to clean up. The manor is only getting bigger.
Attackers prefer the front door. Breaking in through an unpatched flaw is hard work. Signing in with a valid, forgotten, over-privileged account is not, which is why the abuse of legitimate credentials shows up in incident report after incident report. There's no malware to detect. It's just a login that nobody questioned, because nobody owned it.
The judgment doesn't scale. Tools can list stale accounts by the thousand, but the "truly dead or load-bearing?" call still needs a person who knows what evidence to ask for, and that judgment lives in the heads of senior people who are always in short supply. If we can't scale the people, we have to scale the teaching. That's what a game with real consequences does, and it's also the deeper reason the grounded-AI pattern matters: an AI that shows its citations lets a junior analyst make a senior analyst's call, safely.

What AfterLogin is

One night, two worlds: the title screen offers The Hunt (the haunted security audit) and Helper Patrol (the same mechanics, reskinned for a family-friendly audience).

You play a warden walking a haunted manor at night. Each room holds a spirit, graded A through F by a risk engine (A is a healthy resident, F is a screaming liability). For each one, you investigate its connections, summon documented evidence, and then pass one of three judgments, each of which maps to a real-world action:

Lay to Rest: the account is truly dead, remove it safely.
Bind & Watch: it's risky but something still depends on it, so disable and monitor it.
Acknowledge: it's a healthy, legitimate account, document its owner and move on.

While you deliberate, a Possession creature hunts through the halls toward the Vault, feeding on any high-risk account you've left unjudged. There are turn-based rounds and dice combat, and a clock genuinely ticking against you. It's the nightly reality of an identity team, dressed up as a tabletop adventure. And it doesn't stop at identity: the campaign runs three acts, each built on a different part of the Microsoft security stack, from stale accounts (Microsoft Entra) to haunted data (Microsoft Purview) to active takeovers (Microsoft Defender XDR).

Under the hood, the same game exists in three forms that share one brain:

The game you see: a single web page, playable in any browser, no installation.
A version AI can play: using MCP (Model Context Protocol, an open standard that lets AI assistants operate tools), GitHub Copilot can walk the manor by itself, summoning evidence and passing judgments with no human at the keyboard.
A plain service layer: the risk-scoring engine and AI endpoints, which the other two surfaces both rely on.

Why that matters: because the human game and the AI-playable version share the same rules and the same scoring engine, you can directly compare how a person judges a room against how an AI agent judges it. The game doubles as a small test lab for AI decision-making.

How the pieces fit together

Where each Microsoft technology fits

Technology	Role in AfterLogin
Microsoft Foundry (Foundry IQ)	The knowledge layer. When you summon evidence, the AI doesn't make something up; it retrieves cited passages from an indexed library of documents. That "look it up and show your source" pattern is called grounded retrieval, and it's a real integration here, not a mockup.
Azure AI Search	The search infrastructure underneath Foundry IQ. It ranks results by meaning, not just keywords, and can pull out the exact sentence that answers the question.
Azure OpenAI / GitHub Models	Powers the Warden, the in-game advisor that reads the evidence and recommends the least-destructive action. Its credentials stay on the server, never in your browser.
GitHub Copilot + MCP	Copilot helped build the app, and thanks to the MCP server it can also play it, using the exact same rules a human does.
Azure Static Web Apps + Functions	The hosting. The game runs as a simple website with a small cloud function behind it for the evidence lookups.
Microsoft Entra (modelled)	The account data mimics a real Microsoft Entra directory but is entirely synthetic. There's an optional read-only mode that can grade a real directory, with a privacy flag that masks names for public demos.

How we built it

The risk engine is the game

Everything hangs off one honest scoring function. It weighs how powerful an account is (an unused administrator account is a much bigger liability than an unused mailbox), how long it has sat dormant, whether it has multi-factor authentication, and, crucially, whether it's load-bearing: whether some living process still depends on it, like an automated job that runs every night.

For the technical readers

The gameplay consequences map straight onto operational reality. Lay to rest an account that something still depends on and you spawn a Poltergeist, because you just broke a live process (minus 60 points). Shrug at a grade-F corpse and you've left breach bait sitting in a room (minus 20). And the

function grade(idn) { const dormant = daysSince(idn.lastInteractiveSignIn); const priv = isPrivileged(idn); const lb = loadBearing(idn); // live runbook refs, recent token refresh, active OAuth let score = 0; if (priv) score += 45; // privilege = blast radius if (dormant > 365) score += 35; else if (dormant > 180) score += 20; else if (dormant > 90) score += 10; if (!idn.mfaRegistered && idn.accountType !== 'servicePrincipal') score += 15; if (idn.riskLevel === 'high') score += 15; else if (idn.riskLevel === 'medium') score += 8; if (idn.accountEnabled && !lb && dormant > 180) score += 10; // enabled, dead, dormant = pure liability const g = score >= 75 ? 'F' : score >= 55 ? 'D' : score >= 35 ? 'C' : score >= 18 ? 'B' : 'A'; return { grade: g, score, privileged: priv, loadBearing: lb };

Possession monster deliberately moves toward unjudged high-risk accounts on its way to the Vault, which is more or less how real attackers move through a network.

Evidence with receipts: Foundry IQ

Inside the manor: the room roster on the left grades each account A to F, the spirit gives its testimony, and the Case panel on the right walks you through Divine (dependency mapping) and Summon (Foundry IQ cited evidence) before you judge.

The heart of the project is the evidence step. When you summon evidence against a suspicious account, the question goes to a Foundry IQ knowledge base (built on Azure AI Search) loaded with the kinds of documents a real identity team would consult: operational runbooks, system inventories, and security policies. What comes back isn't an AI's opinion. It's cited passages, with sources, like "Runbook AP-Close.ps1, line 88, still calls this account." That one line is the difference between a safe cleanup and a broken month-end close, and it's exactly what we wanted players to feel.

For the technical readers

export async function groundQuery(question, top = 3) { if (!isConfigured()) return null; // caller falls back to baked synthetic evidence const url = `${EP}/indexes/${INDEX}/docs/search?api-version=2024-07-01`; const body = { search: question, top, select: 'id,title,content,source' }; if (SEM) { body.queryType = 'semantic'; body.semanticConfiguration = SEM; body.captions = 'extractive'; body.answers = 'extractive|count-3'; } const res = await fetch(url, { method: 'POST', headers: { 'content-type': 'application/json', 'api-key': KEY }, body: JSON.stringify(body) }); const data = await res.json(); return { grounded: true, question, answers: (data['@search.answers'] || []).map(a => a.text), citations: (data.value || []).map(d => ({ title: d.title, source: d.source, score: d['@search.rerankerScore'] ?? d['@search.score'], snippet: d['@search.captions']?.[0]?.text || String(d.content).slice(0, 200) })) }; }

The Warden: an AI advisor on a short leash

The in-game advisor, the Warden, is deliberately boxed in. It receives the account's signals plus the retrieved evidence, and it must answer in a fixed format: a recommended judgment, a rationale under 180 characters, and a confidence score. The instructions we gave it are what we'd tell a junior analyst: pick the least-destructive action that's still correct. The Warden advises; the deterministic rules of the game decide what actually happens. That split, AI proposes and the engine disposes, is one we'd recommend for any AI system that touches real operations.

Letting Copilot play

Because the manor is exposed through MCP, registering it with GitHub Copilot in VS Code takes four lines of configuration:

{ "servers": { "afterlogin": { "command": "node", "args": ["mcp-server.mjs"] } } }

From there you can ask Copilot to "clear the manor" and watch it investigate rooms, summon evidence, and pass judgments in a loop, playing by the same rules a human does. The first time it correctly spared the break-glass emergency account, the one that looks abandoned but is dormant by design, was a genuinely good moment in the office.

What gave us trouble

Live cloud services fail at the worst possible moment. Anyone who has demoed on conference Wi-Fi knows this. So every cloud integration in AfterLogin checks whether it's configured and quietly falls back to built-in synthetic evidence when it isn't. The game runs fully offline, and lights up real retrieval the moment it's pointed at a Foundry knowledge base. Same code path either way, no demo-mode switch to forget.
The versions drifted apart. Early on, the human game and the AI-playable version had separate logic, and they disagreed in embarrassing ways. The fix was making the scoring engine the single source of truth that everything consumes. Now Copilot playing by itself gets identical outcomes to a human clicking rooms, which is the whole point.
A simple website can't hold secrets. A game that ships as one web page has nowhere safe to keep an API key. Everything that needs one got moved behind two small server endpoints, so credentials never touch the browser.
Teaching without preaching. Nobody wants a lecture disguised as a game. We let the mechanics do the teaching: players internalize "dormant plus powerful plus nothing depends on it means lay it to rest" because the score, the Poltergeist, and the Possession punish every other choice.

What it actually does

We'll be upfront: this is a demo and a teaching tool, not a product, and we don't have benchmark charts for a haunted house. What we can point to:

Evidence changes decisions, provably. The manor's two deceptive cases (a terrible-looking service account that something still depends on, and an emergency account that's dormant on purpose) cannot be judged correctly from the statistics alone. Both hinge on retrieved citations. Skip the evidence step and you lose the game. That's the argument for grounded AI, made playable.
Three surfaces, one brain. One scoring model powers the visual game, the AI-playable version, and the service layer. The game itself ships as a single web page.
Works offline, upgrades gracefully. With no configuration it runs anywhere, no keys and no network needed. Configured, the evidence step returns real, semantically ranked, cited passages from Azure AI Search.
It opens conversations. The game is live on Azure Static Web Apps and has become a genuinely useful icebreaker in our identity-governance work. People who would never read an orphaned-account runbook will happily spend ten minutes banishing a Possession, and they walk away able to explain "truly dead" versus "still depended on." That's more than most slide decks achieve.
Safe by construction. The bundled data is fully synthetic, with no real names or secrets anywhere. The optional live-directory mode is read-only, with a privacy flag that masks identifying details for public demos.

What we learned

Fallbacks earn their keep. Designing every integration to degrade gracefully made the app more demoable and, as a side effect, forced cleaner boundaries in the code. It's now our default pattern for anything AI-backed.
Grounding lands hardest when the ungrounded answer is wrong. The best moments in AfterLogin are the accounts where the statistics lie. If an AI retrieval demo only confirms what the raw data already says, nobody feels the value. Build the deceptive case first.
MCP is a cheap second product. Wrapping the existing game logic in ten MCP tools took a fraction of the total effort, and it turned a visual demo into something an AI agent can be tested against. If your app already has an API, it's about one file away from being usable by an AI assistant.
Constrain the model, trust the engine. Keeping the AI advisory, with a fixed answer format, a short rationale, and a confidence score, avoids both invented actions and outcomes nobody can explain afterwards. The deterministic engine always has the final say.

A field note from our MajorKey practice: this pattern isn't just for games. The identity programs we see work best run access reviews the way the manor does: a deterministic policy engine decides what's allowed, retrieved evidence about ownership, usage, and dependencies sits next to every decision, and AI recommends rather than rules. If your certification campaigns have turned into rubber stamps, the poltergeist has a lesson for them.

Where it goes next

A real Fabric IQ ontology. The web of relationships between accounts, mailboxes, groups, and live jobs is currently modelled in code. Moving it to Microsoft Fabric's semantic layer would make those "spirit threads" a governed, queryable asset.
Agent-vs-agent mode. Two AI agents racing to clear the manor, one tuned for speed and one for safety. We can't think of a more fun way to demonstrate how you evaluate AI agents.
A deeper live-directory mode. Extending the read-only Microsoft Entra integration with sign-in analysis and ownership lookups, behind the same privacy layer.
Scenario packs. The manor layout and its ghosts are just data, so industry-specific versions (healthcare, public sector, finance) are new data files plus new documents to index. Given the range of clients we see at MajorKey, this one is probably next.

Wrapping up

Orphaned accounts are a judgment problem wearing an inventory problem's clothes, and good judgment needs grounded evidence. The pattern under AfterLogin reuses well beyond games: a deterministic rules engine at the core, Foundry IQ retrieval for evidence with citations, a constrained AI for recommendations, and an MCP wrapper so AI agents can work the same process people do. It also sits squarely at the intersection our team at MajorKey Technologies works in every day, where the job is identity security and the interesting question is how AI genuinely evolves it: past chatbots, toward governed, evidence-backed decisions. This started as a side experiment, but that intersection is why it got built.

Try clearing the manor before dawn. Then look at your own organization and ask which rooms are haunted.

Explore, contribute, build on it

Watch the demo: the 90-second walkthrough on YouTube
Get the code: github.com/jlynch160/afterlogin-the-hunt, including the synthetic document library and the indexer script to stand up your own Foundry IQ knowledge base
Run it in 60 seconds: clone the repo and open index.html. No installation, no API keys, no network needed; the cloud integrations light up later if you want them
Make it yours: point the three FOUNDRY_SEARCH_* settings at your own Azure AI Search index and the grimoire starts citing your runbooks

🏆 Agents League: Celebrating the Builders Who Made Agents Battle for Glory

Lee_Stott — Thu, 23 Jul 2026 19:00:00 GMT

The arena has closed, the scores are in, and the champions are crowned. Agents League, the esports-inspired hackathon at the heart of AI Skills Fest 2026, brought together tens of thousands of developers to build, ship, and compete with agentic AI. This is the celebration post: a spotlight on the winners across every category, the numbers behind the movement, and the engineering lessons worth carrying into your next build.

If you competed, watched a Reactor battle, or completed a skilling playlist, this recap is for you. You didn't just attend an event. You helped set a record.

Before we get to the winners, there is a useful playbook hiding in their submissions. Later in this post, the section “What the Winners Teach Us: Patterns Worth Stealing” breaks down the build patterns future attendees should pay attention to: multi-agent orchestration, meeting users where they already work, testing discipline, and accessibility-first design. If you are planning to enter next time, read the winners not just as inspiration, but as a practical checklist for what great agent projects have in common.

🥇 Best Overall: $15,000 Grand Prize

Afterlogin: The Hunt — https://github.com/jlynch160/afterlogin-the-hunt (Creative Apps track)

The top weighted score of the entire field. Afterlogin: The Hunt is described by its builders as "one engine, two faces": a cinematic Security Operations Centre (SOC) night on one side, and a coached training game on the other. The same underlying engine powers both an atmospheric narrative experience and a hands-on learning simulation.

Why it won: it fused genuinely creative design with a real-world use case (security operations training) and executed both at a high polish level. Built with GitHub Copilot for AI-assisted development, it's a masterclass in using one well-architected engine to serve two very different user experiences.

🏅 Category Champions

Best Creative App: StudyMate, "Study with Matey" — https://github.com/adetorojeremiahfadesayo/StudyMate

Tech: GitHub Copilot

StudyMate turns a student's own notes into a gamified, story-driven revision session, complete with XP and progression. Instead of generic flashcards, it grounds the experience in the learner's real material and wraps it in a narrative loop that keeps motivation high. It is a great example of pairing personalisation with game mechanics to solve a persistent problem: staying engaged while revising.

Best Reasoning Agent: DELPHAI — https://github.com/jlynch160/delphai

Tech: Microsoft Foundry

DELPHAI is an 11-agent reasoning council that checks the maths, explains the risk, and refines its own answer before responding. Rather than a single model producing a one-shot response, DELPHAI orchestrates a panel of specialised agents that debate, verify, and converge, a pattern that directly addresses the reliability gap in high-stakes reasoning. It's a textbook demonstration of multi-agent orchestration on Microsoft Foundry.

Best Enterprise Agent: Archon — https://github.com/upgradedev/archon_azure

Tech: Microsoft 365 Copilot

Archon is a 7-agent financial-intelligence pipeline for small and medium businesses, delivered right inside Microsoft Teams. It meets users where they already work, chaining specialised agents to turn raw financial data into actionable intelligence. Archon shows how enterprise-ready agents built for Microsoft 365 Copilot can deliver serious business value without forcing users to leave their daily workflow.

🌟 Special Awards

Award	Winner	What made it stand out
Best Use of IQ Tools	CivicGrant IQ — https://github.com/JonEricEubanks/CivicGrant-IQ	Stood out for making the IQ layers genuinely useful: Foundry IQ, Fabric IQ and Work IQ were combined into a traceable grant-analysis workflow with grounded citations, specialist agents, GraphRAG-style evidence handling, guardrails and evaluation coverage.
Accessibility	CLARO — https://github.com/jagaor/claro SchemeSaathi — https://github.com/red-coder-27/SchemeSaathi Solace — https://github.com/MaksymY11/solace-reasoning	These teams treated accessibility as core product architecture, not a feature add-on: CLARO combined deterministic public-benefit rules with grounded bilingual reasoning; SchemeSaathi made health-scheme discovery usable across Indian languages; Solace delivered cited, plain-language immigration-rights support with safety escalation for sensitive situations.
Hack for Good	Tell My Day — https://github.com/stork066/TellMyDay BRIEF — AI Bias & Research Intelligence Evaluation Framework — https://github.com/shiwakshir/brief-agent ARGUS — Agentic Risk & Governance Unified Screening — https://github.com/iarjunganesh/argus	Each winner used agents for tangible social value: Tell My Day turns picture symbols into grounded stories for non-verbal users; BRIEF audits assumptions and bias before research begins; ARGUS turns manual KYC and governance screening into cited, auditable risk assessment.
Top Student	Athenaeum — https://github.com/theCodeForgerHQ/msf-reasoning-agent Narrative Alchemist — https://github.com/10ANT/narrative-alchemist CurriculumCraft AI — https://github.com/gideonagbavor8/curriculumcraft-ai	These student-led projects stood out for ambition and execution: Athenaeum showed exceptional engineering discipline with a deep safety and test harness; Narrative Alchemist demonstrated a complete four-agent creative production pipeline; CurriculumCraft AI translated AI into practical instructional design for Ghanaian JHS teachers.

Across 8 categories, 14 winners shared a $55,000 prize pool. Congratulations to every one of them, and to the thousands of builders who submitted.

🧠 What the Winners Teach Us: Patterns Worth Stealing

Look across the champions and clear engineering patterns emerge. These are the takeaways AI engineers and developers should carry into their own projects.

1. Multi-agent orchestration beats the monolithic prompt

The two reasoning-heavy winners, DELPHAI (11 agents) and Archon (7 agents), both won by decomposing a hard problem into specialised, cooperating agents. A "council" or "pipeline" of narrow agents that verify each other outperforms a single model asked to do everything at once.

Here's the shape of that pattern, illustrated conceptually with the Microsoft Agent Framework style used with Microsoft Foundry. (This is an illustrative pattern, not the winners' actual code.)

# Illustrative multi-agent "reasoning council" pattern # Conceptual sketch, not the winning team's source code. async def reasoning_council(question: str) -> str: # 1. Specialised agents each own one responsibility solver = create_agent( role="Propose an answer with its reasoning" ) math_check = create_agent( role="Verify every calculation, flag errors" ) risk_agent = create_agent( role="Explain the risks and assumptions" ) # 2. Draft, then critique draft = await solver.run(question) math = await math_check.run(draft) risk = await risk_agent.run(draft) # 3. Refine using the critiques before answering the user refined = await solver.run( f"Question: {question}\n" f"Draft: {draft}\n" f"Math review: {math}\n" f"Risk review: {risk}\n" "Produce a corrected, final answer." ) return refined

The lesson: reliability comes from structure. Give each agent one job, let them check each other, and refine before you respond. That is exactly how DELPHAI "checks the math, explains the risk, and refines its answer."

2. Meet users where they already are

Archon delivers financial intelligence inside Microsoft Teams. StudyMate builds on a student's own notes. Winning agents reduce friction by integrating into existing workflows rather than asking users to adopt a new destination. When you build enterprise agents, target the surface, such as Microsoft 365 Copilot, Teams, and Copilot Studio, where the work already happens.

3. Test rigouris a differentiator, not an afterthought

Athenaeum won Top Student with 611 tests behind its multi-agent learning system. In a field of prototypes, evidence of engineering discipline stood out to judges. Agentic systems are non-deterministic, so a strong evaluation and test harness is what turns a demo into something trustworthy.

4. Design for everyone from day one

CLARO (accessibility-first reasoning) and Tell My Day (assistive storytelling for non-verbal users) prove that responsible, inclusive AI isn't a constraint; it's a source of the most meaningful innovation. Accessibility completions were a top skilling path this year, and the winners reflected it.

🎮 The Three Tracks and Their Toolchains

Agents League ran three build tracks, each anchored to a Microsoft developer platform. The winners map directly onto them:

Creative Apps → GitHub Copilot. AI-assisted development from concept to code. Winners: Afterlogin: The Hunt, StudyMate. (35% of the field.)

Reasoning Agents → Microsoft Foundry. Multi-step reasoning and agent orchestration. Winner: DELPHAI. (46% of the field, the most popular track.)

Enterprise Agents → Microsoft 365 Copilot & Copilot Studio. Business-ready knowledge agents. Winner: Archon. (19% of the field.)

🔴 The Reactor Battles: Learning in Public

The live Microsoft Reactor build battles were where the thousands of developers learned together in real time. The community connected over four live broadcasts. Microsoft Reactor Agents League series

A standout moment: Kyle Daigle, CMO of Microsoft Developer and COO of GitHub, live-coded his own project on stream during the Creative Apps Reactor Battle, an authentic build-along that resonated with the developer audience. As one attendee put it: "These types of sessions are the best, where we can see code and flows. It's like we learn more by visualising."

📚 Skill Up Like a Champion: The Developer Playlists

Winners don't appear from nowhere; many skilled up first. Seven role-based developer paths on AI Skills Navigator drove thousands of completions, spanning GitHub Copilot, Microsoft Foundry, and Microsoft 365. The top five learning playlists:

Develop your first agent with Microsoft Foundry, completions (the runaway favourite)

Learn AI-assisted coding with GitHub Copilot

Accessibility in practice

Optimise agentic DevOps with Azure DevOps & GitHub Copilot

Build and orchestrate agents with Microsoft Foundry

Completion earned free GitHub & Microsoft certification vouchers and Credly badges. If you want to be on next year's winners list, this is where to start.

🔧 One Honest Lesson for Next Time

Transparency matters, so here's a builder-to-builder tip drawn from the judging data. Of the final submissions, the single biggest disqualifier was a missing demo video, followed by no listed repository. Brilliant projects were screened out on a technicality.

The takeaway for future hackathons: treat your demo video and public repository as first-class deliverables, not last-minute additions. A working agent nobody can see or reproduce can't win. Record the demo early, keep the repo public, and document how to run it.

✅ Key Takeaways

Multi-agent architectures dominated the podium: DELPHAI's 11-agent council and Archon's 7-agent pipeline show that decomposition and self-verification win.

Integration and inclusivity are differentiators: winners met users inside Teams, in their own notes, and designed for accessibility from day one.

Engineering rigour gets noticed: 611 tests helped Athenaeum win one of the Top Student awards.

Submission hygiene is non-negotiable: a demo video and public repo are the price of admission to judging.

🚀 Your Next Move

Whether you're a returning competitor or inspired to enter your first battle, here's how to keep building:

🎬 Rewatch the battles: Microsoft Reactor Agents League series
📘 Start skilling: AI Skills Navigator AISF 2026 developer playlists
🤖 Build reasoning agents: explore Microsoft Foundry
💡 Ship with AI assistance: GitHub Copilot
🏢 Build enterprise agents: Microsoft Copilot Studio

To every builder who entered the arena, streamed a battle, or completed a playlist: thank you. You didn't just compete; you showed the world what's possible when developers and agentic AI build together. The champions above earned their moment, but the real story is a global community of 31,000 developers levelling up at once.

The arena will open again. When it does, will your agent be on the podium? 🏆

Agents League was part of AI Skills Fest 2026 and was open to the public at no cost. Figures reflect the completed submission and judging period after excluding Microsoft-affiliated participants. Winner descriptions are drawn from the official Agents League Hackathon summary.

Set Up Plaud Note Pro with Microsoft Foundry

suzarilshah — Wed, 22 Jul 2026 14:59:24 GMT

Prerequisites

Riffado, up and running: follow the setup guide in the official Riffado repository to get it going with Docker Compose.
A Microsoft Foundry (formerly Azure AI Foundry) resource, with the models you want deployed; in my case, whisper for transcription and o3-mini for summaries.
A Plaud device, or any audio recordings you can import into Riffado.

Once Riffado is up, head to the Settings page > Providers > Add Provider, and select Custom. This is where the Azure details will go.

Why "OpenAI-compatible" isn’t one thing on Microsoft Foundry

Azure AI Foundry exposes two different API surfaces on the same resource, and which one serves your model depends on the model:

Surface	Path shape	Serves	OpenAI-compatible?
v1 route	/openai/v1/…	gpt-4o-transcribe, gpt-4o-mini-transcribe, chat models, embeddings	Yes: Bearer auth, model in the body, no api-version needed
Classic route	/openai/deployments/{name}/…	Whisper (and other legacy audio)	No: deployment name lives in the URL, and ?api-version= is mandatory

A generic OpenAI client (Riffado's included) can only speak the first dialect. It has nowhere to put a deployment name in the path and no way to append a query parameter. That single fact drives everything below.

Part 1 - Transcription

Whisper and the DeploymentNotFound mystery

Symptom

My very first transcription attempt in Riffado failed with 404 Resource not found. Off to a flying start. Configured provider: base URL https://<resource>.services.ai.azure.com, model whisper.

Dead end #1: the missing path

The first bug was mine: the base URL had no path. Riffado's OpenAI client appends /audio/transcriptions to whatever you give it, so requests were hitting https://<resource>…/audio/transcriptions, a path that doesn't exist on the resource at all. Fixing the base URL to end in /openai/v1 got us to a more interesting error:

POST /openai/v1/audio/transcriptions · model=whisper

{"error":{"code":"DeploymentNotFound","message":"The API deployment for

this resource does not exist. If you created the deployment within the

last 5 minutes, please wait a moment and try again."}}

Dead end #2: catalog ≠ deployment

Worth checking before anything else: selecting a model in the Foundry catalog is not deploying it. GET /openai/v1/models lists everything you could deploy; only Deployments → Deploy model creates an endpoint that answers. If you get DeploymentNotFound, first confirm a deployment actually exists (the listing below requires only the API key):

enumerate real deployments (classic control-plane, key auth)

curl -s -H "api-key: $KEY" \

  "https://<resource>.openai.azure.com/openai/deployments?api-version=2023-03-15-preview"

# → {"data":[{"id":"whisper","model":"whisper","status":"succeeded",…}]}

The actual cause

Here is the part that nearly drove me mad: the deployment existed and was succeeded, yet the v1 route still said DeploymentNotFound. Because Whisper deployments are not served on the v1 route at all. They only answer on the classic path. Verified side by side with the same tiny WAV file:

Request	Result
POST /openai/v1/audio/transcriptions · model=whisper · Bearer	404 DeploymentNotFound
POST /openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01 · Bearer	200 {"text":"you"}
Same classic path, without ?api-version=	404 Resource not found

Three constraints, then: Whisper needs the classic path; the classic path needs api-version; Riffado can send neither. One piece of good news hiding in the table: the classic route accepts Authorization: Bearer, not just Azure's api-key header, so the shim doesn't have to touch auth at all.

The fix: a Caddy shim

Drop a stock caddy:2-alpine container into the Compose network. Riffado points at it as if it were OpenAI; the shim rewrites the path, injects api-version, and proxies to Azure. The Bearer header passes through untouched.

azure-shim.Caddyfile

{ admin off auto_https off } :80 { @transcribe path /v1/audio/transcriptions /audio/transcriptions handle @transcribe { rewrite * /openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01 reverse_proxy https://<resource>.services.ai.azure.com { header_up Host <resource>.services.ai.azure.com } } handle { respond "azure-shim ok" 200 } }

docker-compose.yml (added service)

azure-shim: image: caddy:2-alpine restart: unless-stopped volumes: - ./azure-shim.Caddyfile:/etc/caddy/Caddyfile:ro

Riffado's provider settings become:

Field	Value
Base URL	http://azure-shim/v1
Model	whisper (must equal the deployment name)
API key	the Azure resource key (forwarded as Bearer)

Verified

From inside the Riffado container: POST http://azure-shim/v1/audio/transcriptions → 200 {"text":"…"}. Transcription works end-to-end in the UI.

Part 2 · Summaries & titles

o3-mini and the empty answer

Symptom

The summary button showed "An unexpected error occurred." The container logs were more honest:

riffado-app logs

Error generating title: TypeError: undefined is not an object

(evaluating 'C.choices[0]')

Riffado calls chat/completions and reads choices[0] without checking whether the response was an error. So anything the API refuses becomes "an unexpected error." What was it refusing?

Cause 1: reasoning models reject the classic knobs

o3-mini belongs to Azure/OpenAI's o-series reasoning models, which hard-reject parameters every classic chat client sends. Riffado sends temperature: 0.7 and max_tokens: 50 for titles (0.5 / 2000 for summaries), and o3-mini answers:

POST /openai/v1/chat/completions · model=o3-mini

HTTP 400

{"error":{"message":"Unsupported parameter: 'max_tokens' is not supported

with this model. Use 'max_completion_tokens' instead.", …}}

# and with max_tokens fixed:

HTTP 400

{"error":{"message":"Unsupported parameter: 'temperature' is not supported

with this model.", …}}

Cause 2: reasoning tokens starve the output

Stripping the bad params gets you to 200, and then comes a subtler failure, my personal favourite of this whole saga. Reasoning models spend completion tokens on internal "thinking" before emitting a single visible character. Riffado's 50-token title budget is consumed entirely by reasoning, and the reply comes back syntactically valid and empty:

max_completion_tokens	reasoning_effort	finish_reason	content
50	not set	length	"" (all 50 spent reasoning)
2000	not set	stop	"Q3 Budget Planning Strategy Meeting"
2000	low	stop	same, less reasoning overhead

The fix: a Node shim that rewrites the request body

Caddy can rewrite paths but not JSON bodies, so this shim is ~60 lines of dependency-free Node on node:20-alpine. Per request it: converts max_tokens → max_completion_tokens, strips temperature / top_p / penalties, floors the token budget at 4000, sets reasoning_effort: "low", maps /v1/* → /openai/v1/*, and forwards to the Azure resource.

o3-shim.js

docker-compose.yml (added service)

o3-shim: image: node:20-alpine restart: unless-stopped working_dir: /app command: ["node", "/app/o3-shim.js"] volumes: - ./o3-shim.js:/app/o3-shim.js:ro

Add a second provider in Riffado (base URL http://o3-shim/v1, model o3-mini, the resource's API key) and set it as the default enhancement provider (summaries/titles), keeping the Whisper one as default for transcription.

Riffado's exact title request (temperature: 0.7, max_tokens: 50) through the shim → 200, finish_reason: stop, real title text. A full meeting-transcript summary returns structured key points and action items.

The final shape

Reading it left to right: Riffado never talks to Azure directly. Transcription requests pass through azure-shim, a stock Caddy container that rewrites each request onto Whisper's classic deployment path and injects the mandatory api-version parameter. Summary and title requests pass through o3-shim, a tiny Node server that rewrites the request body into the shape o3-mini accepts and floors the token budget so the model's internal reasoning cannot starve the actual answer. As far as Riffado is concerned, it is simply talking to two ordinary OpenAI providers.

Both shims live on the Compose network only; nothing is exposed publicly. Riffado is unmodified.

Verification checklist

Each layer, testable in isolation. Run these before blaming the app:

smoke tests

# 1. Key + resource alive? (v1 models listing, Bearer auth)

curl -s -H "Authorization: Bearer $KEY" \

  https://<resource>.services.ai.azure.com/openai/v1/models | head -c 200

# 2. Whisper answers on the classic path?

curl -s -H "Authorization: Bearer $KEY" -F file=@test.wav \

  "https://<resource>.services.ai.azure.com/openai/deployments/whisper/audio/transcriptions?api-version=2024-06-01"


# 3. Shim translates correctly? (from inside the compose network)

docker exec riffado-app node -e "fetch('http://azure-shim/')

  .then(r=>r.text()).then(console.log)"


# 4. o3-mini via shim, sending the params Riffado sends?

#    (temperature + max_tokens:50; the shim must absorb both)

If you'd rather not run shims

Both shims exist because of the specific models chosen. Pick models that live natively on the v1 route and Riffado connects directly, with base URL https://<resource>.services.ai.azure.com/openai/v1 and zero extra containers:

Transcription: deploy gpt-4o-mini-transcribe (or gpt-4o-transcribe) instead of Whisper.
Summaries: deploy a non-reasoning chat model such as gpt-4o-mini, which happily accepts temperature and max_tokens.

The shim approach earns its keep when you're standardized on specific models (Whisper's transcription quality, o3-mini's reasoning), or when you want a control point to add logging, retries, or budget caps later.

For reference, this is what the finished setup looks like on Riffado's side. Each shim is registered as a plain Custom provider. Here is the whisper provider pointing at azure-shim, with Use for transcription ticked:

And once both are saved, they sit side by side in the providers list, whisper tagged for transcription and o3-mini tagged for enhancement:

A quick look at the Foundry portal

In the Microsoft Foundry portal, head over to Models > AI Services and you will find a pleasant surprise: fifteen AI service models already deployed and ready to use, covering the Azure Speech family (including Voice Live and Speech to Text), Azure Translator, Azure Language, and Content Understanding:

You can of course deploy another model for this, but the pre-deployed ones are a handy cost-saving option. Click on the Azure Speech – Voice Live radio button and you will be shown the Base URL and API Key, which you can then paste into the provider settings on Riffado's Settings page.

A quick note on cost: these services are not free. They are billed pay-as-you-go based on usage. Azure Speech transcription is charged per audio hour, and Voice Live pricing is tiered by the model you choose. The free tier does include a monthly allowance, though. Check the Azure Speech pricing page before committing.

And if you would rather deploy a dedicated transcription model such as whisper, Foundry gives you the flexibility to do just that. Open the model page in the catalogue, click Deploy, and go with Default settings unless you need custom quotas or guardrails:

Let's test the setup

On your Plaud device, just tap to start recording. The little LED bars light up to show it is listening:

Plaud Note Pro physical device

Or skip the device entirely and upload an audio file straight into Riffado using the Upload Audio button. Either way, the recording lands on the Recordings page; hit Transcribe and let the spinner do its thing:

As you can see below, whisper, the transcription model we deployed earlier, even managed to transcribe a recording in Malay without a hitch. My 3:32 test clip came back as 186 words of clean Malay, with the language correctly detected and tagged:

I have also set o3-mini as the enhancement provider, and it enhanced the transcription with a proper summary, key points, and title as well! The Meeting Notes-style summary came straight out of o3-mini through the shim, with zero manual prompting.

Wrapping up

What started as a TikTok-fuelled impulse buy nearly killed off by subscription pricing ended up as a fully self-hosted pipeline: Plaud for recording, Riffado as the interface, and Microsoft Foundry serving whisper and o3-mini behind two tiny shims. The total extra infrastructure came to two containers and roughly sixty lines of code, and not a single monthly subscription in sight.

If you try this setup and run into a failure mode I have not covered here, do share it in the comments. Half the fun is in the debugging.

Microsoft Agent Framework Multi-Agent Workflow Architecture for Automated Kubernetes Assessments

cyruswong — Tue, 21 Jul 2026 07:46:08 GMT

Why this system generates tests (design rationale)

This project does not generate tests just to "check code." It generates tests because, in a Kubernetes learning game, the test suite is the grading contract.

The design goals are:

Scale content creation: instructors should not hand-author every task and checker.
Keep grading objective: student success is measured against Kubernetes API state, not subjective review.
Avoid fragile tasks: generated tasks must survive empty/wrong cluster states without crashing.
Make failures repairable: when checks break, the system should patch and re-validate automatically.

So the pipeline generates a full task package (setup, answer, check, cleanup) where tests define exactly what "correct" means.

Lifecycle: from concept to production-ready grader

Reader mental model: this is a content compiler with validation stages, not a single chat completion.

Phase 1: Pedagogical intent -> structured concept

The Idea Agent converts a topic into a constrained concept object (objective, progression, task IDs, difficulty variants). Memory rules block duplicates and previously failed concepts.

Phase 2: Concept -> executable grading package

The Generator Agent turns that concept into files:

student instructions (instruction.md)
learning material (concept.md)
parameter source (session.json)
setup and answer manifests (setup.template.yaml, answer.template.yaml)
deterministic pytest flow (test_01 ... test_06)

At this point, output is still untrusted draft content.

Phase 3: Structural correctness gate

Deterministic validation checks file presence, syntax, JSON/YAML shape, and template correctness. This catches basic integrity issues before cluster execution.

Phase 4: Behavioral correctness gate (real cluster)

Pytest executes against Kubernetes and verifies runtime behavior using real kubectl-derived state. This proves that generated checks actually evaluate cluster resources as intended.

Phase 5: Anti-false-positive gate (skip-answer mode)

The same suite runs with SKIP_ANSWER_TESTS=True to verify grader integrity:

answer deployment is skipped
test_05_check.py must fail

If it still passes, the grader is invalid (it would accept wrong student submissions).

Phase 6: Self-healing repair loop

On any failure, deterministic error logs are fed into the Fixer Agent, which patches only broken files. The workflow then re-enters validation + test gates.

Phase 7: Finalization

If all gates pass -> task is kept as production-ready content.
If retries are exhausted -> task is moved to unsuccessful/ with FAILURE_REPORT.txt for human triage.

This lifecycle explains the core architecture decision: LLMs generate candidate graders, deterministic execution certifies them.

Concrete generated sample (what the pipeline actually produces)

Below is a representative generated task for topic: ConfigMap Environment Variable Injection.

Generated directory layout

tests/game01/050_configmap_env_injection/ ├── __init__.py ├── instruction.md ├── concept.md ├── session.json ├── setup.template.yaml ├── answer.template.yaml ├── test_01_setup.py ├── test_02_ready.py ├── test_03_answer.py ├── test_05_check.py └── test_06_cleanup.py

session.json (runtime variables)

{ "namespace": "{{random_name()}}{{random_number(100,999)}}{{student_id()}}", "configmap_name": "{{random_name()}}", "deployment_name": "{{random_name()}}", "container_name": "app", "env_key": "APP_MODE", "env_value": "production" }

Why this exists: task values are randomized per student/session, so tests verify behavior by variable contract instead of hardcoded names.

setup.template.yaml (baseline state only)

apiVersion: v1 kind: Namespace metadata: name: {{ namespace }}

Why this exists: setup should create prerequisites only. It must not accidentally include the final answer.

answer.template.yaml (expected correct solution)

apiVersion: v1 kind: ConfigMap metadata: name: {{ configmap_name }} namespace: {{ namespace }} data: {{ env_key }}: "{{ env_value }}" --- apiVersion: apps/v1 kind: Deployment metadata: name: {{ deployment_name }} namespace: {{ namespace }} spec: replicas: 1 selector: matchLabels: app: env-demo template: metadata: labels: app: env-demo spec: containers: - name: {{ container_name }} image: nginx:latest env: - name: {{ env_key }} valueFrom: configMapKeyRef: name: {{ configmap_name }} key: {{ env_key }}

Why this exists: defines canonical "correct cluster state" that graders must detect.

test_02_ready.py (wait for setup resources)

import json import time from tests.helper.kubectrl_helper import build_kube_config, run_kubectl_command class TestReady: def test_001_namespace_active(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) time.sleep(2) result = run_kubectl_command( kube_config, f"kubectl get namespace {json_input['namespace']} -o json", ) data = json.loads(result) assert data.get("status", {}).get("phase") == "Active"

Why this matters: validates setup-stage readiness only. It should not check answer resources yet.

test_05_check.py (student grading contract)

import json from tests.helper.kubectrl_helper import build_kube_config, run_kubectl_command class TestCheck: def test_001_configmap_key_exists(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) result = run_kubectl_command( kube_config, f"kubectl get configmap {json_input['configmap_name']} -n {json_input['namespace']} -o json", ) data = json.loads(result) assert data["data"][json_input["env_key"]] == json_input["env_value"] def test_002_deployment_uses_configmap_env(self, json_input): kube_config = build_kube_config( json_input["cert_file"], json_input["key_file"], json_input["host"] ) result = run_kubectl_command( kube_config, f"kubectl get deployment {json_input['deployment_name']} -n {json_input['namespace']} -o json", ) data = json.loads(result) env = data["spec"]["template"]["spec"]["containers"][0].get("env", []) matched = [ e for e in env if e.get("name") == json_input["env_key"] and e.get("valueFrom", {}).get("configMapKeyRef", {}).get("name") == json_input["configmap_name"] and e.get("valueFrom", {}).get("configMapKeyRef", {}).get("key") == json_input["env_key"] ] assert matched, "Deployment container must consume env var from ConfigMap key"

Why this matters: this is the real grading logic. If a student deploys wrong resource wiring, this test fails with explicit reason.

Why skip-answer validation is essential for this sample

When SKIP_ANSWER_TESTS=True, answer deployment is skipped. In this mode:

test_03_answer.py should be skipped
test_05_check.py must fail (ConfigMap/Deployment wiring is absent)

If test_05_check.py still passes, the grader is broken (false positive), and the workflow routes to Fixer.

Rule Builder Workflow Flowchart

Multi-graph architecture views

1) Control-plane graph (orchestration DAG)

2) Runtime sequence (who calls what)

3) State machine (task lifecycle)

4) Prompt lifecycle graph (how prompts evolve)

5) MCP + Kubernetes execution boundary graph

Core Agent Framework Primitives Used

This repository is a practical example of Agent Framework as a graph orchestrator, not just an agent wrapper.

WorkflowBuilder builds a typed DAG with explicit edges.
@executor functions implement deterministic nodes (validation, pytest, decisions, routing prep).
AgentExecutor wraps LLM agents so they behave as graph nodes.
WorkflowContext shared state carries typed data and retry metadata between nodes.
add_multi_selection_edge_group(...) + selector functions enforce conditional routing.
MCPStdioTool connects filesystem MCP tools into agents for controlled file I/O.

Production graph construction (from workflow/builder.py) looks like this:

workflow = ( WorkflowBuilder(start_executor=initialize_retry) .add_edge(initialize_retry, generator_executor) .add_edge(generator_executor, parse_generated_task) .add_edge(parse_generated_task, run_validation) .add_edge(run_validation, run_pytest) .add_edge(run_pytest, make_decision) .add_multi_selection_edge_group( make_decision, [keep_task, remove_task], selection_func=select_action, ) .add_edge(keep_task, run_pytest_skip_answer) .add_multi_selection_edge_group( run_pytest_skip_answer, [check_loop, complete_workflow], selection_func=select_skip_answer_action, ) .add_multi_selection_edge_group( check_loop, [fix_task, complete_workflow], selection_func=select_loop_action, ) .add_edge(fix_task, fixer_executor) .add_edge(fixer_executor, parse_generated_task) .build() )

This is the architectural heart of the system: agents and deterministic executors are first-class nodes in the same graph.

Detailed Node-by-Node Mechanics

1. Idea Agent (🧠): Concept Synthesis with Memory Constraints

The Idea Agent (agents/k8s_task_idea_agent.py) generates a structured concept with three difficulty variations (BEGINNER/INTERMEDIATE/ADVANCED). It is memory-aware:

task_ideas_memory.json tracks successful concepts.
task_ideas_failure_memory.json tracks concepts that failed downstream.
Memory constraints are injected using AgentMiddleware (system-level prompt injection) to avoid duplicate or previously failed concepts.

For Responses-only models, the agent switches to a tool-call contract (save_k8s_task_concept) instead of structured response formatting.

2. Generator Agent (⚙️): MCP-Backed File Authoring

The Generator Agent receives a strict prompt and writes task files through MCP filesystem tools. Key framework details:

Built through chat_client.as_agent(...).
MCP tool attached via tools=mcp_tool.
Function-call execution observability added with LoggingFunctionMiddleware.
Uses absolute-path-only policy in instructions to prevent path drift.

3. Deterministic Validation + Test (✅): Non-LLM Gates

After generation, the graph moves through deterministic executors:

run_validation calls pure Python checks (k8s_task_validator).
run_pytest executes pytest --import-mode=importlib --rootdir=. ....
Raw pytest output is persisted in workflow state for later fixing.

This is critical: no LLM is asked whether code is correct.

4. Skip-Answer Test (🧪): Grader Correctness Gate

Even if standard tests pass, the workflow enforces a second tier:

SKIP_ANSWER_TESTS=True pytest --import-mode=importlib --rootdir=.

Implementation detail: the executor writes JUnit XML, parses it, and asserts that:

test_03_answer.py is skipped
test_05_check.py fails as expected

If test_05_check.py does not fail, the task is treated as invalid and sent back to retry/fix.

5. Fixer Agent + Retry Loop (🔧): Bounded Self-Healing

On failure, fix_task builds a targeted prompt containing:

failure reasons from deterministic nodes
full captured pytest output
explicit rule to patch only broken files in place

The Fixer Agent runs through AgentExecutor, writes patches via MCP, and the graph loops back to parse_generated_task.

Retries are stateful (retry_count, max_retries) and hard-bounded. On exhaustion, complete_workflow moves the task to unsuccessful/<game>/ and writes FAILURE_REPORT.txt.

Agent Prompt Design (The Part That Makes It Work)

If you want to understand why this pipeline works, you need to inspect prompts as operational contracts, not generic instructions.

Real Idea Agent Prompt (from code)

IDEA_AGENT_INSTRUCTIONS = ( "You are a Kubernetes task idea generator that creates detailed task concepts with three difficulty variations. " "Read official K8s documentation and propose comprehensive learning concepts for a Kubernetes game. " "\n\nYour task:\n" "1. Choose ONE Kubernetes concept not yet covered (check context for existing concepts)\n" "2. Generate exactly 3 variations: BEGINNER, INTERMEDIATE, and ADVANCED\n" "3. Use 3-digit task IDs (001-999) in format: XXX_concept_name_level (e.g., 041_secrets_basic)\n" "4. Each variation should build on the previous one with increasing complexity\n" "5. Include practical, hands-on scenarios covering: Workloads, Services, Storage, Configuration, Security, Scheduling, Policies\n" "\nProvide the concept, tags, description, and 3 variations with task_id, difficulty, title, objective, key_skills, and estimated_time." )

Responses-only models use a stricter tool-call version:

IDEA_AGENT_INSTRUCTIONS_TOOL_CALL = ( IDEA_AGENT_INSTRUCTIONS + "\n\n" "**CRITICAL**: You MUST call the save_k8s_task_concept tool to save your generated concept.\n" "...\n" "Always call save_k8s_task_concept with your generated concept." )

Idea Agent Prompt Contract

The Idea Agent prompt enforces:

one concept per run
exactly three difficulty variations
strict task ID format (XXX_concept_name_level)
practical skill progression

Core pattern:

You are a Kubernetes task idea generator... 1. Choose ONE Kubernetes concept not yet covered 2. Generate exactly 3 variations: BEGINNER, INTERMEDIATE, ADVANCED 3. Use 3-digit task IDs in format XXX_concept_name_level ...

It is strengthened by runtime memory injection:

previously generated concepts are blocked
previously failed concepts are blocked

For Responses-only models, the contract becomes tool-driven:

CRITICAL: You MUST call the save_k8s_task_concept tool...

This reduces ambiguity in output structure and makes downstream parsing deterministic.

Real Generator Agent Prompt (from code)

def _get_generator_instructions(): return ( "You are a Kubernetes task generator with filesystem tools.\n" f"The MCP filesystem is rooted at: {PATHS.tests_root.parent}\n" f"You MUST use ABSOLUTE paths for ALL file operations.\n" f"Task directory: {PATHS.game_root}/XXX_task_name/\n" "...\n" "CRITICAL: test_02_ready.py checks resources from setup.template.yaml, NOT answer.template.yaml.\n" "...\n" "MUST use polling loops (60s timeout, 15s interval)\n" "MUST use try/except and safe .get() JSON access\n" )

The generator prompt is long on purpose: it encodes path correctness, file schema, YAML/Jinja structure, and testing strategy in a single deterministic contract.

Generator Agent Prompt Contract

The Generator prompt is intentionally long and prescriptive because it defines filesystem safety and grading correctness requirements.

Key constraints encoded in the prompt:

Absolute path writes only (prevents writing to wrong workspace paths)
No directory creation (directory is pre-created by executor)
Required file set (instruction.md, concept.md, session.json, templates, tests)
test-flow invariants:
- test_01_setup.py deploys setup
- test_02_ready.py checks setup resources only
- test_03_answer.py deploys answer
- test_05_check.py validates final solution
robust test coding style: polling loops, try/except, .get()-based JSON parsing, explicit debug output

Example contract fragment:

CRITICAL PATH RULES: ✅ CORRECT: /abs/path/tests/gameXX/050_task/file.py ❌ WRONG: tests/gameXX/050_task/file.py (relative) CRITICAL: test_02_ready.py checks resources from setup.template.yaml, NOT answer.template.yaml.

This is why generation quality is high before the Fixer loop even starts.

Real Runtime Retry Prompt Builder (from code)

def _build_retry_generation_prompt(combined: CombinedValidationResult) -> str: task_id = combined.test.task_id failure_reasons = _build_failure_reasons(combined) return ( f"Generate a complete Kubernetes learning task with ID '{task_id}' about '{combined.target_topic}'. " f"This is retry attempt {combined.retry_count + 1} of {combined.max_retries}. " f"\n\n⚠️ PREVIOUS ATTEMPT FAILED:" f"\n{chr(10).join([f' - {reason}' for reason in failure_reasons])}" f"\n\nIMPORTANT: You MUST use the exact task ID '{task_id}' - do not generate a new ID." f"\n\n✅ Directory already exists: {PATHS.game_root}/{task_id}/" f"\nWrite all files directly into this directory. Do NOT call create_directory." "..." )

This means retries are not generic retries; they are failure-conditioned retries with precise constraints.

Fixer Agent Prompt Contract

The Fixer prompt is a repair protocol, not a regeneration prompt. It includes:

exact failure reasons from deterministic validators
raw pytest output
instruction to read current task files first
strict directive to patch only broken files

Core behavior constraints:

DO NOT rewrite all files. Make TARGETED FIXES to ONLY the broken files. Use ABSOLUTE paths for all file operations.

This keeps retries cheap, preserves working artifacts, and improves convergence speed.

Real Runtime Fix Prompt Builder (from code)

def _build_fix_prompt(combined: CombinedValidationResult, raw_test_output: str) -> str: task_id = combined.test.task_id failure_reasons = _build_failure_reasons(combined) prompt = ( f"Fix the failed Kubernetes task '{task_id}' located in '{PATHS.game_root}/{task_id}/'." f"\n\nThis is fix attempt {combined.retry_count + 1} of {combined.max_retries}." f"\n\n⚠️ TASK FAILED WITH THESE ERRORS:" f"\n{chr(10).join([f' - {reason}' for reason in failure_reasons])}" ) if raw_test_output: prompt += f"\n\n📋 FULL TEST OUTPUT:\n```\n{raw_test_output}\n```" prompt += ( f"\n\n🔍 YOUR TASK:" f"\n1. READ all files from '{PATHS.game_root}/{task_id}/'" f"\n6. Make TARGETED FIXES to ONLY the broken files" f"\n7. WRITE ONLY the fixed files back" f"\n\n⚠️ CRITICAL: DO NOT rewrite all files! Only fix the broken ones!" ) return prompt

How Prompt Output Enters the Agent Framework Graph

The prompt builders above are used by deterministic executors and sent to agent nodes through AgentExecutorRequest:

await ctx.send_message( AgentExecutorRequest( messages=[Message(role="user", contents=[fix_prompt])], should_respond=True ) )

So prompt generation and graph routing are tightly coupled: each route transition emits a specific prompt payload into the next LLM node.

Runtime-Constructed Prompts in Executors

The most important prompts are built dynamically in workflow executors:

_build_retry_generation_prompt(...)
_build_fix_prompt(...)

These functions inject live context:

retry_count / max_retries
concept + objective metadata
validation/test failure reasons
full captured test logs

So each retry is context-rich and specific, not another blind generation attempt.

Prompt + Middleware + Deterministic Gates = Reliability

In this repository, reliability does not come from prompt text alone. It comes from three layers working together:

Prompt contracts constrain agent behavior.
Middleware injects memory and logs tool invocations.
Deterministic executors enforce objective pass/fail gates.

That combination is why the workflow remains auditable and predictable even when LLM outputs vary.

Agent Framework Execution Model in This Repo

Strongly-Typed Message Passing

workflow/models.py defines transport models used between nodes:

ValidationResult and TestResult (Pydantic)
CombinedValidationResult (dataclass with should_keep and should_retry)
InitialWorkflowState (seed payload for each run)

This keeps node contracts explicit and simplifies selector logic.

Fail-Fast Shared State Management

Executors use ctx.get_state(...) with a sentinel (_MISSING) and raise explicit exceptions if required state is absent. This prevents hidden fallback behavior and catches graph/data wiring errors early.

Conditional Routing with Selectors

Selectors (workflow/selectors.py) encode graph decisions:

select_action → keep vs remove
select_skip_answer_action → complete vs loop
select_loop_action → fix vs complete

This separates decision policy from executor implementation.

Streaming Workflow Runtime

workflow.run(initial_state, stream=True) emits output events incrementally. The runner (workflow/runner.py) consumes these events to detect successful completions and update concept memory accordingly.

Agent Construction and API Selection Strategy

The repository uses Azure CLI auth (AzureCliCredential) and dynamically selects API mode by deployment name (agents/config.py):

Chat Completions path: OpenAIChatCompletionClient
Responses-only model path: OpenAIChatClient or custom ResponsesAgent

Why this matters: some codex-class deployments are Responses-only, so the architecture supports both without changing workflow logic.

How MCP Actually Controls Kubernetes (Important Distinction)

In this repo, MCP is used for filesystem control; Kubernetes control is done through kubectl tools.

1) MCP server role: controlled file I/O

The workflow starts MCP stdio servers (official filesystem server) and mounts them into agents:

docs_mcp_tool = MCPStdioTool( name="filesystem_docs", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", str(PATHS.k8s_docs_root)], load_prompts=False, ) tests_mcp_tool = MCPStdioTool( name="filesystem_tests", command="npx", args=["-y", "@modelcontextprotocol/server-filesystem", str(PATHS.tests_root.parent)], load_prompts=False, )

Those MCP tools are passed into Generator/Fixer agents, which then call MCP file functions (read/write/list) inside allowed roots only.

2) Kubernetes cluster control role: kubectl execution tool

Cluster actions are not performed by MCP filesystem server; they are performed by a dedicated function tool:

def run_kubectl_command(command: str) -> str: kubeconfig_path = os.environ.get("KUBECONFIG", "/home/developer/.kube/config") cmd_list = ["kubectl"] + command.split() result = subprocess.run( cmd_list, capture_output=True, text=True, check=True, env={**os.environ, "KUBECONFIG": kubeconfig_path}, ) return result.stdout

And the Kubernetes agent forces tool usage:

agent = responses_client.as_agent( name="KubernetesAgent", instructions="...You MUST use the run_kubectl_command tool...", tools=[run_kubectl_command], default_options={"tool_choice": "required"}, )

So the control plane is:

MCP filesystem → manipulate generated task files.
kubectl tool → query/mutate real cluster state.
deterministic pytest/validator executors → accept or reject results.

3) End-to-end command flow in practice

When generated tests run, they execute real kubectl get ... -o json checks in test code, and the deterministic runner captures raw output:

pytest_command = f"pytest --import-mode=importlib --rootdir=. {task_with_val.task_directory}/" result = run_pytest_command(pytest_command) ctx.set_state(f"raw_output_{task_with_val.task_id}", raw_output)

This means Kubernetes state verification is always grounded in live command output, not model speculation.

Should MCP run Kubernetes tests?

Short answer: not in this design.

Current architecture keeps test execution deterministic and local:

pytest is run by run_pytest_command(...) (pure Python subprocess runner)
test results are parsed and stored in workflow state
retry/fix routing uses those deterministic outputs

This is intentional. If test execution were delegated to an LLM-facing MCP command tool, you would lose strict control over execution semantics and error handling.

Recommended pattern:

Use MCP for file/document access and controlled editing.
Use deterministic executors for pytest and validation.
Use LLM agents only for generation and repair.

If you still want MCP-driven test execution, add a separate locked-down command MCP server (only whitelisted pytest/kubectl commands), but keep pass/fail decision logic in deterministic executors.

How tests are run in this workflow (with code)

The workflow executes tests in deterministic executors, not inside LLM agents.

1) Workflow node calls pytest runner

run_pytest executor builds the command and calls the pure Python runner:

@executor(id="run_pytest") async def run_pytest(task_with_val: TaskWithValidation, ctx: WorkflowContext[TestResult]) -> None: from agents.pytest_runner import run_pytest_command pytest_command = f"pytest --import-mode=importlib --rootdir=. {task_with_val.task_directory}/" result = run_pytest_command(pytest_command) raw_output = result["details"][0] if result.get("details") else "" ctx.set_state(f"raw_output_{task_with_val.task_id}", raw_output) ...

2) Deterministic subprocess execution

The runner normalizes command flags and executes pytest via subprocess:

def run_pytest_command(command: str) -> dict[str, Any]: normalized_command = _normalize_pytest_command(command) # adds -s if needed cmd_list = shlex.split(normalized_command) result = subprocess.run( cmd_list, capture_output=True, text=True, check=False, cwd=str(PATHS.pytest_rootdir), ) combined_output = result.stdout + "\n" + result.stderr _save_test_output(normalized_command, combined_output, skip_answer) ...

Exit codes are interpreted deterministically:

0 → pass
5 → no tests collected (fail)
others → fail with exit code reason

3) Skip-answer validation tier

After normal pass, the workflow runs pytest again with SKIP_ANSWER_TESTS=True and parses JUnit XML:

os.environ["SKIP_ANSWER_TESTS"] = "True" pytest_command = f"pytest --import-mode=importlib --rootdir=. --junitxml={junit_path} {task_dir}/" result = run_pytest_command(pytest_command) test_05_failed, test_03_skipped = _parse_skip_answer_junit(junit_path)

The parser checks per-testcase outcomes:

if "test_05_check.py" in context and has_failure_or_error: test_05_failed = True if "test_03_answer.py" in context and has_skipped: test_03_skipped = True

4) How failures trigger fix loop

If pytest fails (or skip-answer logic fails), failure reasons and raw output are pushed into state, then the Fixer Agent receives a generated fix prompt containing that output:

ctx.set_state(f"failure_reasons_{task_id}", reasons) ctx.set_state(f"raw_output_{task_id}", raw_output) fix_prompt = _build_fix_prompt(combined, raw_test_output) await ctx.send_message( AgentExecutorRequest( messages=[Message(role="user", contents=[fix_prompt])], should_respond=True ) )

That is the key loop: deterministic test output drives LLM repair, then deterministic tests re-run.

ResponsesAgent Internals (Advanced Agent Framework Pattern)

The custom ResponsesAgent (agents/responses_agent.py) demonstrates a lower-level integration pattern:

Connect MCP tool lazily.
Call Responses API.
Parse ResponseFunctionToolCall items.
Execute tools (MCP + custom callables).
Feed function_call_output back to model.
Repeat until final text response.

It also runs a middleware chain around tool invocations, preserving observability and consistency with standard agent paths.

Why This Architecture Is Robust

This design works because Agent Framework is used as a deterministic orchestration layer around probabilistic generation:

LLM creativity is constrained by typed state and strict prompts.
deterministic executors act as objective quality gates.
retries are targeted, bounded, and auditable.
failures produce durable forensic artifacts (FAILURE_REPORT.txt + test logs).

For Kubernetes education pipelines, this yields high throughput without sacrificing grader reliability.

GitHub Repo - https://github.com/wongcyrus/k8s-game-rule-builder

About the Author

Cyrus Wong is the senior lecturer of Hong Kong Institute of Information Technology (HKIIT) @ IVE(Lee Wai Lee).and he focuses on teaching public Cloud technologies. He is a passionate advocate for the adoption of cloud technology across various media and events. With his extensive knowledge and expertise, he has earned prestigious recognitions such as AWS AI Hero, Microsoft MVP- Microsoft Foundry, and Google Developer Expert - Cloud(AI).

Building a Fully Managed Multi-Agent Pipeline with Microsoft Foundry

ShivamGoyal — Wed, 15 Jul 2026 07:00:00 GMT

Hey everyone! I am Shivam Goyal, a Microsoft MVP, and I am super excited to share a workshop I created that is going to save you a massive amount of time.

Designing smart AI workflows is arguably one of the most creative and enjoyable parts of engineering. Trying to force a single, massive prompt to execute a complex, multi-step business pipeline perfectly? Not so much. We have all been there: you write a giant prompt to handle a multi-step task, only for the AI to get confused, miss critical details, and give completely different answers every time you run it.

To solve this giant prompt problem, I built a hands-on multi-agent workshop. Today, we are looking at how you can use the Foundry Toolkit for VS Code alongside the project blueprints I designed to build, debug, and deploy specialized multi-agent teams without any infrastructure headaches.

What is the Foundry Toolkit for VS Code?

The Foundry Toolkit for VS Code is a unified extension that brings cloud-scale AI development right into your local code editor. Instead of constantly jumping between web portals, command lines, and code files, the toolkit gives you a single place to manage your entire AI application.

With this toolkit installed in Visual Studio Code, you can:

Access the Model Catalog: Instantly browse and connect your code to leading AI models.
Build Locally: Write your agent logic using standard frameworks like the Microsoft Agent Framework (MAF).
Inspect & Debug: Trace data flow and agent conversations in real-time before moving to production.
Deploy with One Click: Ship your local project directly to the cloud as a fully managed Foundry Agent Service.

Inside My Workshop: Lab 02 Multi-Agent Workflow

To show you exactly how to make the most of this toolkit, I built a practical, real-world scenario in Lab 02: Multi-Agent Workflow, which you can find inside the open-source repository.

In this lab, I walk you through building a "Resume → Job Fit Evaluator" pipeline. Instead of relying on a single prompt, the workflow orchestrates a squad of four specialized AI agents working together using smart design patterns:

1. The Fan-Out Pattern (Parallel Work)

When a resume is submitted, the system splits the task and feeds it to two agents at the exact same time:

The Tech Stack Analyst: Focuses entirely on programming languages, frameworks, and tools.
The Experience & Impact Scorer: Evaluates career history, performance metrics, and leadership.

2. The Fan-In Pattern (Consolidation)

Once the parallel analysis is complete, the workflow channels their notes into the final two agents to consolidate the data:

The Fit Evaluator: Synthesizes the information into a single compatibility score.
The Roadmap Architect: Generates a custom step-by-step learning path to help the candidate bridge any skill gaps.

The Development Workflow: Step-by-Step

The lab I put together provides a straightforward walkthrough that takes you from an empty directory to a live cloud application without ever leaving your code editor.

Step 1: Click to Build Your Workspace

You don't need to write complicated setup code from scratch. By using the Agent Builder interface inside the VS Code extension, you can click a button to automatically generate all the starter configurations and project folders I've mapped out for you.

Step 2: Give the Agents Their Jobs

Next, you customize what each agent does using a simple configuration file (agent.yaml). This is where you tell the agents which AI models to use from the Foundry Model Catalog. You then add your project keys to connect to your cloud workspace:

FOUNDRY_PROJECT_ENDPOINT=https://<your-workspace>.services.ai.azure.com/api/projects/<your-project> AZURE_AI_MODEL_DEPLsOYMENT_NAME=<your-ai-model>

Step 3: Test and Fix on Your Computer

Before sharing your project with the world, you can run the whole AI team locally on your machine. Using a tool called the Agent Inspector, you can see exactly how the agents talk to each other, trace data steps, and tweak your text instructions until they work perfectly.

Step 4: One-Click Cloud Launch

When your local tests work great, you don't need to be a server or container expert to deploy them. The VS Code extension automatically packages your code and registers it with the Foundry Agent Service as a live Hosted Agent.

Step 5: Test the Live App

Once your AI team is live in the cloud, you can open the built-in Remote Agent Playground web page. Drag and drop a real resume into the window, hit run, and watch the streaming logs show your cloud agents processing the data together in real-time.

Ready to Build Your Own AI Team?

Building reliable AI tools means moving away from massive, unpredictable prompts and moving toward small, organized teams of agents using standard patterns like Fan-Out and Fan-In. The combination of the Foundry Toolkit and the structured labs I've created makes it easy to build, test, and host these systems without worrying about server infrastructure.

The entire workshop is free, open-source, and ready for you to clone today. Jump into Lab 02 and see how easy multi-agent orchestration can be!

Get Started Now: Explore my lab repository at Foundry_Toolkit_for_VSCode_Lab/workshop/lab02-multi-agent at main · microsoft-foundry/Foundry_Toolkit_for_VSCode_Lab

Let's Connect!

If you enjoyed this walkthrough, have questions about the workshop, or want to share your own agent workflows, let's keep the conversation going:

Portfolio

Build and Deploy Your First AI Agent with Microsoft Foundry Toolkit

bethanyjep — Mon, 13 Jul 2026 07:53:38 GMT

Click here for the full lab.

What if every incident report, pipeline failure, or security alert could be instantly rewritten for a non-technical audience? In this post, you'll build an "Explain Like I'm an Executive" agent, a hosted AI agent that takes dense technical input and produces plain-English executive summaries, using the Microsoft Foundry Toolkit for VS Code.

By the end, you'll have a working agent you can test locally and deploy to the cloud with a single click.

What You'll Build

The agent follows a simple flow:

Under the hood it uses:

Microsoft Agent Framework - for agent logic and structure
Foundry Toolkit for VS Code - to scaffold, test, and deploy
An AI model (e.g., gpt-4.1-mini) - to generate the summaries

The final output always follows a structured format:

Executive Summary:

What happened: (plain-language description)
Business impact: (clear, non-technical impact)
Next step: (action or mitigation)
Date: (YYYY-MM-DD)

Foundry Toolkit

The Foundry Toolkit handles the end-to-end lifecycle:

Scaffold - A wizard generates your project structure with one command
Configure - Point to your AI model and write agent instructions (a system prompt)
Test locally - Press F5 to launch a built-in chat UI for instant feedback
Deploy - One click packages your agent as a container and deploys it to Azure
Verify - Test safety and edge cases against the live endpoint

The key building blocks:

Component	Role
Agent Framework	Provides the agent runtime and tool-calling capabilities
Foundry Toolkit	VS Code extension for scaffolding, debugging, and deployment
AI Model (e.g., gpt-4.1-mini)	Powers the natural language understanding and generation
Azure Container Registry + Foundry	Hosts and scales your agent in the cloud

What Makes a Good Agent

The difference between a basic chatbot and a reliable agent comes down to well-crafted instructions. For this agent, the system prompt defines:

Role - "Translate technical information into executive-friendly summaries"
Audience - Senior leaders who care about impact, not implementation details
Output format - A consistent structure (what happened, business impact, next step)
Rules - Keep it brief, don't fabricate, don't leak instructions
Safety constraints - Resist prompt injection and role override attempts

You can also give agents tools, Python functions they can call at runtime (like fetching today's date or querying an API).

The Developer Experience

The Foundry Toolkit streamlines what would otherwise be a multi-step DevOps process:

Without the toolkit	With the toolkit
Manually write Dockerfiles, YAML configs, server boilerplate	Wizard generates everything
Set up a separate test harness	Built-in Agent Inspector (chat UI) launches with F5
Build container, push to registry, register agent manually	Single "Deploy" button handles it all
Monitor via Azure Portal	Status, logs, and playground available in VS Code

Deployment takes 2–5 minutes. Your agent gets a managed /responses endpoint with auto-scaling.

Cleanup

If you want to remove Azure resources after experimenting:

az group delete --name <your-resource-group> --yes --no-wait

Or delete just the hosted agent from the Foundry portal under Build → Agents.

Next Steps

Explore the Foundry Toolkit Lab, to build your own agent.
Check out Lab 02: Multi-Agent Orchestration to learn how to build a multi-agent workflow with orchestration patterns
Add tools - Connect APIs, databases, or custom functions
Microsoft Foundry docs - Full platform reference

From Game to Operations: Exporting a Foundry-Designed Workforce as a Portable Bundle

Princeps — Thu, 02 Jul 2026 08:25:44 GMT

Part 5 of 5.

Across this series the game has reasoned about a founder, designed a digital workforce for their venture, executed chapters through Microsoft Foundry agents, and gated every artifact behind a human. That is a complete loop - inside the game. This final post is about the seam where it leaves the game: how the Org Designer's output becomes a portable artifact a real digital-worker platform can ingest and provision, without dragging a single proprietary dependency into the public repo.

The most useful thing a reasoning agent can produce is not a paragraph. It is a structured plan another system can execute - behind its own human gate.

Why the Org Designer is the keystone worker

In Part 1 the pipeline's second stage was the Org Designer: given a venture, it designs the smallest org that can actually deliver it - one human operator plus the digital workers that form the execution layer. Every other system in the game hangs off that output. The World Designer decomposes the venture into chapters; bind_world_to_org stamps each chapter with its owning worker; the economy prices each worker's monthly burn. The org the model designs is the org that executes.

You can see the artifact directly in play - a worker delivers a positioning brief and a rendered org chart, and it passes the gate:

A worker delivers a rendered org chart that passes the verification gate

That org chart is not decoration. It is structured state - roles, kinds (human / digital_worker / hybrid), mandates, reporting lines, KPIs, tool wishes, and a model-class hint per worker. Which means it can be exported.

The org being exported: OrgRole and OrgBlueprint

To understand the bundle you have to see the state it is built from. The org is an OrgBlueprint in submission/state/schema.py - a company summary, an operating model, and a list of OrgRole seats. Each role is a typed record, not a sentence:

# submission/state/schema.py
class OrgRole(BaseModel):
    id: str
    title: str
    kind: str = "digital_worker"   # human | digital_worker | hybrid
    mandate: str = ""              # what this seat is accountable for
    reports_to: Optional[str] = None
    kpis: List[str] = Field(default_factory=list)
    tools: List[str] = Field(default_factory=list)
    deployment_hint: str = ""      # which model class fits this worker
    lifecycle_stage: str = ""      # discovery|positioning|mvp|gtm|retention|ops
    monthly_cost_usd: int = 0      # the worker's monthly wage (the burn)
    why: str = ""                  # why this seat must exist

Because every field is structured, the converter has something real to map - it is not parsing prose back into fields, it is re-shaping records it already trusts. kind becomes the human/digital split; deployment_hint becomes the portable model_class; reports_to becomes the org chart's edges; mandate, kpis, and tools become the worker's brief. The blueprint also carries derived economics (monthly_burn_usd, leverage_ratio, monthly_savings_usd) that ride into the bundle unchanged.

How the org gets designed

That blueprint is itself the output of a Foundry agent. design_org in submission/agents/org_designer.py runs on the org-designer deployment (a stronger model, because it shapes the whole run), asks for a structured roster, and parses it through the same tolerant _extract_json from Part 2 - falling back to a deterministic blueprint when there are no credentials:

# submission/agents/org_designer.py
def design_org(brief, source="pitch", source_ref="", summary_hint=""):
    client = get_foundry_client()
    deployment = model_for("orgdesigner") or model_for("narrator")
    if not (client and deployment and is_live()):
        return _finalize(_fallback_blueprint(brief), brief, source, source_ref, summary_hint)
    resp = create_chat_completion(deployment, [...], max_completion_tokens=8000)
    parsed = _extract_json(resp.choices[0].message.content or "")
    if parsed and len(parsed.get("roles", [])) >= 2:
        return _finalize(parsed, brief, source, source_ref, summary_hint)
    return _finalize(_fallback_blueprint(brief), brief, source, source_ref, summary_hint)

_finalize is where the model's creative roster meets deterministic mechanics: it normalises every role, prices each seat, and computes the headline stats. The model decides who the company needs; the code decides what they cost. That division - which we have now seen at the gate (Part 2), in consequences (Part 3), and here - is the spine of the whole project, and it is why an exported bundle's economics are trustworthy even though a model drew the org.

The org binds to the quest

The designed org does not sit in a corner - it is stamped onto the quest line. bind_world_to_org in worker_factory.py walks the stages and assigns each one the digital worker whose lifecycle stage matches, so every chapter has an owning worker before play begins:

# submission/agents/worker_factory.py
def bind_world_to_org(world, org):
    bindings = {}
    for stage in world.stages:
        worker = _match_worker_for_stage(stage, org)
        if worker:
            stage.assigned_worker_id = worker.id
            stage.assigned_worker_title = worker.title
            bindings[stage.id] = worker.title
    return bindings

This is why the export is meaningful: the org in the bundle is the same org that executed the run. The worker that owns the GTM chapter in the game is the worker the bundle hands the receiving platform for GTM. Nothing is invented at export time.

The bridge: export, don't import

The temptation, when you have a game and a real platform, is to wire them together - run the platform's workers inside the game. That is the wrong move for two reasons. First, the game's reasoning core must stay 100% Microsoft Foundry (the rubric rule, and the architectural spine of this series). Second, the repo must run after git clone with no proprietary dependencies. So the bridge is export, not import: the game emits a neutral bundle; any platform ingests it.

No platform-specific code lives in the game repo. The converter is pure and offline - it turns the in-game OrgBlueprint into a platform-neutral workforce_bundle and ships a CLI that runs in simulation after a fresh clone.

The converter is pure and offline

The module that does the work, submission/tools/export_org_blueprint.py, is deliberately dependency-free: no platform SDK, no network, no credentials. It splits the chartered roles into humans and digital workers and emits one spec per worker:

# submission/tools/export_org_blueprint.py
def org_to_workforce_bundle(org):
    roles   = org.get("roles") or []
    humans  = [r for r in roles if r.get("kind") == "human"]
    workers = [r for r in roles if r.get("kind") != "human"]
    worker_specs = [{
        "worker_key": _slug(role.get("id") or role.get("title", "")),
        "name": role.get("title", "Digital Worker"),
        "kind": role.get("kind", "digital_worker"),
        "system_message": _system_message(role, org),
        "model_class": _MODEL_CLASS.get(role.get("deployment_hint", ""), "fast"),
        "tools": role.get("tools") or [], "kpis": role.get("kpis") or [],
        "reports_to": _slug(role.get("reports_to") or ""),
        # ... seniority, monthly_cost_usd, why ...
    } for role in workers]
    # ... company, economics, team, provisioning ...

Because it is pure, the same function backs both the live HTTP endpoint and a no-server CLI, and it runs in simulation after a fresh clone - the export is forkable like everything else in this series.

The bundle: a contract, not a dump

The export is a single JSON document with a stable format tag (campaign.workforce_bundle, versioned), designed so both a human approver and a downstream LLM can parse and act on it:

{
  "format": "campaign.workforce_bundle",
  "version": 1,
  "company": { "summary": "...", "operating_model": "...", "source": "pitch" },
  "economics": { "headcount": 6, "digital_worker_count": 5,
                 "human_count": 1, "monthly_burn_usd": 23625, "leverage_ratio": 5.0 },
  "team": { "name": "...", "purpose": "...", "members": ["..."], "owner": "operator" },
  "humans": [ { "id": "operator", "title": "Founder / Operator", "mandate": "...", "kpis": ["..."] } ],
  "workers": [
    {
      "worker_key": "strategy_positioning_lead",
      "name": "Strategy & Positioning Lead",
      "role": "strategist",
      "kind": "digital_worker",
      "system_message": "<worker_identity>...generated starter prompt...</worker_identity>",
      "model_class": "reasoning",
      "tools": ["..."], "kpis": ["..."], "reports_to": "operator"
    }
  ],
  "org_chart_mermaid": "graph TD\n  ...",
  "provisioning": {
    "status": "pending_human_approval",
    "note": "Nothing in this bundle is provisioned. The receiving platform must
             present it for explicit human approval before creating anything."
  }
}

Three design choices make it portable:

A generated system_message per worker, same shape for every role - so a receiving platform (or its own org-architect agent) can parse, refine, and version it instead of re-deriving prompts.
A model_class hint (reasoning / fast / creative), not a hard model name - the receiver maps it onto its own fleet.
A provisioning.status of pending_human_approval baked into the artifact itself. The bundle carries its own gate: the game's verification-gate pattern (Part 2), extended across the system boundary.

Every worker carries its own starter prompt

The single most portable field is system_message. Every worker - whatever its role - gets a generated starter prompt in the same XML shape, so a receiving platform (or its own org-architect agent) can parse, refine, and version it instead of re-deriving prompts from scratch:

# submission/tools/export_org_blueprint.py - _system_message
return (
    f"<worker_identity>\n"
    f"  <role>{role.get('title')}</role>\n"
    f"  <mission>{role.get('mandate', '')}</mission>\n"
    f"  <reports_to>{reports_to}</reports_to>\n"
    f"  <kpis>{kpis}</kpis>\n"
    f"  <guardrails>\n"
    f"    - Work only within this mandate; escalate judgment calls to {reports_to}.\n"
    f"    - No legal or financial commitments without explicit human approval.\n"
    f"  </guardrails>\n"
    f"</worker_identity>"
)

Those guardrails are not decoration. Every exported worker ships with "no legal or financial commitments without explicit human approval" baked into its own identity - the human gate is written into the worker's prompt, not just the bundle's status field.

The org chart travels as text

There is a fitting symmetry with this whole series, which has leaned on Mermaid for every diagram: the bundle ships the org chart as Mermaid source, generated by org_to_mermaid. A receiver can render it, diff it, or hand it to a model - it is text, not a picture:

# submission/tools/export_org_blueprint.py - org_to_mermaid
lines = ["graph TD"]
lines.append("  classDef human fill:#f6d55c,stroke:#5b8cff,stroke-width:3px")
lines.append("  classDef digital fill:#3caea3,stroke:#2dd4bf")
for role in roles:
    cls = "human" if role["kind"] == "human" else "digital"
    lines.append(f'  n_{_slug(role["id"])}["{title}"]:::{cls}')
for role in roles:                       # reporting lines
    if role.get("reports_to"):
        lines.append(f"  n_{_slug(role['reports_to'])} --> n_{_slug(role['id'])}")

Humans and digital workers get different CSS classes, so the rendered chart shows at a glance who is a person and who is a digital worker - the same human/digital distinction the economics priced, carried into the picture.

The economics travel with the org

A workforce plan that does not say what it costs is a wish list. The bundle carries an economics block - headcount, digital-worker count, human count, monthly burn, and a leverage ratio - computed by the game's worker_economics model when the org was designed, not invented at export time:

"economics": {
  "headcount": 6, "digital_worker_count": 5, "human_count": 1,
  "monthly_burn_usd": 23625, "monthly_inference_usd": 1200, "leverage_ratio": 5.0
}

Each worker also carries its own monthly_cost_usd, so a receiving platform can show a real budget before it provisions anything - one human operator leveraging five digital workers at a burn a solo founder could actually carry. That leverage ratio (digital workers per human) is the headline number the whole game is about: how much execution one person can direct. Exporting it means the off-ramp inherits the argument, not just the boxes and arrows.

Pricing the workforce

Those numbers are not vibes. submission/agents/worker_economics.py prices each seat from the human salary it replaces - a deliberate game-balance fraction, so burn is a constraint the player actually feels rather than a rounding error:

# submission/agents/worker_economics.py
WORKER_COST_FRACTION_OF_HUMAN = 0.75   # a worker costs 3/4 of the human it replaces

def worker_cost_from_human(human_median_usd):
    return int(round(max(0, human_median_usd) * WORKER_COST_FRACTION_OF_HUMAN))

Each worker's monthly_cost_usd is three quarters of the present-world salary for that seat; the much cheaper real compute (inference_usd) is tracked separately for the efficiency story. Sum the workers and you get monthly_burn_usd; divide digital workers by human operators and you get leverage_ratio. When a receiving platform reads the bundle, those are the figures it shows before provisioning - a real budget for one operator running a digital team, derived the same way whether the org was designed on Foundry or in simulation.

The export endpoint and the button

The bridge is one endpoint and one button. GET /api/org/export reads the chartered org from state, converts it, logs an ORG_EXPORTED event to the replay log (Part 4), and returns the bundle as a download. The Digital Workforce panel in the UI has an Export workforce bundle button that calls it - so "you played the game; now your org is a deployable workforce" is one click.

@app.get("/api/org/export")
def export_org():
    state = store.load()
    if not state.org or not state.org.roles:
        raise HTTPException(status_code=404, detail="No chartered org to export.")
    bundle = org_to_workforce_bundle(state.org.model_dump())
    store.log_event("ORG_EXPORTED", "org_designer",
                    f"Exported {len(bundle['workers'])} workers, pending approval.")
    return JSONResponse(content=bundle,
        headers={"Content-Disposition": 'attachment; filename="workforce_bundle.json"'})

Because the converter is dependency-free, it also runs from the command line with no server and no credentials:

python3 submission/tools/export_org_blueprint.py \
  --pitch "Low-cost 3D-printed solar cells for off-grid homes"
# -> workforce_bundle.json : 5 digital workers + 1 human, pending approval

The CLI: same converter, no server

Because the converter is pure, the same org_to_workforce_bundle function powers a no-server CLI - export_org_blueprint.py - which can either read a saved game state or design a fresh org from a pitch and export it, all in simulation, no credentials:

# submission/tools/export_org_blueprint.py - _main
if args.pitch:
    from agents.org_designer import design_org
    org = design_org(args.pitch, source="pitch", source_ref=args.pitch)
else:
    org = json.load(open(state_path)).get("org") or {}   # from a played game
bundle = org_to_workforce_bundle(org)
print(json.dumps(bundle, indent=2))

That one entry point is the whole forkability story for the bridge: --pitch "..." designs an org and prints a deployable workforce bundle on a fresh clone, with no Azure, no server, and no platform SDK anywhere in the path. The endpoint and the CLI share the exact same converter, so what you can click in the live app you can also pipe in a shell.

Best practice: the receiver re-gates, the bundle is a draft

A portable bundle is an input, not an order. On the receiving side, the right pattern is an org-architect worker that treats the bundle as a draft blueprint: it maps each entry's model_class onto the platform's current model tiers, matches incoming worker mandates against existing templates (reuse before invent), re-renders the org chart in its own conventions, fills gaps the game did not model (departments, OKR cascades), and then presents the result for explicit human approval - exactly the CEO-gate pattern, one system later.

That symmetry is the point. The game gates artifacts behind a human; the bundle declares itself pending; the receiver gates provisioning behind a human. The human-at-the-root law (Part 2) survives the trip across the boundary.

Why this matters beyond the game

A reasoning agent that only produces prose is a better chatbot. A reasoning agent that produces a structured, portable, gated plan another system can execute is the beginning of an operating loop. The game is a teaching tool and a playable argument; the bundle is the off-ramp where the argument becomes useful - the designed workforce stops being a screen and becomes something a platform can stand up, behind its own approval gate.

Responsible AI

The bundle is inert by construction. Its provisioning.status is pending_human_approval and its note says, in the artifact itself, that nothing is provisioned until a human signs off. Nothing in it is executable on import; it is a description, not a command. Combined with the receiver re-gating provisioning, that means no chain of automated steps can stand up a workforce without a person in the loop - the same reliability story this whole series is built on, carried one boundary further.

The whole series, in one artifact

It is worth seeing the bundle as the place every earlier post converges. The org inside it was decomposed from a pitch (Part 1) and designed on a Foundry model. Every artifact that filled it out passed a human verification gate (Part 2). The CEO decisions that shaped which workers exist are the binding memory the workforce learned from (Part 3). The whole thing was produced by a stack that runs local-first and fully traced, so the ORG_EXPORTED event sits in the same replay log as everything that led to it (Part 4). And now it leaves the game as a portable, gated plan (Part 5). The bundle is not a feature bolted on at the end; it is the receipt for the entire loop.

Where this lives in the repo

Concern	File	Key symbol
Pure offline converter + CLI	`submission/tools/export_org_blueprint.py`	`org_to_workforce_bundle`, `_system_message`, `org_to_mermaid`
Export endpoint + replay event	`submission/tools/server.py`	`GET /api/org/export`, `ORG_EXPORTED`
The org being exported	`submission/agents/org_designer.py`	`design_org`, `OrgBlueprint`

Try it

Charter an org in the game, then export it:

Play the live app -> design your workforce -> Export workforce bundle.

Or from the CLI, no credentials:

git clone https://github.com/princepspolycap/agentsleague-afterbuild
cd agentsleague-afterbuild && python3 -m venv .venv && source .venv/bin/activate
pip install -r submission/requirements.txt
python3 submission/tools/export_org_blueprint.py --pitch "Your idea here"

Key takeaways

The most useful agent output is a structured plan another system can execute, not prose.
Bridge by export, not import: emit a neutral, versioned bundle; keep proprietary platforms out of the reasoning repo.
Carry the human gate across the boundary - the bundle declares itself pending_human_approval, and the receiver re-gates provisioning.
Use a model_class hint, not a hard model name, so the receiver maps onto its own fleet.
One endpoint, one button: "you played the game; now your org is a deployable workforce."

The dungeon is open, the workforce is designed, and now it is portable. A mission too big to command gets aligned one company at a time - and the org you charter in a browser game can walk out as a plan a real platform will run, with a human still holding the seal.

Part 5 of 5. Start over at Part 1, or clone the repo and pitch your own company.

Play it: worldforge-game...azurecontainerapps.io
Code: github.com/princepspolycap/agentsleague-afterbuild
Live battle replay: Agents League - Reasoning Agents

GitHub Copilot App Canvas Is a Runtime

Lee_Stott — Mon, 29 Jun 2026 07:00:00 GMT

There is a quiet shift happening in how we build software with AI. We are moving from writing static code to orchestrating living systems where developers and AI agents co-create, observe, and evolve a solution in real time. This post is a working theory of what GitHub Copilot App Canvas is actually for, grounded in a real, runnable demo you can clone today: leestott/agent-runtime-canvas.

The Agent Runtime canvas open beside the chat — control bar, activity spotlight, requirement & constraints, and the live agent roster.

The headline claim, which the rest of this post defends with code:

Traditional UIs are for using software. Canvas is for shaping software while it runs.

1. The misconception worth getting out of the way

The first instinct most engineers have when they see Canvas is to build a UI with it a dashboard, a DevOps board, an admin panel. That is the wrong mental model, and it leads to disappointment. A Kanban board rendered in Canvas is just a worse version of a tool that already exists.

Canvas is not where your users live. It is where your system becomes visible to you and to the AI while you are still figuring it out. The distinction matters:

You don't build Canvas instead of your UI. You use Canvas to figure out, test, and evolve the UI and the system before and during building it.
Canvas solves problems your final UI should never try to solve in a visible way agent coordination, intermediate state, test validation, failure propagation. These are observability concerns, not end-user features.
Canvas is intended for test validation and the implementation of agent-driven solutions not for shipping a production control panel.

A useful analogy: Figma is Human-to-Human one person designs a static artifact for another person to read. Canvas is Human-to-AI-to-System a shared surface where a human, an AI agent, and a running system all act on the same live model. Figma shows you a picture of the software. Canvas is a runtime where things actually execute.

2. The positioning, stated plainly

Here is the thesis the demo is built to prove:

Canvas redefines software development by shifting from writing static code to orchestrating living systems, where developers and AI co-create, observe, and evolve solutions in real time. Instead of building UIs for users, we build interactive environments for agents — turning debugging, testing, and execution into a continuous, visual feedback loop that accelerates innovation and brings ideas to production faster than ever.

Read that again with the demo in mind, because the demo is not a slide, it is a working Copilot CLI extension that renders exactly this loop.

3. What we built: the Agent Runtime canvas

agent-runtime-canvas is a GitHub Copilot CLI canvas extension called Agent Runtime. It turns Canvas into a runtime observability and control plane for a multi-agent software system that is being designed, tested, and evolved in real time.

The canvas renders a single living SystemModel that both humans and the AI agent edit at the same time. The agent drives it through five canvas actions; the human drives it through panel controls. Every change streams to the iframe over Server-Sent Events (SSE), so the system visibly evolves through interaction.

The seven panels: a system you can watch think

Panel	What it makes observable
Requirement & constraints	The feature under design plus editable policies and constraints
Agents	Active agents, their responsibilities, and live state (idle / working / done / error / blocked)
Task Flow	The dependency graph of tasks across agents, with live status
Artifacts	The intermediate outputs each task emits
Validation	Test cases, pass/fail, expected vs. actual, and the reasoning behind each verdict
Live State	The shared memory objects the agents read and write — directly human-editable
Timeline	A change-over-time log, including before→after state diffs

None of these are things you would put in front of an end user. All of them are things you desperately want to see while you and an AI are co-designing an agentic system.

The five agent actions

The AI co-creates and evolves the system by calling five actions, declared in the canvas extension:

Action	Effect
`decompose_system`	Break a requirement into collaborating agents + a task-flow graph
`execute_workflow`	Coordinate agents to advance tasks (`step` / `run` / `pause` / `resume` / `reset`)
`validate_output`	Run evaluation tests, return structured pass/fail + reasoning
`update_system_design`	Modify architecture/logic: requirement, constraints, agents, tasks
`track_state`	Persist/update a shared state object, recording the diff on the timeline

The critical detail is that human controls and agent actions funnel through the exact same store. There is no separate "AI view" and "human view" — one model, two kinds of participant.

4. How it actually works (the parts that matter)

The extension is deliberately small and dependency-free. It uses only Node's built-in modules plus github/copilot-sdk, which the CLI auto-resolves. Three files do the work:

.github/extensions/agent-runtime/
  extension.mjs   # wiring: loopback HTTP server, SSE, /control, 5 canvas actions
  store.mjs       # durable SystemModel + execution engine + validation
  ui.mjs          # iframe renderer (system view, validation, state, timeline)

One shared model, broadcast on every mutation

The heart of the demo is the SystemStore. It is an EventEmitter: every mutation bumps a version, appends a timeline entry, persists to disk, and broadcasts a fresh snapshot to all connected panels. This is the single line that makes "humans and AI edit the same live system" true rather than aspirational:

// store.mjs — every change is versioned, logged, persisted, and broadcast.
_commit(eventType, summary, detail) {
  this.model.version += 1;
  this.model.updatedAt = now();
  if (eventType) {
    this.model.timeline.unshift({
      id: uid("ev"), ts: now(), type: eventType, summary, detail: detail || null,
    });
    this.model.timeline = this.model.timeline.slice(0, 200);
  }
  this._queueSave();          // best-effort JSON persistence under ~/.copilot
  this.emit("change", this.model);  // fan out to every SSE client
  return this.model;
}

The agent action and the human button hit the same method

In extension.mjs, the canvas action handler and the iframe's /control POST both call store.execute(...). That symmetry is the whole point — neither the human nor the AI is privileged:

// extension.mjs — a human control POST maps onto the same store method
// the AI agent calls through the execute_workflow canvas action.
function applyControl(store, body) {
  switch (body.action) {
    case "execute":  return store.execute(body.mode || "step", body);
    case "validate": return store.validate(body.tests);
    case "decompose":return store.decompose(body.requirement, body);
    case "inject_failure": return store.injectFailure(body.taskKey);
    case "edit_state":     return store.editState(body.key, body.value);
    // ...requirement, constraints, clear_failures, update_design
  }
}

Execution you can watch one task at a time

The engine advances the task graph through a visible begin→dwell→finish lifecycle so the active agent is always observable. A ready task is one whose dependencies are all done:

// store.mjs — the scheduler only starts a task when its deps are satisfied.
_readyTask() {
  return this.model.tasks.find(
    (t) =>
      t.status === "pending" &&
      t.deps.every((d) => {
        const dep = this.model.tasks.find((x) => x.id === d);
        return dep && dep.status === "done";
      }),
  );
}

When a task finishes, its agent emits an artifact and writes to shared state; when a dependency fails, the engine walks the graph to a fixpoint and marks every downstream task blocked. That is failure propagation you can see — exactly the kind of thing a production UI would (correctly) hide, and exactly the kind of thing you need exposed while designing the system.

Validation as a first-class, re-runnable citizen

The default evaluation suite asserts properties of the running system, not of static code — every test returns an expected value, an actual value, and a human -readable reason:

// store.mjs — tests assert properties of the live system model.
_defaultTests() {
  const t = (name, target, assertion) => ({ id: uid("test"), name, target, assertion });
  return [
    t("All tasks reach a terminal state", "tasks", "no_pending"),
    t("No tasks failed", "tasks", "none_failed"),
    t("Every completed task emitted an artifact", "artifacts", "artifact_per_done"),
    t("Design state populated before build", "state", "design_before_build"),
    t("Decision recorded by Reviewer", "state", "has_decision"),
  ];
}

This is the "continuous, visual feedback loop" from the thesis, made concrete: decompose → execute → validate → redesign → re-validate, with the Timeline recording every before→after transition.

5. Run it yourself

You need a GitHub Copilot CLI / app with canvas support (the canvas-renderer capability) and this repo opened as your workspace. There is no npm install the SDK is auto-resolved and the extension uses only built-in Node modules.

Clone and open the workspace.
```
git clone https://github.com/leestott/agent-runtime-canvas.git
cd agent-runtime-canvas
```
The extension auto-discovers from .github/extensions/agent-runtime/.
Open the canvas with a requirement. Ask Copilot:
Open the Agent Runtime canvas with the requirement "Add CSV export to the reports page".
Walk the loop. Decompose into five agents and a six-task graph, press Run ▶, watch the spotlight track the active agent, press Run tests ✓ for 5/5 green, then Inject failure ⚡ to watch downstream tasks go blocked and validation drop to 4/5 — and recover.

State persists per documentId under ~/.copilot/extensions/agent-runtime/artifacts/, so a reload resumes exactly where you left off. The companion demoscript.md in the repo gives you a tight, timed walkthrough.

6. Why this is an observability story

Once you accept that Canvas is a runtime rather than a UI, the most compelling use case becomes observability of agentic systems. Agentic software is notoriously hard to debug: the interesting behavior lives in intermediate state, coordination order, and the moments where one agent's failure cascades into another's. A production UI is designed to hide all of that. A Canvas is designed to surface it, temporarily, while you are shaping the system — and then get out of the way.

This reframes Canvas alongside the broader Microsoft and GitHub agent tooling story. As teams adopt the GitHub Copilot SDK and patterns like the open Model Context Protocol to wire agents into real systems, the gap is rarely "can the agent act?" it is "can a human see what the agent did, judge it, and steer it?" Canvas is a candidate answer to that second question. When you take agents toward production on Azure with services like Microsoft Foundry, the same instinct applies: build the evaluation and observability loop first, and let it shape the system before you commit a single end-user pixel.

7. The open question: why can't Canvas be multi-user?

There is an obvious next frontier, and it is worth stating as an honest open question rather than a finished feature. Everything that makes Canvas valuable also makes it a natural collaborative surface:

It is a shared space.
It is visual.
It is collaborative.
Multiple participants — human and AI — interact with the same surface.

If Figma earned its place by making Human-to-Human design multiplayer, the provocative question is whether a project- or repo-scoped Canvas can make Human-to-AI-to-System development multiplayer too: several engineers and several agents shaping one running system on one surface. The demo here is single-user by design, but its architecture — one shared store, versioned, broadcast to every subscriber — is already the shape you would need. That is a genuine research direction, and worth experimenting with as licensing and access broaden.

8. Honest limitations

In the spirit of building credibility rather than hype:

This is a demonstration. The decomposition, artifacts, and state are synthesized to make the runtime loop legible — it models an agentic system rather than running arbitrary production agents.
It is single-user and single-machine. The loopback HTTP server and per-document store are local by design; multi-user is an aspiration, not a shipped capability.
Access is gated. Canvas support requires a Copilot CLI/app build with the canvas-renderer capability. Licensing and preview access are the biggest practical blockers to wider experimentation today.
Persistence is best-effort. State is written to a local JSON artifact; treat it as demo durability, not a database.

Key takeaways

Don't build a UI in Canvas. Use Canvas to shape, test, and evolve a system — and the UI — while it runs.
Traditional UIs are for using software; Canvas is for shaping software while it runs.
Canvas is Human-to-AI-to-System, a runtime where things execute — not a static design surface.
Its strongest use case is observability and validation of agentic systems: surface the intermediate state your production UI should hide.
The shared-model architecture — one versioned store broadcast to every participant — is what makes human + AI co-editing real, and what hints at a multi-user future.

Next steps

Clone and run the demo: github.com/leestott/agent-runtime-canvas.
Read the extension source under .github/extensions/agent-runtime/ — start with store.mjs.
Explore the building blocks: the GitHub Copilot SDK, the Model Context Protocol, and Microsoft Foundry for taking agentic systems toward production.
Try the multi-user thought experiment: fork the store, add a second subscriber, and ask what changes when two humans and two agents share one surface.

Master the Command Line with GitHub Copilot CLI:

Lee_Stott — Sun, 28 Jun 2026 11:26:35 GMT

If you are a student aiming to become an AI engineer or a software developer, the terminal is about to become your most powerful classroom. https://github.com/features/copilot/cli/ brings an AI agent directly into your command line, and its slash commands (typed as /something) are the shortcuts that unlock its real capabilities.

The problem most students hit is simple: they install a powerful tool and then only ever use 10% of it. They type questions, get answers, and never discover the commands that turn Copilot CLI from a chatbot into a genuine pair programmer. This post fixes that. We will walk through the most useful slash commands, explain why you would reach for each one, and give you concrete student scenarios for every command.

Why This Matters Now

AI-assisted development is no longer optional in the industry. Employers increasingly expect graduates to be fluent with AI developer tools, not just programming languages. Learning the Copilot CLI slash commands early gives you two advantages:

Speed: You spend less time context-switching between docs, terminal, and editor.
Good habits: Commands like code review and security review teach you professional workflows while you are still learning.

Everything below is grounded in the actual command set shipped in Copilot CLI. To see the full, current list at any time, just type /help inside the CLI.

How to Run a Slash Command

Slash commands are typed at the Copilot CLI prompt. Start a command with a forward slash and the CLI shows you an autocomplete menu:

# Launch the CLI
copilot

# Then, at the prompt, type a slash to browse commands
/

# Or jump straight to one
/model
/plan
/review

A few related shortcuts are worth memorising on day one:

? — show quick help
@ — mention files so Copilot reads them
# — mention GitHub issues and pull requests
! — execute a raw shell command without leaving the prompt

The Most Useful Slash Commands for Students

The table below groups the highest-value commands by the job you are trying to do. Each row includes a realistic student scenario so you know exactly when to reach for it.

Learning and Planning

Command	What it does	Student scenario: why use it
`/plan`	Creates an implementation plan before any code is written.	You have a coursework project ("build a sentiment classifier") but no idea where to start. Run `/plan` to get a step-by-step roadmap you can follow and learn from, instead of diving in blind.
`/research`	Runs a deep research investigation using GitHub search and web sources.	For a dissertation or capstone, you need to compare approaches (e.g. "vector databases for RAG"). Use `/research` to gather grounded, cited findings rather than guessing.
`/ask`	Asks a quick side question without adding it to the conversation history.	Mid-project you forget what a Python decorator does. Ask with `/ask` so your main task context stays clean and focused.
`/model`	Selects which AI model to use (or `auto` to let Copilot pick).	A simple formatting fix needs a fast model; a tricky algorithm needs a stronger one. Learn to match the model to the task — a real engineering skill.

Writing and Reviewing Code

Command	What it does	Student scenario: why use it
`/diff`	Reviews the changes made in the current directory.	Before submitting an assignment, run `/diff` to see exactly what changed — catch that debug `print()` you forgot to remove.
`/review`	Runs a code review agent to analyse your changes.	No teaching assistant available at 2am? `/review` gives you professional-style feedback on bugs and logic errors so you learn before the deadline, not after grading.
`/security-review`	Analyses staged and unstaged changes for security vulnerabilities.	Building a web app for a module? Run `/security-review` to spot issues like injection flaws — and start building the security mindset employers want.
`/pr`	Operates on pull requests for the current branch.	Contributing to a group project or open source? Use `/pr` to manage pull requests and learn the collaboration workflow used in every real engineering team.
`/ide`	Connects Copilot to an IDE workspace.	You prefer working in VS Code. Connect with `/ide` so Copilot understands your open files and editor context.

Managing Your Work Session

Command	What it does	Student scenario: why use it
`/resume`	Switches to a different saved session.	You worked on a lab yesterday and want to continue today. `/resume` brings back the full context instead of starting from scratch.
`/context`	Shows context-window token usage and a visualization.	Copilot seems to be "forgetting" earlier details. Check `/context` to understand how much conversation history fits — a core concept for any aspiring AI engineer.
`/compact`	Summarises conversation history to reduce context usage.	Long debugging session running out of context? `/compact` condenses it so you can keep going without losing the thread.
`/undo` / `/rewind`	Rewinds the last turn and reverts file changes.	Copilot made an edit that broke your tests. `/undo` safely rolls it back so you can experiment fearlessly.
`/usage`	Displays session usage metrics and statistics.	Curious how much you are relying on the AI? `/usage` helps you stay aware of your consumption and learning balance.

Setting Up and Extending the Environment

Command	What it does	Student scenario: why use it
`/init`	Initialises Copilot instructions for the current repository.	Starting a new project repo? `/init` sets up custom instructions so Copilot follows your project's conventions consistently.
`/mcp`	Manages Model Context Protocol (MCP) server configuration.	Want Copilot to query a database or external tool? `/mcp` connects MCP servers — a cutting-edge skill for AI engineering portfolios.
`/agent`	Browses and selects specialised agents.	Different tasks suit different agents. `/agent` lets you pick the right specialist for the job.
`/memory`	Shows memory status, or enables/disables memory across sessions.	Want Copilot to remember your preferences (e.g. "I use Python type hints")? Manage that with `/memory`.

A Realistic Student Workflow, End to End

Here is how these commands fit together for a typical assignment — building a small machine learning script. Notice how the commands chain into a professional development loop:

# 1. Plan the work before touching code
/plan

# 2. Pick an appropriate model for the task
/model

# 3. Let Copilot reference your data file
@data/train.csv

# 4. After Copilot writes code, see what changed
/diff

# 5. Get an automated code review
/review

# 6. Check for security issues before you submit
/security-review

# 7. If an edit broke something, roll it back
/undo

This loop —> plan, build, review, secure, iterate, is exactly the cycle used by professional engineering teams. By practising it now with Copilot CLI, you are rehearsing the workflow you will use in your first job.

Responsible Use: Learn With AI, Not Instead Of It

A quick but important note for students. AI assistance is a learning accelerator, not a replacement for understanding. Keep these principles in mind:

Read the explanations, not just the code. Use /ask and /review to understand why something works.
Check your institution's policy. Many courses have rules about AI use in assessed work, make sure you comply and cite appropriately.
Never paste secrets. Keep API keys, passwords, and personal data out of prompts.
Verify before you trust. Run the code, read the security review, and confirm claims against official documentation.

Key Takeaways

Slash commands turn Copilot CLI from a Q&A box into a full development partner.
Start with /plan, /diff, /review, and /security-review they build professional habits immediately.
Use /model, /context, and /compact to understand how AI systems actually work under the hood.
Type /help any time to see the complete, current command list for your version.

Next Steps and Resources

Read the official guide: Use GitHub Copilot CLI
Explore the broader docs: GitHub Copilot documentation
Open the CLI and run /help to browse every command interactively.
Pick one assignment this week and run the full plan → review → security-review loop on it.

The fastest way to learn is to try. Launch Copilot CLI, type a single /, and start exploring. Your future engineering self will thank you.

Educator Developer Blog articles

Adding a Fallback Model to Hermes with Microsoft Foundry

Part 1: Deploying a Model on Microsoft Foundry

Part 2: Adding Foundry to Hermes as a Fallback

Part 3: Promoting Foundry to the Primary Model

Part 4: Testing the Switch

Part 5: The Obstacle, and What It Actually Taught Me

Command Cheat Sheet

Conclusion

Agent Governance Toolkit: Giving AI Agents Rules of the Road

Why "Trust the Prompt" Was Never a Robust Security Model

Hands-On Implementation: Enforcing Runtime Policies in Python

Strategic and Risk Governance Implications

Production-Grade Governance as the Baseline for Autonomous AI

Understanding Harnesses: The Concept That Changes How You Build Agents in Copilot Studio

What a harness is

The three harnesses

Model choice is the headline

Standard harness: explicit construction

GitHub Copilot harness: goal-directed reasoning

Topics are gone. Skills replaced them.

Agents and workflows are now two equal pillars

How to choose

Two things to know before you start

Getting started

What is still rolling out

Build AI agents without leaving VS Code, join our 3-part Reactor Series

Building agents, without leaving your IDE.

What you will Build

Explain like I am an Executive agent

The Resume → Job Fit Evaluator

The sessions

Follow along on GitHub....

Save your seat and see you there

👉 Register for the full series

CivicGrant IQ: The Five-Agent AI That Reads Federal Grants So Cities Don't Have To

The meeting that started this

What it does

The architecture

Two patterns worth stealing

Results

What I learned

Try it or build on it

ARGUS: Compliance Infrastructure That Believes Financial Access Is a Human Right

What ARGUS does

What's next: ARGUS v2

Try it yourself

Grounded AI That Won't Invent Facts: Building "Tell My Day" with Microsoft Foundry IQ

The problem, and why it is the sharpest possible test

What the app does

The Microsoft stack at a glance

How it works, step by step

1. Stand up the knowledge base (one command)

2. Retrieve grounded facts at request time

3. Ground the model with a no-invention prompt

4. Verify every quote on the server (the part that matters most)

Deploying free on Azure Static Web Apps

De-risking the build (a method worth teaching)

Accessibility is the product, not a coat of paint

Results and impact

Lessons learned

See it, or build on it

How I Built CurriculumCraft AI — A Microsoft Hackathon Winning EdTech Solution for Ghana's Teachers

A Problem I Lived as a Teacher

The Solution — CurriculumCraft AI

Architecture and Technology

Microsoft Foundry IQ Integration

GitHub Copilot — AI-Assisted Development

Implementation — Key Challenges and How I Overcame Them

Results and Impact

Lessons Learned

Future Enhancements

Conclusion

Building Archon: Where Models Stop and Financial Controls Begin

The business problem behind Archon

What Archon does today

Where the model stops

The payroll example: cost is not cash flow

From upload to reviewable evidence

Why absence is not success