Building an AI agent is getting easier. Building an agent system that knows which agent should act, what context to share, when the work is done, and where a person must approve is the harder architectural problem.
That is what multi-agent orchestration is for, and the word is borrowed from music on purpose. An orchestra is not the most instruments you can fit on a stage. It is a set of sections, each playing only its own part, following one score, under one conductor who never picks up an instrument but decides when the piece is finished. Hold that picture; every figure in this post uses it. The agents are the sections. The workflow is the score. The amber seal is the conductor.
Microsoft Foundry Agent Service is the managed platform: models, tools, identity, runtime, observability, evaluations, publishing. Microsoft Agent Framework supplies the orchestration patterns that decide how specialized agents collaborate on that platform.
The goal is not the most agents. It is the least complex architecture that completes the work reliably, securely, and economically.
Foundry is the platform, not the pattern
Keep the two separate. Foundry supplies the operational foundation:
- Models from the Foundry catalog
- Prompt agents and hosted agents on a managed runtime
- Tool access, including MCP servers and custom functions
- Microsoft Entra identity and role-based access control
- Tracing, metrics, evaluations, and Application Insights
- Agent versioning and publishing
The hall, not the score. Models, agents, monitoring, guardrails, and governance all live in one Foundry project; none of these screens decides how agents work together.
The orchestration pattern decides how agents divide, route, review, and finish the work. Agent Framework ships five: sequential, concurrent, handoff, group chat, and Magentic. You can also define custom workflows from executors, edges, routing rules, and events.
Start with one agent
Multi-agent is not the default. Microsoft recommends the lowest level of complexity that reliably meets the requirement. A direct model call may suffice for a focused transformation. A single agent with tools is the usual enterprise default for work within a single domain. Orchestration earns its place when work crosses domains, requires parallel specialization, benefits from separate security boundaries, or a single agent becomes overwhelmed by too many instructions, tools, and knowledge sources.
A separate agent is easier to justify when the capability:
- Has a distinct set of tools or knowledge
- Requires a different security boundary
- Has a reusable responsibility
- Needs to be evaluated independently
- Is maintained by a separate team
If none of those apply, another agent adds coordination cost, not value.
A single agent with a bounded set of tools and knowledge. This is the enterprise default; add everything below only when it earns its place.
1. Sequential orchestration
Work passes through agents in a fixed order; each processes the previous agent's output. Microsoft also calls this a pipeline, prompt chaining, or linear delegation. The route is deterministic, not chosen by the agents.
Use it when:
- Stages have clear dependencies and must run in order
- Each stage progressively improves the last one's output
- Routing must be predictable
- A human approval gate belongs at a defined point
For a municipal records request: one agent classifies, one extracts search criteria, one checks policy requirements, and a final agent drafts the response for human review. The workflow enforces the stages; agents perform bounded reasoning inside them.
2. Concurrent orchestration
The same request goes to several agents at once. Each works independently; results are collected, compared, or synthesized afterward. Also known as fan-out/fan-in, scatter-gather, or map-reduce. The merge can be rules, voting, weighted combination, or model-based synthesis.
Use it when:
- Work can be done independently and in parallel
- Multiple professional perspectives are needed
- There is a clear conflict-resolution strategy
- Agents do not need to update the same mutable state
In the public sector, concurrent agents should return structured findings rather than each writing to the authoritative record. A coordinating workflow validates and combines them before committing to Dataverse or another system of record.
3. Handoff orchestration
An active agent transfers control to a specialist after identifying the domain, a process known as routing, triage, transfer, dispatch, or delegation. Only one specialist handles the request at a time, with a clear exit path back to staff, not a failure.
A reliable handoff requires details such as why the transfer occurred, authorized context, ownership of the next step, final response delivery, actions if no specialist accepts, and escalation procedures. Connected agents were the traditional method, but Microsoft plans to retire them by March 31, 2027, in favor of the Agent Service and current workflow tools.
4. Group chat orchestration
Multiple agents collaborate in an ongoing conversation, with a chat manager directing who speaks next to achieve a result. This pattern resembles a roundtable or debate and suits maker-checker reviews.
The cycle involves a maker creating an artifact, checkers evaluating it against set criteria, the maker revising, and the process ending with approval or a set number of iterations to prevent endless loops. Unresolved issues are escalated to a person. Microsoft advises clear criteria, iteration limits, fallback plans, and notes that coordination becomes complex with more than three agents.
In local government, this setup is used for review and advice, where agents flag policy conflicts, accessibility issues, or missing evidence, and an authorized employee makes the final decision.
5. Magentic orchestration
For open-ended tasks where the path cannot be known up front. A manager agent builds a plan, assigns specialists, watches progress, detects stalls, and replans as it learns. Also called dynamic, adaptive, or task-ledger orchestration.
Microsoft Research's Magentic-One keeps a Task Ledger for facts and plans and a Progress Ledger for assignments and status, revising the plan when progress stalls. Do not reach for it because it is the most autonomous. It is the wrong tool for low-complexity, time-sensitive, or already-well-defined work, and its dynamic planning makes cost and behavior harder to predict.
Why hybrid usually wins
Most local government processes mix deterministic and nondeterministic work. A permit review needs agents to interpret documents, extract conditions, search policy, and explain findings. It also has mandatory review stages, approval authorities, notification requirements, and retention obligations that should never depend on agent discretion.
The finding crosses the boundary as data, not as authority.
- Foundry agents interpret, classify, research, and recommend
- Agent Framework coordinates the reasoning
- MCP or controlled tools expose bounded data and actions
- Dataverse holds authoritative process state
- Power Automate or Logic Apps runs deterministic integration steps
- Human reviewers approve consequential decisions
- Application Insights and Foundry evaluations watch the whole thing
At MGP Inc., we generally recommend this hybrid architecture for local government clients adopting Microsoft Foundry: agents perform bounded reasoning, deterministic workflows control the process, authoritative state remains in governed systems, and employees retain approval authority over consequential decisions.
This is an architectural recommendation, and it aligns with Microsoft's guidance on least privilege, durable state, human participation, security trimming, output validation, and end-to-end observability.
Governance is designed in, not bolted on
Every added agent multiplies model calls, tool interactions, context transfers, and failure points. A production Foundry architecture covers four areas.
Identity and authorization
Each agent gets only the models, knowledge, and tools it needs for its responsibilities. No connected or hosted agent may become a path around the requesting user's authorization boundary.
State and context
Do not forward full conversation history by default; pass the minimum the next step needs. Externalize long-running state so execution resumes instead of replaying. Compact and prune as transitions accumulate.
Reliability
Transitions need timeouts, retries, output validation, checkpoints, circuit breakers, and graceful degradation. A malformed result from one agent must not become trusted input for another.
Observability and evaluation
One trace should connect the user request, parent agent, child agent, model call, tool call, approval checkpoint, and final outcome. Cost belongs in the same view: every added agent is another set of model calls to pay for.
Foundry's Monitor tab on a single deployment: requests, tokens, and estimated cost. Multiply this by every agent in the team before you add the next one.
A single trace in Foundry: the request, the agents it touched, every model and tool call underneath, and the outcome. This records performance.
Final takeaway
The right Foundry architecture is rarely the one with the most agents. It uses the fewest agents necessary, gives each a distinct responsibility and limited authority, matches the pattern to the work, and wraps deterministic controls around anything that touches people, records, money, or public services.
Each pattern arranges the sections differently. Sequential plays them one after another. Concurrent has them play at once and resolves the chord. Handoff passes the melody to whichever section the piece calls for. Group chat lets sections rehearse a passage until it meets the score. Magentic lets the manager rewrite the score mid-performance. Foundry is the hall: the stage, the acoustics, the lights, the recording that proves what was played.
The future is not one agent that does everything. It is a governed team of specialists that knows how to work together, when to ask for help, and where its authority ends. The sections play. The conductor decides.
Microsoft resources
