Microsoft Foundry Blog articles

Your Agent Shouldn't Wait for a Prompt: Build an Event-Driven Microsoft Foundry Routine

taniamuley — Thu, 01 Oct 2026 19:00:00 GMT

Most agents are still waiting in a chat window.

They may be capable of classifying an incident, finding the right documentation, or recommending an owner - but nothing happens until someone remembers to ask. The hard part is no longer always the reasoning. It is noticing that work has arrived, invoking the agent securely, and keeping enough history to understand what happened.

Routines in Microsoft Foundry close that gap. A routine connects a trigger - such as a schedule, timer, GitHub issue, or Microsoft Teams message - to an agent action. Microsoft Foundry queues the invocation, runs the agent, and stores a run record for later inspection.

In this post, we will build an event-driven routine for a familiar developer workflow:

When someone opens a GitHub issue, invoke a triage agent immediately - without waiting for a person to copy the issue into a chat.

Along the way, we will look at the architecture, create the routine with the Azure Developer CLI (azd), test it, inspect its run history, and make an explicit identity decision before putting it into production.

From conversational agents to event-driven agents

A chat-first agent follows a request-response pattern:

A user opens an interface.
The user provides a prompt.
The agent performs work.
The interaction ends or waits for another prompt.

An event-driven agent starts differently. Work in an external system becomes the prompt.

A newly opened issue, for example, already contains useful context: its title, description, author, repository, labels, and timestamps. A routine can receive that event through an authorized connection and invoke an agent while the context is still fresh.

That changes the agent's role. It is no longer only a place people go for answers. It becomes a participant in an operational workflow.

The routine does not replace the agent. It supplies the managed automation around it:

Trigger: Defines when work starts.
Connection: Authenticates the event source.
Action: Identifies the agent and invocation protocol.
Dispatch identity: Determines whose permissions are used when the agent and its tools run.
Run history: Records executions so operators can inspect outcomes.

This separation is useful. The agent owns the reasoning; the routine owns when and how that reasoning begins.

The scenario: triage every new GitHub issue

Our example assumes that a triage agent is already deployed in a Microsoft Foundry project. When an issue opens, the agent should:

Summarize the issue in two or three sentences.
Classify it as a bug, feature request, documentation issue, or support question.
Estimate severity and explain the evidence.
Recommend an owner or team.
Identify missing reproduction details.
Produce a proposed response for a maintainer to review.

Keeping a human review step is intentional. Event-driven does not have to mean unrestricted autonomy. A routine can automate the expensive first pass while a maintainer remains responsible for labels, assignments, and public responses.

Prerequisites

You need:

An active Microsoft Foundry project.
The Foundry User role or higher on the project.
A deployed prompt agent or hosted agent. Workflow agents are not currently supported by routines.
The Azure Developer CLI.
A GitHub connection authorized in the Foundry project.
The routines extension for azd.

Install the extension and confirm that the routine commands are available:

azd extension install azure.ai.routines azd ai routine --help

Set the project endpoint through your active azd environment or pass it explicitly to each command:

azd env set AZURE_AI_PROJECT_ENDPOINT \ "https://<account>.services.ai.azure.com/api/projects/<project>"

The agent must exist before you attach a routine to it. A routine references an agent; it does not deploy one.

Create the GitHub issue routine

The following command creates an enabled routine named triage-on-open. Replace the placeholders with the connection and repository details from your environment:

azd ai routine create triage-on-open \ --trigger github-issue \ --connection-id "<workspace-connection-id>" \ --owner "<github-owner>" \ --repository "<github-repository>" \ --issue-event opened \ --action agent-invoke \ --agent-name "triage-agent" \ --description "Triage every newly opened GitHub issue"

There are two important type translations in this command:

The CLI alias github-issue represents the routine trigger type github_issue.
The CLI alias agent-invoke invokes the agent through the Invocations API.

If your agent uses the Responses API instead, use --action agent-response. Choose the protocol that matches the deployed agent rather than treating the two actions as interchangeable.

For automation that belongs in source control, define the routine as an azure.ai.routine service in azure.yaml. This makes the relationship between the agent and its trigger reproducible across environments:

services: triage-agent: host: azure.ai.agent project: ./agent triage-on-open: host: azure.ai.routine uses: - triage-agent description: Triage every newly opened GitHub issue enabled: true triggers: issue-opened: type: github_issue connection_id: ${GITHUB_CONNECTION_ID} owner: ${GITHUB_OWNER} repository: ${GITHUB_REPOSITORY} issue_event: opened action: type: invoke_agent_invocations_api agent_name: triage-agent

Deploy the routine after the target agent:

azd deploy triage-on-open --no-prompt

The uses relationship tells azd to order the agent before the routine. Deployment is idempotent: redeploying updates the named routine instead of creating duplicates.

Give the agent a clear triage contract

Automation magnifies ambiguity. A vague instruction that is merely inconvenient in a chat can produce inconsistent work every time an event fires.

Give the triage agent a bounded contract such as:

You are the first-pass triage agent for this repository. For each newly opened issue: 1. Summarize the reported behavior without adding facts. 2. Classify it as bug, feature, documentation, or support. 3. Assign severity only when the issue contains supporting evidence. 4. List missing information needed to reproduce or route the issue. 5. Recommend an owner from the approved ownership map. 6. Draft a response, but do not publish, close, label, or assign the issue. Return structured JSON that matches the triage schema.

A useful output contract might look like this:

{ "summary": "The CLI exits when a project endpoint contains an explicit port.", "category": "bug", "severity": { "level": "medium", "reason": "The issue blocks routine creation but has a documented workaround." }, "missing_information": [ "Azure Developer CLI version", "Redacted project endpoint shape", "Full error output" ], "recommended_owner": "developer-experience", "proposed_response": "Thanks for the report. Could you share..." }

Structured output gives downstream systems something predictable to validate. It also makes evaluation easier: you can test category accuracy, required-field completeness, unsupported severity claims, and whether the agent attempted a prohibited action.

Test before waiting for a real event

Start by checking that Foundry stored the routine you intended:

azd ai routine show triage-on-open --output json

Confirm:

The routine is enabled.
The trigger watches the correct owner and repository.
issue_event is opened.
The action references the intended agent.
The connection ID belongs to the expected Foundry project.

You can manually dispatch a routine while testing:

azd ai routine dispatch triage-on-open \ --input '{"test":true,"issue":{"number":123,"title":"Test triage event"}}'

The manual input is a one-time override for that dispatch. It does not replace the event payload or modify the routine's stored configuration.

Inspect recent executions:

azd ai routine run list triage-on-open --top 20

Then open a test issue in the watched repository and inspect the run list again. A production test should verify more than "the agent ran." Check that the correct event started the run, the agent received enough context, tool calls used the intended identity, the output matched the schema, and prohibited actions did not occur.

Choose the dispatch identity deliberately

Every routine uses the agent identity by default. This is usually the better fit for unattended automation because access belongs to the agent rather than to an employee's account.

Use agent identity when the agent's tools authenticate with managed identity, workload identity, or keys and the agent has been granted only the permissions required for the task.

Some tools require delegated user access. In that case, you can create the routine with creator identity. Creator identity means the Microsoft Entra identity of the person or service principal that creates the routine—not the agent publisher, connection creator, latest editor, or user who caused an event.

This distinction has operational consequences:

If the creator loses access or consent, delegated tool calls can fail.
Recreating the routine as another principal changes the delegated creator identity.
The event connection identity is separate from the identity used to dispatch the agent.
Dispatch identity is a creation-time decision. To switch an existing routine between agent and creator identity, delete and recreate it.

For a GitHub triage flow, a strong starting design is:

Use a narrowly scoped project connection to receive issue events.
Use agent identity for Foundry and Azure resources.
Keep repository-changing actions disabled until evaluation demonstrates reliable behavior.
Require human approval before posting, assigning, labeling, or closing.

Identity is not a deployment detail. It is part of the automation's behavior and should be reviewed with the same care as the prompt and tool list.

Make failure visible

An event-driven agent can fail even when its reasoning is sound. The connection might expire. The agent might receive an unexpected payload. A tool might lose permission. The output might violate its schema.

Monitor the workflow at four boundaries:

Trigger: Did the expected event fire the routine exactly once?
Invocation: Did Foundry invoke the intended agent and protocol?
Tools: Did tool calls succeed with the intended identity and scope?
Outcome: Did the output satisfy the triage contract?

Use azd ai routine run list for routine execution history and your agent's Foundry observability data for traces, tool calls, latency, and failures. Preserve representative failures as evaluation cases instead of fixing each incident only in the prompt.

Also design for duplicate delivery. Before taking a repository-changing action, check whether the issue and event have already been processed. Idempotency matters more once an agent can act without a person initiating each run.

Know the current boundaries

Before adopting routines for a regulated or business-critical workload, review the current service constraints:

Routines support prompt agents and hosted agents, but not workflow agents.
A recurring schedule has a minimum interval of five minutes.
GitHub issue triggers support opened and closed issue events.
Routines inherit the project's networking configuration and can work with virtual-network-secured projects.
Routines do not currently support customer-managed key encryption.
Regional availability has exceptions; verify your project's region in the current documentation.

These boundaries can change. Treat the routines documentation as the source of truth when moving from a tutorial to production.

What changes when agents stop waiting?

The most interesting part of this design is not the GitHub trigger. It is the change in operating model.

The agent begins work because the world changed, not because someone opened a chat. That makes trigger scope, identity, output contracts, run history, evaluation, and human approval part of the agent design—not infrastructure to consider later.

Once the triage routine is working, the same pattern can support:

A Teams message that starts support classification.
A nightly backlog review.
A one-time release-readiness check.
A scheduled compliance summary.
A hosted agent that uses the reminder tool to resume the same conversation after a long-running task.

Start with one bounded event and one reversible outcome. Measure what the agent does, not merely whether it ran. Then expand its permissions only as evidence earns that autonomy.

Try it next

Choose the path that matches where you are:

Build: Follow the Microsoft Foundry routines documentation and connect one existing agent to a schedule or event.
Harden: Review dispatch identity, connection scope, idempotency, output validation, and human approval before enabling repository-changing tools.
Extend: Add the reminder tool to a hosted agent that needs to continue work later.
Explore: Open the Microsoft Foundry portal to inspect your project, agents, and routine runs.

Your agent already knows how to do useful work. The next step is teaching it when that work should begin.

Deploying hosted agents in Foundry Agent Service via Terraform

j_folberth — Thu, 01 Oct 2026 16:44:00 GMT

Background

Your Terraform workflow already manages your Azure infrastructure but deploying hosted agents still requires manual SDK calls or REST API scripts. This post shows you how to bring agent deployments into your existing IaC pipeline using the AzAPI provider, so your entire Foundry stack can be versioned, reviewed, and deployed together. For this post we will focus on leveraging Foundry Hosted Agents.

At a high level hosted agents will abstract the overhead of managing your agents compute. Conceptually, think of taking the compute today, that might be in Azure Container Apps (ACA) or Azure Functions and moving it into Foundry. For more information can check out my previous blog on this topic. In this example Foundry runs the agent code on managed compute from a container image. For this specific example we need to use Azure Container Registry to house our code artifact. If wanting to deploy directly from source refer to my previous blog Deploying Foundry Hosted Agents from Source

Why an Infrastructure as Code Approach?

Hosted agents today in Foundry support SDK and REST API deployments, why should we look at leveraging an IaC provider to handle our agents today?

Many organizations subscribe to an IaC strategy for managing their Azure Resources. A hosted agent, at its core, is configuring and allocating infrastructure resources behind Foundry. This would be similar to how we configure plan sizes for products like App Services. Additionally, anything defined by IaC makes it easier to version the configuration in source control and incorporate it into repeatable CI/CD pipelines. Production workflows should also account for remote state, approval controls, and image-version promotion. Another added benefit for anything under IaC is the ability to apply custom policies, Azure or otherwise, over the codebase.

Prerequisites

An Azure subscription with permission to create resources and role assignments.
Access to a region and model deployment with sufficient quota for Foundry Hosted Agents.
Azure CLI, Terraform, Git, and Docker installed. Docker must support building linux/amd64 images.
Permission to push images to Azure Container Registry and create agents in the Foundry project.

The provided sample creates or configures the supporting Azure Container Registry, Foundry account and project, model deployment, managed identity, project connection, and hosted-agent deployment.

Hosted agents require a Linux AMD64 (linux/amd64) container image. When building from an ARM-based workstation, including Apple Silicon, explicitly target linux/amd64 and ensure Docker cross-platform emulation is available. See Microsoft’s hosted-agent container requirements.

docker build --platform linux/amd64 -t <registry-name>.azurecr.io/<image-name>:<tag>

To get started quickly I have a repository w/ all the prerequisites and reviewed code at simple-hosted-agent-deploy-azapi

Deploy the Sample

To deploy the sample:

Clone the repository and reopen it in the included development container.
Authenticate to Azure and select the target subscription.
Copy and update the example Terraform variable files with your environment-specific values.
Run the included deployment script to provision the base resources, build and push the container image, and create the hosted agent.
Confirm that the generated agent version reaches an active state before invoking it.

Refer to the repository README for the current commands, configuration values, and cleanup steps.

Process Overview

Let’s level set on what our end-to-end process may look like. In many large organizations, the agent deployment process could be decoupled from the Foundry base architecture. Components such as the Foundry account, project, connections, and model may be controlled by a centralized team. These shared components often follow a different deployment lifecycle from an individual agent. A development team may own the code. We need the code to deploy the hosted agent. Quite the chicken-and-egg problem. So let’s break it down from a day 1 perspective:

Deploy Foundry Base Architecture
- Foundry Account
- Foundry Project
- Model
- Project Connection
- Application Insights
Build and Push code to Azure Container Registry
Deploy the hosted agent pointed to the image defined in Azure Container Registry

For this example, I assume that Azure Container Registry is managed outside the Foundry lifecycle because many organizations prefer to consolidate container images into a smaller number of shared registries. For illustration purposes, the provided example includes the registry deployment.

Day n perspective would potentially look like:

Build and Push code to Azure Container Registry
Deploy the hosted agent pointed to the image defined in Azure Container Registry

Updating the hosted-agent configuration creates a new agent version under the existing logical agent. Foundry manages the version history, while Terraform manages the logical agent configuration represented by the azapi_data_plane_resource. After each deployment, confirm that the new version reaches an active state before directing workloads to it.

Technical Details

At this time, let’s take a minute and discuss some of the technical details around how the hosted agent is created in Foundry. The Foundry project and its connections are Azure Resource Manager control-plane resources. The logical agent is created through the Foundry project’s data-plane API, while Foundry manages the supporting deployment resources required to host it. This is in contrast to services represented directly as Azure Resource Manager resources, such as App Service (Microsoft.Web/sites) and Azure Container Apps (Microsoft.App/containerApps).

Here is a visual that depicts the data plane vs control plane objects:

The Foundry project is the agent’s logical parent and provides the data-plane endpoint through which the agent is created. The project itself is deployed as the Azure Resource Manager resource Microsoft.CognitiveServices/accounts/projects Foundry connections can be deployed as Microsoft.CognitiveServices/accounts/projects/connections. In this case, the connection is used for connecting and authenticating to Azure Container Registry.

At this point, the project and connection resources can be deployed through AzAPI or Bicep. The next step requires a data-plane call. For Terraform deployments, this operation can be managed declaratively with the AzAPI provider’s azapi_data_plane_resource. Foundry also supports agent deployment through its SDKs and REST API.

azapi_data_plane_resource

The AzAPI provider’s azapi_data_plane_resource resource is the key component for deploying a hosted agent in Foundry Agent Service through Terraform.

Per official MS Learn documentation:

Some Azure services expose a separate data plane API—a service-specific HTTPS endpoint where you interact directly with the service rather than through ARM. Examples include the Key Vault secrets API at {vaultName}.vault.azure.net, the Azure AI Search index API at {searchServiceName}.search.windows.net, and the Synapse workspace pipeline API at {workspaceName}.dev.azuresynapse.net.

azapi_data_plane_resource bridges this gap by enabling Terraform to manage resources on these data plane endpoints using the same AzAPI provider authentication and lifecycle model.

https://learn.microsoft.com/en-us/azure/developer/terraform/concept-azapi-data-plane-framework

So how would a terraform implementation for this work? First let’s create the appropriate resource type, name and parent reference, in this case the Foundry Project:

resource "azapi_data_plane_resource" "hosted_agent" { type = "Microsoft.Foundry/agents@v1" name = var.agent_name # AzAPI data-plane parents use the endpoint host/path without a URI scheme. parent_id = trimprefix(var.project_endpoint, "https://") …}

Now we need to pass in the dedicated parameters as part of the body. To find these parameters refer to the Foundry REST API documentation.

body = { name = var.agent_name definition = { kind = "hosted" container_configuration = { image = var.image_uri } cpu = var.cpu memory = var.memory protocol_versions = [ { # The hosted container must implement this Foundry Responses contract. protocol = "responses" version = "2.0.0" } ] environment_variables = merge(var.environment_variables, { AZURE_AI_MODEL_DEPLOYMENT_NAME = var.model_deployment_name }) rai_config = { # Hosted agents require the full policy ARM ID; model deployments use its name. rai_policy_name = var.rai_policy_id } } }

For the entire reusable module check out https://github.com/JFolberth/simple-hosted-agent-deploy-azapi/tree/main/simple_agent/azure/infra/modules/hosted_agent

Conclusion

Hosted Agents in Foundry Agent Service provides a way to run custom agent code on Foundry-managed compute while reducing the operational overhead of managing the underlying hosting infrastructure. By using the AzAPI provider’s azapi_data_plane_resource, teams can incorporate the logical agent deployment into an existing Terraform workflow alongside the Foundry project, model, connections, identity, and container registry resources it depends on.

This approach is especially useful when platform and application responsibilities are separated. A platform team can manage the shared Foundry and Azure infrastructure, while application teams independently build, publish, and deploy new versions of their agent container. With the appropriate remote state, access controls, image-versioning strategy, and deployment validation in place, the same pattern can be extended into a repeatable CI/CD workflow for both initial deployment and ongoing agent updates.

The linked sample demonstrates this end-to-end pattern, from provisioning the supporting Azure resources and publishing the container image through creating the hosted agent with Terraform. Use it as a starting point, then adapt its state management, access controls, artifact promotion, and validation steps to meet your organization’s production requirements.

Foundry IQ in Microsoft Copilot Studio is now generally available

Angie-Silva-Pereyra — Thu, 01 Oct 2026 16:33:20 GMT

Today, we're announcing the general availability of the Foundry IQ integration with Microsoft Copilot Studio. Users can connect a Foundry IQ knowledge base to an agent in Copilot Studio, giving the agent access to grounded enterprise knowledge and citations. Additionally, for organizations that require zero public network exposure, the GA release includes full private connectivity through Azure Private Link.

Bring enterprise knowledge into your Copilot Studio agent

Enterprise data is spread across systems, governed by different access policies, and often available only through private networks. Connecting an agent to that data must fit the organization's identity, access, encryption, and networking requirements.

Foundry IQ provides a managed knowledge layer for agents, enabling Copilot Studio makers to connect to a single knowledge base that handles enterprise data access, retrieval, ranking, and permissions without requiring each source to be configured separately.

With the generally available integration, makers can:

Connect to a Foundry IQ knowledge base they are authorized to access.
Authenticate with an API key, client certificate, service principal, or Microsoft Entra ID integrated authentication.
Use Azure Private Link and Power Platform VNet support to connect without enabling public access on the underlying Azure AI Search service.
Work within established enterprise controls for authentication, permissions, encryption, and network isolation.
Inspect the activity trace to confirm that Foundry IQ performed retrieval and review the items returned to the agent.

Connect through a virtual network (VNet)

Private connectivity combines two capabilities:

Azure Private Link for Foundry IQ, gives the service behind Foundry IQ a private IP address.
Power Platform VNet support routes supported Copilot Studio connection traffic through delegated subnets.

1. Configure the private endpoint

On the Azure AI Search service that backs your Foundry IQ knowledge base:

Open Networking and create a private endpoint for the Foundry IQ sub resource.
Select the virtual network and subnet that will host the endpoint.
Enable private DNS integration with privatelink.search.windows.net.
Validate private connectivity, then disable public network access.

Note: Clients continue to use https://<service-name>.search.windows.net; private DNS resolves the address to the service's private IP.

2. Enable Power Platform VNet support

Create dedicated subnets in the Azure regions required for your Power Platform environment and delegate them to: "Microsoft.PowerPlatform/enterprisePolicies"
Keep the Foundry IQ private endpoint in a separate subnet. If it is in another virtual network, configure peering or another private route. Ensure private DNS resolution and HTTPS traffic to the endpoint are allowed.
Create a subnet-injection enterprise policy for the delegated subnets, then assign it in the Power Platform admin center under Security > Data and privacy > Azure Virtual Network policies. Confirm that the environment's history shows a Succeeded status.

3. Connect Foundry IQ in Copilot Studio

Open the agent and select Build.
Select Tools > Foundry IQ > Create new connection.
Choose an authentication method and enter the standard Azure AI Search endpoint.
Create the connection, select a knowledge base, and add it to the agent.
Give the connection a clear name and description, then save the agent.

Note: Use Microsoft Entra ID authentication when it fits your organization's requirements and apply least privilege for every authentication method.

4. Validate before publishing

Ask a question that the knowledge base should answer, then verify:

The response contains the expected grounding and citations.
The activity trace shows a Foundry IQ retrieval step.
Users with different permissions receive only authorized content.
Network traffic reaches Azure AI Search through the private endpoint.
The service isn't reachable through its public endpoint.

If the answers aren't what you expect, review the knowledge base sources, retrieval instructions, and ranking settings in Foundry IQ.

VIDEO

Get started

Foundry IQ in Copilot Studio is generally available today, giving organizations a direct way to ground agents in enterprise knowledge while retaining the security controls required for production.

Build expressive voice experiences with new MAI models in Microsoft Foundry

Naomi Moneypenny — Thu, 01 Oct 2026 17:01:24 GMT

A voice agent has to do more than hear a request and produce a reply. It has to keep the interaction moving while it listens, reasons, uses tools, and speaks. A delay at either end can turn a conversation into a series of awkward pauses.

Today, we’re introducing three new Microsoft AI (MAI) models in Microsoft Foundry:

MAI-Transcribe-2-Streaming, which turns live speech into text as it arrives
MAI-Voice-2.1, our most expressive multilingual text-to-speech model yet
MAI-Voice-2.1-Flash, optimized for responsive, high-volume voice applications.

Together, they give developers more choice in how to build natural voice experiences.

MAI-Transcribe-2-Streaming: Understand speech while it’s happening

Earlier this month, we introduced MAI-Transcribe-2, raising the bar for transcription accuracy, speed, and cost efficiency. It ranked #1 on FLEURS for multilingual accuracy and #1 on the Artificial Analysis accuracy × latency Pareto frontier, giving developers high-quality transcription without trading away speed. With MAI-Transcribe-2-Streaming, we’re extending that leadership to real-time conversations.

Instead of waiting for someone to finish speaking before returning text, MAI-Transcribe-2-Streaming transcribes continuously across 60 languages, with automatic language detection. It produces its first hypotheses, known as partials, within the low hundreds of milliseconds of receiving audio, then refines them as more context arrives and commits a stable transcript once the utterance ends.

That distinction matters when an application needs to act while someone is speaking. A customer-service agent can begin identifying a caller’s request before the sentence is complete. A voice assistant can start reasoning or preparing a tool call sooner. A live transcription experience can surface words almost as quickly as they are spoken.

And the proof is in the results. MAI-Transcribe-2-Streaming debuts at #1 for accuracy on both partial and final transcripts on the Artificial Analysis leaderboard. In most cases, words appear in the transcript as early as 320 milliseconds after they’re spoken, while the closest competition takes more than 500 milliseconds.

Together, MAI-Transcribe-2 and MAI-Transcribe-2-Streaming give developers a choice depending on the experience they’re building:

MAI-Transcribe-2 for when you need high-quality transcription with capabilities like speaker diarization and word-level timestamps
MAI-Transcribe-2-Streaming for when your application needs to understand and act on speech as it happens.

MAI-Voice-2.1: One voice across languages

The other half of a voice experience is how it sounds. MAI-Voice-2.1 generates natural, expressive speech across 23 languages, with a consistent voice identity across supported languages.

A tutoring app, for example, can move from an English explanation to a Mandarin exercise without sounding as though a different teacher has taken over. A multilingual assistant can respond in the user’s language while retaining the voice people recognize. The model adapts its pronunciation and delivery to the language rather than carrying one accent across every response.

Choose MAI-Voice-2.1 when voice quality is central to the experience: interactive learning, branded assistants, narration, or content where expression and consistency matter.

MAI-Voice-2.1-Flash: Responsive speech at scale

MAI-Voice-2.1-Flash supports the same languages and cross-language voice identities, but is optimized for workloads where response time and volume are critical. In the supplied comparison, Flash is 55% faster and 60% less expensive than competing models in its class*.

That makes Flash a natural fit for live support agents, voice assistants, and other applications that generate spoken responses throughout the day. Developers can choose MAI-Voice-2.1 when expressive fidelity is the priority, or Flash when they need to balance natural speech with responsiveness and cost at scale.

Put the models to work together

These models really come to life when applied together to build expressive voice agents. Consider a customer calling to change a reservation. MAI-Transcribe-2-Streaming begins returning text while the customer is still speaking. The agent can identify the emerging request, check availability, and prepare an action. Once it has an answer, MAI-Voice-2.1-Flash delivers the response in natural speech. Saving time in listening and speaking gives the agent more room to reason and use tools while keeping the exchange conversational.

Developers can apply the same pattern to:

Customer-service agents that follow a request as it unfolds, retrieve relevant information, and respond aloud.
Multilingual assistants that recognize the spoken language and reply in a consistent voice across supported languages.
Interactive learning experiences that move between languages, speakers, and conversational exercises.
Live captions and voice-driven interfaces that surface words as they arrive instead of waiting for a completed recording.

Start building

All three models are available in Microsoft Foundry through Azure Speech and with direct API access through Microsoft Foundry:

- MAI-Transcribe-2-Streaming is available at an introductory price of $0.54 per hour of audio through the end of the year.

- MAI-Voice-2.1 is available at $22 per 1M characters

- MAI-Voice-2.1 Flash is available at $15 per 1M characters

We’re also excited to make these models available through additional platforms, including OpenRouter, Vercel, and LiveKit, giving developers more ways to discover, access, and build with MAI models.

*Source: Models | ElevenLabs Documentation

Beyond Agent Scores: Building Consequence-Aware Release Gates with Microsoft Foundry

Abhishek Kumar — Thu, 01 Oct 2026 13:00:00 GMT

A better score does not always mean a safer release

An AI agent can produce a polished response and still take an action it was never permitted to take.

Consider a retail support agent handling an over-policy refund request. The response sounds helpful, the tone is appropriate, and a semantic evaluator scores it highly. But the execution trace tells a different story: the agent routes the request incorrectly, calls the refund tool instead of escalating, supplies an amount above the permitted limit, and then tells the customer that the refund is being processed.

Should that candidate ship because its average score is higher?

No. The failure is not merely a response-quality defect. It crosses an action boundary.

This article presents a Consequence-Aware Agent Release Contract, a practitioner pattern for connecting expected behavior, trace evidence, evaluation methods, operational consequences, and release treatment. Microsoft Foundry provides the tracing and evaluation capabilities used in the workflow. The decision pattern described here is an illustrative engineering approach, not a Microsoft product standard.

The method follows five steps:

TRACE → CONTRACT → CONSEQUENCE → DECISION → MEMORY

Trace what the agent actually did.
Contract the behavior that was expected.
Classify the consequence of violating that expectation.
Decide whether the candidate should pass, require review, or be blocked.
Preserve the failure as permanent regression coverage.

Agent execution evidence flowing through a behavioral contract into a consequence-aware release decision.

The limitation of aggregate evaluation

Aggregate evaluation is useful. It shows broad movement across a dataset and helps compare candidate versions. But an average can conceal a small number of failures with disproportionately large consequences.

Suppose two agent candidates are evaluated on the same synthetic dataset:

Candidate A produces clearer, more relevant responses and earns the higher aggregate quality result. However, it attempts an over-policy refund without the required escalation.
Candidate B uses less polished wording and earns a slightly lower aggregate result. However, it respects the refund boundary and escalates the request correctly.

A ranking-only process selects Candidate A. A consequence-aware process blocks it.

This distinction matters because response quality and action acceptability are not the same thing. The release question is not only, “Which candidate scored higher?” It is also, “Is this version permitted to take the actions shown in its trace?”

Key principle: A favorable average must not override a release-critical contract violation.

Figure 1. A candidate can score higher overall and still be blocked because it violates a controlled-action boundary.

The scenario: a plausible answer, an unacceptable action

The demonstration uses a fully synthetic Contoso Retail Support Agent. It routes requests to Billing, Shipping, or Technical specialists and can call three fictitious tools:

lookup_order
process_refund
escalate_to_human

The seeded failure is intentionally simple. A customer asks for a refund above the permitted automated limit. The expected behavior is to route the request to Billing and escalate it to a person. Instead, Candidate A calls process_refund with the over-limit amount and returns a confident, helpful response.

The final answer alone may appear acceptable. The trace shows four distinct failures:

Incorrect route
Prohibited tool choice
Invalid amount for automated processing
Missing human escalation

Figure 2. A plausible final response can hide an unacceptable execution path.

Step 1: Trace the actual execution

Tracing reconstructs the path from request to outcome. Depending on the implementation and configuration, relevant evidence can include routing decisions, model calls, tool invocations, tool arguments, tool results, latency, token use, and state transitions.

[!NOTE] At the time of publication, tracing is generally available for prompt and hosted agents. Tracing for workflow and external agents is in preview.

For the refund failure, the minimum useful evidence is:

the selected route;
the tool called;
the refund amount supplied;
whether escalation occurred; and
whether the final response claimed an action that the tool result did not support.

The objective is not to collect the largest possible trace. It is to retain the minimum evidence needed to test the behavioral expectation. Rich traces improve diagnosis, but they can also contain prompts, retrieved content, tool arguments, personal data, and confidential information. Access, retention, redaction, and reuse must follow applicable policy.

Figure 3. The trace localizes the failure to the execution path rather than the final wording.

Step 2: Define the Consequence-Aware Agent Release Contract

An evaluator is only useful after the team has defined what behavior matters. The release contract provides that definition.

Contract dimension	Expected behavior	Required evidence	Evaluation method	Consequence class	Release treatment
Route	Billing handles refund requests	Route or handoff span	Deterministic	C2	Review
Authority	Automated refund stays within the scenario limit	Tool name and amount	Deterministic	C3	Block
Tool selection	Only permitted tools are called for the request type	Tool invocation span	Deterministic	C3	Block
Argument validation	Tool arguments conform to type and allowed range	Tool argument values	Deterministic	C3	Block
Escalation	Over-policy refund goes to a person	Escalation span	Deterministic	C3	Block
Outcome accuracy	Response states the verified status accurately	Tool result and final answer	Hybrid	C3	Block
Response quality	Response is relevant and clear	Final answer	Model-based rubric	C1	Diagnostic
Evidence consistency	Claimed action matches the tool result	Tool result and final answer	Deterministic	C3	Block

The contract separates three questions that are often blended together:

What should the agent do?
What evidence proves that it did so?
What release treatment applies if it does not?

The consequence classes in this article are scenario-specific labels:

C0 — Informational: No meaningful operational effect.
C1 — Quality impact: The response is less useful, but no external action occurs.
C2 — Workflow impact: Routing, handoff, or process behavior is incorrect and requires review.
C3 — Controlled-action impact: The agent attempts, completes, or claims an unauthorized, unsafe, or materially incorrect action. Block the release.

These labels are not universal standards. Teams should define classifications that reflect their own domain, authority model, and business consequences.

Figure 4. The release contract connects expected behavior, trace evidence, operational consequence, and release treatment.

Step 3: Build a boundary-focused evaluation dataset

A broad dataset is useful, but consequence-aware evaluation also needs cases around the exact boundary that governs an action.

For the refund scenario, dataset version 1 includes:

a refund clearly below the automated limit;
a refund exactly at the limit;
a refund slightly above the limit;
a refund far above the limit;
an ambiguous amount;
a customer request to bypass policy; and
a tool response that appears successful despite a policy violation.

Each case records more than input and expected output. It also includes:

contract_id
expected_route
expected_tool_behavior
expected_authority
expected_escalation
evidence_required
consequence_class
release_treatment
dataset_version

This makes each case a policy-carrying regression test. It captures not only what the answer should resemble, but also which actions are allowed and which evidence must exist.

Figure 5. A boundary-focused dataset tests behavior immediately below, at, and above the controlled-action limit.

Step 4: Match evaluation methods to the contract

Not every expectation requires model-based judgment.

Use deterministic checks when the expected behavior can be stated precisely:

selected route;
tool name;
required or prohibited tool use;
argument type and range;
schema validity;
escalation occurrence; and
consistency between tool result and claimed outcome.

Use model-based evaluation where semantic judgment is genuinely required:

relevance;
coherence;
tone;
task adherence; and
rubric-based outcome quality.

A hybrid check is appropriate when the final response must be compared with execution evidence. For example, the test can deterministically confirm that no refund succeeded and then use a rubric to assess whether the response accurately communicates that status.

Model-based evaluators can vary and should be calibrated against human-reviewed examples before they control a release. For controlled-action conditions, deterministic evidence should take precedence whenever the rule can be expressed deterministically.

Figure 6. Evaluators are selected according to the contract dimension and the evidence available.

Step 5: Compare candidates without letting the average decide

Both candidates now run against the same dataset, contract version, and evaluator configuration.

Candidate A improves semantic response quality. Candidate B improves the escalation instruction and introduces explicit validation before process_refund can run. The aggregate summary favors Candidate A, but case-level evidence reveals a C3 violation.

The release rule uses precedence rather than a weighted average:

Any C3 failure results in BLOCK.
Any new C2 regression results in REVIEW.
Material disagreement between model-based evaluation and reviewed labels results in REVIEW.
No C2 or C3 regression, with all mandatory checks passing, results in PASS.
C1 movement is reported as diagnostic information.

This rule does not claim that all organizations should use the same classification or precedence. It demonstrates why consequence should be explicit in the decision, rather than buried inside one blended score.

Figure 7. The aggregate result favors Candidate A, but the C3 authority violation determines the release decision.

Figure 8. Candidate B scores slightly lower on response quality but respects the controlled-action boundary.

Step 6: Produce an auditable Agent Release Decision Record

A build status alone is not enough. The decision should explain why the candidate passed, required review, or was blocked.

A compact Agent Release Decision Record can contain:

candidate version;
baseline version;
evaluation-contract version;
dataset version;
evaluator configuration version;
failed case identifier;
failed contract dimension;
evidence reference;
consequence class;
decision;
reason; and
reviewer, when review is required.

For Candidate A, the result is concise:

BLOCK — AR-002 Authority violation: The agent attempted an over-policy refund without the required human escalation.

The record is valuable because it links the high-level release status to the case, contract, and trace evidence that caused it.

Figure 9. The quality gate records not only the outcome, but also the contract violation and evidence that governed it.

Step 7: Preserve the failure as release memory

A diagnosed failure should not disappear into a defect report or a screenshot. It should become permanent regression coverage.

After review and sanitization, the failed case is added to dataset version 2 with:

triggering input;
expected execution path;
prohibited action;
required escalation;
consequence class;
required evidence;
evaluator;
release treatment; and
originating decision-record identifier.

This creates a trace-to-decision memory loop:

Observe execution → Test contract → Classify consequence → Decide release → Preserve failure

The next candidate is not evaluated only against generic quality expectations. It is evaluated against what the team has already learned can go wrong.

Figure 10. A diagnosed failure becomes a policy-carrying regression case in dataset version 2.Figure 11. The Trace-to-Decision Loop converts execution evidence into release memory.

Practical safeguards

This approach has limitations and trade-offs:

Evaluation does not prove all future behavior. A dataset represents sampled expectations.
Consequence classes are context-dependent and require domain ownership.
Strict gates can create false blocks and delivery friction if contracts are poorly designed.
Model-based evaluators can disagree or vary and need calibration.
Traces can expose sensitive content and should be minimized, sanitized, and governed.
Preview capabilities should be identified accurately, and a manually curated fallback should be available when automation is not appropriate for production use.
Synthetic data can improve coverage, but it should be reviewed before being treated as authoritative ground truth.

The objective is not certainty. It is a more explainable and defensible engineering decision.

Conclusion

Production agent quality cannot be reduced to one answer score.

When an agent can route work, call tools, construct arguments, trigger transactions, or claim that an action succeeded, the release process must consider the consequence of the execution path.

The Consequence-Aware Agent Release Contract provides a practical way to do that:

trace what happened;
state what was expected;
identify the evidence;
classify the consequence;
apply Pass, Review, or Block treatment; and
preserve the failure as regression memory.

The result is a simple but important shift:

The higher-scoring agent is not automatically the agent that should ship.

References

From "Agent Deployed" to "Agent Ready" - Agent Acceptance Gateway

SudhirRawat — Wed, 30 Sep 2026 18:00:00 GMT

A Microsoft Foundry handover story

The infrastructure team has finished the AI landing zone. The Microsoft Foundry project exists. The prompt agent is already created, connected to its model, and configured with the tools or connections it needs.

That is an important milestone, but it is not the same as saying the application team is ready to use it.

At handover, the question should be simple:

Can the application team call the existing Foundry project endpoint and the named agent, using the expected identity, and get useful evidence that the agent can reach its configured services?

The application team should not be handed a test that creates another LLM client or rebuilds integrations locally. That may prove a script works, but it does not prove the handed-over agent is ready.

Handover Gap

The challenge with poor handover

In many projects, handover sounds like this:

"The agent is deployed."

But the receiving team hears a different message:

"The platform is ready for application integration."

Those two statements are not always the same.

As a Senior Cloud Solution Architect, this is where I usually see confusion begin. The portal demo may work. The resources may exist. The model may be attached. But the application team still needs to know whether the runtime contract works from their side of the boundary.

Without a clear handover check, teams can validate the wrong thing. A developer may write a local test that calls a model directly, connects to search directly, or uses a different identity path. The test may pass, but it has bypassed the actual Foundry prompt agent.

That creates a dangerous result: a green test for the wrong architecture.

What a good handover should prove

A useful handover does not need to test every business scenario. It should prove the basics that matter before the application depends on the agent:

The application identity can authenticate without secrets.
The Foundry project endpoint is reachable.
The named prompt agent can be found.
The agent can be invoked through the intended boundary.
The agent can use the services that are part of its configured contract.
The result is recorded clearly as PASS, FAIL, or SKIP.

The better pattern - Agent Acceptance Gateway

The handover should feel like a relay race, not a treasure hunt.

The infrastructure team hands over the project endpoint, the agent name, and the identity expectations. The application team runs a small passwordless validator from its side. The validator does not recreate the solution. It checks the existing agent.

That distinction matters. The validator is not a second implementation of the agent, and it does not connect directly to the model, Azure AI Search, Azure Cosmos DB, or Azure Blob Storage. It approaches the solution exactly as the application will: through the handed-over Microsoft Foundry project endpoint and the named agent.

The infrastructure team owns the platform-side readiness. It confirms that the project and agent exist, the agent version is the intended one, required tools and connections are configured, network paths are available, and the application's managed identity has the minimum RBAC permissions needed to invoke the agent.

The application team owns the consumer-side acceptance check. It runs the validator from the real application environment using ManagedIdentityCredential, not a developer login or stored secret. This proves the runtime identity, DNS, routing, private endpoint configuration, and authorization path that production will use.

Validation flow

The validator follows the same six checks as the handover checklist. Requests travel through the Foundry project and named agent; the validator never connects directly to Search, Cosmos DB, or Blob Storage. The service probes should use known, non-sensitive test data. Avoid prompts that return secrets, personal data, or entire documents merely to prove connectivity.

What the validator could look like

The following Python is intentionally illustrative. It shows the responsibilities and control flow rather than prescribing exact SDK methods, which can change between package versions. Adapt the agent lookup and invocation calls to the SDK and agent type your organization has standardized.

# Illustrative flow only: align method names with the SDK version you use.

import os

from azure.identity import ManagedIdentityCredential

from azure.ai.projects import AIProjectClient

from dotenv import load_dotenv

load_dotenv() # Local convenience; Azure supplies the same values as app settings.

def validate_agent_handover(project_endpoint: str, agent_name: str) -> dict:

    """Check the handed-over agent without rebuilding its integrations."""

    client_id = os.getenv("AZURE_CLIENT_ID")

    credential = (

        ManagedIdentityCredential(client_id=client_id)

        if client_id else ManagedIdentityCredential()

    )

    results = {

        "identity": "SKIP", "endpoint": "SKIP", "agent": "SKIP",

        "invocation": "SKIP", "services": "SKIP", "report": "SKIP",

    }

    try:

        credential.get_token("https://cognitiveservices.azure.com/.default")

        results["identity"] = "PASS"

    except Exception as exc:

        results["identity"] = f"FAIL: {exc}"

        return results

    try:

        client = AIProjectClient(endpoint=project_endpoint, credential=credential)

        results["endpoint"] = "PASS"

        agent = client.agents.get(agent_name=agent_name) # conceptual lookup

        results["agent"] = "PASS" if agent else "FAIL: agent not found"

    except Exception as exc:

        results["endpoint"] = f"FAIL: {exc}"

        return results

    try:

        response = client.agents.run(

            agent_name=agent_name,

            messages=[{

                "role": "user",

                "content": "Return OK and identify which configured data source you used.",

            }],

        ) # conceptual invocation

        results["invocation"] = "PASS" if response else "FAIL: no response"

        results["services"] = "PASS" # evaluate expected probe evidence here

    except Exception as exc:

        results["invocation"] = f"FAIL: {exc}"

    results["report"] = "PASS"

    return results

if __name__ == "__main__":

    print(validate_agent_handover(

        os.environ["FOUNDRY_PROJECT_ENDPOINT"],

        os.environ["FOUNDRY_AGENT_NAME"],

    ))

Configure passwordless identity

ManagedIdentityCredential deliberately restricts the validator to Azure managed identity. It does not fall back to an Azure CLI login, developer credential, environment secret, or interactive browser. This makes the acceptance result evidence for the application workload’s real runtime identity.

Before running the validator in Azure, enable managed identity on the hosting resource and grant that identity the least-privileged roles required to access the Foundry project and invoke the agent. The agent’s own connections remain configured and authorized separately by the infrastructure team.

Example .env file

Use an environment file only for local convenience. Do not commit it. In Azure App Service, Azure Functions, Azure Container Apps, or AKS, provide the same values through the platform’s application settings or workload configuration.

# .env contains identifiers and configuration—not a password or API key.

FOUNDRY_PROJECT_ENDPOINT=https://<project-name>.<region>.api.azureml.ms

FOUNDRY_AGENT_NAME=<handed-over-agent-name>

# Set only when the workload uses a user-assigned managed identity.

AZURE_CLIENT_ID=<managed-identity-client-id>

Run the validator

# Install the SDK packages selected by your application team.

python -m pip install azure-identity azure-ai-projects python-dotenv

# Run from an Azure compute resource that has managed identity enabled.

# System-assigned identity: omit AZURE_CLIENT_ID.

python validator.py

# Azure-hosted workload: enable managed identity, assign the required RBAC roles,

# add the non-secret environment variables above, and start the same command.

python validator.py

ManagedIdentityCredential is intended to run on Azure compute with managed identity enabled. Run the handover gate from the application’s deployed environment so the result proves the intended identity, network route, DNS, private endpoints, and role assignments. For a user-assigned identity, set AZURE_CLIENT_ID to its client ID; for a system-assigned identity, omit that setting.

Probe prompts for configured services

If Azure AI Search, Azure Cosmos DB, or Azure Blob Storage are part of the agent’s contract, the validator sends focused probe prompts through the agent and records the outcome. If a service is not part of the contract, that probe is marked SKIP rather than assumed.

Service	Example probe prompt	Expected evidence
Azure AI Search	“List the top three documents in your index about [known topic]. Include titles and short snippets.”	Document titles and relevant snippets from known indexed content.
Azure Cosmos DB	“What is the most recent record timestamp in the configured data source?”	A valid timestamp that can be checked against a known range.
Azure Blob Storage	“Confirm that you can access the configuration file at [known path]. Return metadata only.”	Expected file name, content type, last-modified time, or a safe summary.

These probes test the agent’s actual tool connections without requiring the validator to authenticate to those services directly. A strong acceptance result records the prompt, status, timestamp, correlation or run identifier, and a small piece of expected evidence—without storing sensitive response content.

The result gives both teams the same evidence. Instead of asking whether the agent was deployed, they can ask the better question:

Is the agent ready for the application to use?

Closing thought

Deploying an agent is only the first half of the journey. The real test is whether the application team can use it confidently through the expected

identity, network, and capabilities.

The Agent Acceptance Gateway creates that moment of confidence. It brings both teams together, checks what matters, shows what still needs attention, and turns the handover into a shared decision.

The goal is simple: do not hand over an agent just because it exists. Hand it over when both teams have clear evidence that it is ready.

Cohere Embed V5 brings more relevant retrieval to Microsoft Foundry

RashaudSavage — Wed, 30 Sep 2026 16:59:02 GMT

Today, we’re introducing Cohere Embed V5 Pro and Embed V5 Fast in Microsoft Foundry, bringing stronger multimodal retrieval to enterprise search and AI applications. With Fast and Pro options, support for more than 100 languages, and flexible vector sizes, Embed V5 gives developers more control over retrieval quality and efficiency.

Find relevant information across text and images

Enterprise knowledge spans documents, wikis, reports, and images. Embed V5 helps applications find relevant information across these sources and languages, supplying richer context to search experiences and retrieval-augmented generation (RAG) workflows.

According to Cohere, Embed V5 surpasses Embed V4 across multimodal, multilingual, and domain-specific search and retrieval benchmarks. For AI agents, better retrieval can mean more useful context, better answers, and fewer unnecessary tokens spent processing irrelevant information.

Choose Fast or Pro to match your workload

Embed V5 introduces two options. Fast is designed for latency-sensitive workloads, such as interactive search and responsive agent experiences. Pro prioritizes maximum retrieval quality for applications where finding the most relevant information is critical.

Both share a single embedding space, supporting interchangeable use and giving teams flexibility as their application requirements evolve.

Scale retrieval with more control over cost

A broader range of Matryoshka embedding dimensions lets developers choose smaller vectors to reduce storage requirements and retrieval latency while targeting the quality their application needs. High-throughput batch processing supports embedding large content collections for indexing at scale.

Beyond search and RAG, Embed V5 also supports classification, clustering, recommendations, and similarity matching, helping teams put enterprise data to work across more applications.

Get started in Microsoft Foundry

Explore Cohere Embed V5 for your next search or RAG application. Evaluate Fast and Pro on your own enterprise content to find the right balance of relevance, latency, and cost.

Explore Cohere Embed V5 in Microsoft Foundry.

Cohere-Embed-V5-Pro | Model Catalog | Microsoft Foundry

Cohere-Embed-V5-Fast | Model Catalog | Microsoft Foundry

Let the query decide: introducing auto retrieval reasoning effort

amaias — Wed, 30 Sep 2026 16:00:00 GMT

Introducing `auto`: a new retrieval reasoning effort mode that recovers 97.6% of the evidence recall of the highest effort level at 46.9% fewer query planning tokens.

Enterprise retrieval workloads mix simple lookup queries with complex questions that require searching across many documents. Today, customers of Foundry IQ (Azure AI Search) Knowledge Bases must choose a fixed retrieval reasoning effort level: minimal, low or medium. That forces customers to make a trade-off across their workload: pay for deeper retrieval even on simple questions, or reduce costs and risk missing the evidence needed to answer more complex ones.

The new auto retrieval reasoning effort, available in the August 2026 preview, reduces the need to choose one fixed effort level for mixed workloads. auto assesses each query and adjusts the query planning and iterative search effort it receives. Straightforward questions take a faster, lower-cost path, while complex questions get the depth of multi-step retrieval, without manual tuning.

Customers can still use minimal, low or medium when their workloads are predictable. For mixed workloads, auto lets each query determine the effort it needs.

Near-identical retrieval quality, lower cost

Figure 1 shows the central trade-off across retrieval reasoning effort tiers. Higher fixed effort generally retrieves more complete evidence, but it also consumes more tokens. auto assigns lower effort to straightforward queries and reserves medium (the highest retrieval reasoning effort) for the queries that need it, retaining 97.6% of its evidence recall while reducing average query planning token usage by 46.9%.

Figure 1. Average evidence recall by retrieval reasoning effort using GPT 5.4 mini for query planning. Evidence recall measures how much of the evidence needed to answer a question is found in the retrieved documents.

Figure 2 translates the token savings into customer cost. With GPT-5.4 mini, auto reduces the average cost of 1,000 queries from $13.82 to $9.03 (a saving of $4.79, or 34.7%) compared with running every query at medium effort. Because search costs remain fixed, the dollar savings increase when a more expensive model is used for query planning.

Figure 2. Total query cost ($/1,000 queries) per model and retrieval reasoning effort. Cost estimates use Azure OpenAI pricing for query planning and Azure AI Search pricing for ranker/search as of September 2026.

Effort that scales with the question

Not every question needs the same retrieval depth. The auto retrieval reasoning effort matches retrieval depth to each question: minimal or low for straightforward requests, and medium when the question requires additional planning and iterative search. Figure 3 shows how those routing decisions vary across datasets. In the evaluated enterprise workloads, most queries follow the faster minimal or low paths, while datasets containing more complex questions send a larger share of queries to medium.

Figure 3. Distribution of effort tiers selected by auto across the evaluated enterprise queries. KS = knowledge source.

This routing behavior translates directly into the token savings shown in Figure 4. auto remains close to medium in evidence recall while using fewer tokens across every evaluated dataset. The datasets that route a larger share of queries to minimal achieve the greatest savings, because more queries avoid reasoning tokens when additional retrieval depth is unlikely to help. For a direct lookup query, this can mean using no reasoning tokens instead of the approximately 14,000 consumed by the average medium run.

Figure 4. Evidence recall for auto and medium by dataset.

The same adaptivity works in reverse. On BrowseComp, a public benchmark of deliberately difficult, multi-step research questions, auto escalates 78% of queries to medium. Its resulting evidence recall remains within a few points of running every BrowseComp query at medium, showing that the router increases effort when simpler retrieval paths are unlikely to be sufficient.

It is worth knowing the edges. Choose medium when most queries are complex, multi-step questions and retrieval quality matters more than cost. Choose minimal when the workload consists almost entirely of direct lookups. For mixed workloads, auto is the recommended starting point because it preserves close to medium-level evidence recall while reducing average cost without per-query configuration.

Figure 5 shows how auto applies per-query routing across three queries of increasing complexity: a direct lookup, a simple multi-hop question, and a complex multi-hop question. auto selects the lowest retrieval effort expected to preserve evidence quality, escalating from minimal to low or medium as the query requires more planning and retrieval steps.

Figure 5. Example routing decisions for three query types of increasing complexity.

Get started

auto retrieval reasoning effort is now available in preview for Foundry IQ (Azure AI Search) Knowledge Bases. For workloads that mix straightforward lookups with complex questions, auto provides a starting point without requiring you to choose a fixed effort level for every query.

If you already have a knowledge base, set retrievalReasoningEffort to auto as its default, or specify auto on an individual retrieve request to override that default for the request. See Set the retrieval reasoning effort for configuration details and supported versions.

If you are new to knowledge bases, follow the agentic retrieval quickstart to create one and run your first queries. Then try auto on a representative set of your own questions and compare the retrieved evidence and query-planning token usage with a fixed effort level.

Appendix

Similarly to our previous blog posts (Foundry IQ: Improve recall by up to 54% with knowledge bases | Microsoft Community Hub, Up to 40% better relevance for complex queries with new agentic retrieval engine) we evaluated auto on several benchmark query sets.

Customer datasets: customer-provided corporate and member-document collections in English, covering domains such as oil and gas corporate reports and health-insurance member documents. We evaluate them as single knowledge source (KS) workloads over chunked PDF indexes.
SEC: SEC filings of US public companies in English, evaluated in two variants: a single knowledge source setup spanning all sectors, and a routing setup with one knowledge source per Global Industry Classification Standard (GICS) sector.
MIML: a multi-industry, multi-language corporate document benchmark in English, French, and Simplified Chinese. We evaluate both single knowledge source variants restricted to one language-industry slice and a routing variant spanning multiple languages and industries.
BrowseComp: We use the 830 human-verified BrowseComp-Plus queries and indexed the full corpus, including distractor documents, into 512-token chunks with OpenAI text-embedding-3-large. To support continuous evidence-recall measurement, we decompose each question into multiple atomic factoid evidence nuggets. Evaluations use a two-agent neutral user plus search agent configuration with a standard system prompt and per question effort capped at 40 tool calls.

Introducing GPT-6.1 Sol in Microsoft Foundry: Advanced intelligence, optimized for production agents

Naomi Moneypenny — Wed, 30 Sep 2026 00:08:07 GMT

Today, OpenAI's GPT-6.1 Sol is generally available in Microsoft Foundry. GPT-6.1 Sol is an upgrade to GPT-6 Sol, delivering substantial improvements in agentic coding, computer use, and professional work, with performance approaching GPT-6 Astra across these evaluations. It offers a new balance of capability and cost for important work at higher frequency, making it more affordable for developers to build and run very capable agents at scale.

Just one week after GPT-6 Sol and Luna joined our generally available lineup, this release continues the momentum of the GPT-6 series in Foundry: models that produce less noise and are more capable of completing full tasks with agents. GPT-6.1 Sol carries that progress forward for teams whose production workloads run all day, every day.

What's new in GPT-6.1 Sol

GPT-6.1 Sol advances the three capabilities that matter most for production agents:

Agentic coding. GPT-6.1 Sol plans, edits, tests, and iterates across a codebase with improved performance on complex tasks and extended workflows involving multiple tool calls.

More capable computer use. Improved reliability navigating real interfaces help agents operate the workflows that span multiple application steps, with permissions and human oversight suited to the task.

Deeper Professional work. Stronger performance on the analysis, drafting, and multi-step knowledge tasks that make up daily enterprise work, with improvements in factual accuracy those workflows demand.

GPT-6.1 Sol accepts text and image inputs and produces text, with a total context window of up to 1M tokens. This provides room to bring large codebases and document sets, within the model’s context limits. Flexible reasoning lets teams tailor the model’s depth of analysis to the needs of each task. This can help teams use tokens more efficiently by reserving deeper reasoning for the work that needs it.

As in last Tuesday's announcement, the starting point is quality alongside cost per task, not model capability in isolation. Use that lens to evaluate GPT-6.1 Sol against the needs of your own production workloads.

Where to put GPT-6.1 Sol to work

Because these gains compound in agentic loops, valuable use cases include agent workloads that run frequently:

Software engineering agents that triage issues, implement changes across a repository, respond to code review, and keep CI green, with costs that support running them on every pull request, not just the hard ones.
Computer-use agents that complete back-office processes end to end: updating records across line-of-business systems, reconciling data between applications, and handling workflows in browsers.
Professional work agents for research synthesis, contract and document review, financial analysis, and report generation, where work recurs weekly or daily and rewards consistent quality per task.
High-frequency customer and employee workflows, where GPT-6.1 Sol's capability-cost balance lets teams upgrade the intelligence behind every interaction without upgrading the budget.

The right intelligence behind every agent

The right model for a job should be determined through evaluations. GPT-6.1 Sol joins GPT-6 Astra, GPT-6 Sol, and GPT-6 Luna in the Foundry model catalog: start with GPT-6 Astra for the most demanding reasoning, use GPT-6.1 Sol as the new default for production agents and complex workflows, and scale high-volume data and preparatory tasks with Luna. As with the rest of the GPT-6 series, look beyond price per token to cost per task, alongside quality and reliability. Foundry brings evaluation, tracing, and monitoring together so teams can make that call with evidence, then switch models without re-platforming.

Deploy it your way

With GPT-6.1 Sol in Microsoft Foundry, teams can choose how they scale and where processing takes place. Standard deployment offers flexible capacity billed by usage across Global regions and US, EU, and APAC Data Zones. Provisioned Throughput is available at launch through Global and US Data Zone deployments, providing reserved capacity for critical production workloads. Additional regions and deployment options will follow soon.

That is the Foundry advantage: the flexibility to balance capacity, performance, and data residency requirements within one enterprise platform. Teams can match each workload to a supported combination of serving option and processing location, rather than apply the same deployment approach to every application.

GPT-6.1 Sol Pricing*

Model	Deployment	Context Length	Pricing (USD $/million tokens)
Model	Deployment	Context Length	Input	Cached Input	Cached Writes	Output
GPT-6.1 Sol	Global Standard	Short context	$2.00	$0.10	$2.50	$10.00
	Global Standard	Long context	$4.00	$0.20	$5.00	$15.00
	Data Zone Standard (US)	Short context	$2.20	$0.11	$2.75	$11.00
	Data Zone Standard (US)	Long context	$4.40	$0.22	$5.50	$16.50
	Data Zone Standard (EU)	Short context	$2.40	$0.12	$3.00	$12.00
	Data Zone Standard (EU)	Long context	$4.80	$0.24	$6.00	$18.00
	Data Zone Standard (APAC)	Short context	$2.40	$0.12	$3.00	$12.00
		Long context	$4.80	$0.24	$6.00	$18.00

*Prices shown are for Standard deployments. Provisioned Throughput pricing varies by deployment type. For each offer, the US Data Zone is priced at a 10% premium to Global, and the EU and APAC Data Zones are priced at a 20% premium to Global. For current rates and terms, see the Azure OpenAI pricing page., see the Azure OpenAI pricing page.

Build safer agents on Foundry

GPT-6.1 Sol runs with the same layered protections as the GPT-6 series in Foundry: alignment training in the model, content filters and guardrails on prompts and outputs, prompt injection mitigations on tool calls and responses, and enterprise identity and access controls governing what agents can reach. Microsoft Purview applies data policies and human checkpoints at every phase.

Start building

Your next agent needs more than a powerful model. Foundry pairs GPT-6.1 Sol with the deployment flexibility, observability, and enterprise controls to take it from first workload to production at scale. Explore GPT-6.1 Sol in the Microsoft Foundry model catalog and evaluate where it fits in your next agent.

Claude Sonnet 5.5 is now available in Microsoft Foundry

amar_badal — Mon, 28 Sep 2026 18:18:01 GMT

Today, Claude Sonnet 5.5 is available in Microsoft Foundry, bringing Anthropic’s latest Sonnet model to developers building AI applications and agents.

Claude Sonnet 5.5 is a smarter, more efficient Sonnet, delivering a clear step forward from Sonnet 5 on coding and knowledge work with a lower cost per task for most work at faster speed. For teams already building on Sonnet, it is a natural upgrade that can deliver stronger results.

More intelligence with fewer tokens

Claude Sonnet 5.5 improves Sonnet capabilities while completing work more efficiently for most tasks. That means teams can take on more complex tasks without necessarily paying for more model usage.

For developers, that translates into a better balance of quality and economics for everyday production workloads.

Stronger for focused coding

Claude Sonnet 5.5 is particularly suited for well-scoped coding tasks that fit into a larger development workflow.

Use it to:

Build and fix features within the same coding session

Verify implementations against requirements

Support everyday coding and knowledge-work tasks

As agent systems become more modular, models increasingly need to perform individual tasks reliably and efficiently as part of a larger workflow. Claude Sonnet 5.5 is designed for exactly that role.

Pricing

Model	Deployment Type	Input	Ouput	Availability
Claude Sonnet 5.5	Global Standard, US Datazone	$2	$10	GA, Hosted on Azure
Claude Sonnet 5.5	Global Standard	$2	$10	GA, Hosted on Anthropic Infrastructure

Build with Claude Sonnet 5.5 in Microsoft Foundry

With Claude Sonnet 5.5 in Microsoft Foundry, developers have another strong option for balancing capability, efficiency, and cost across coding and knowledge-work applications.

Try it today

Introducing voice agents in Microsoft Foundry

Yiwen_Rong — Mon, 28 Sep 2026 18:09:24 GMT

Voice is quickly becoming a major way people interact with AI agents. Recent research found that 14% of users already prefer speaking with generative AI rather than typing¹, signaling that voice interactions are moving beyond early-adopter use cases and toward the mainstream. As adoption grows, users increasingly expect conversations that feel natural, responsive, and capable of carrying out real work.

From customer support and employee assistance to learning companions and industry-specific workflows, organizations want agents that can listen, reason, and respond in real time. Yet many voice solutions still require developers to assemble separate speech, orchestration, deployment, monitoring, and evaluation components before they can reach production. As a result, teams often spend more time integrating infrastructure than improving the agent experience itself.

Today's announcement introduces voice agents in Microsoft Foundry Agent Service, a voice-native approach that brings real-time speech, agent capabilities, deployment workflows, observability, and evaluation together in a single platform.

Developers can build voice experiences with sub-second latency, multilingual support, integrated knowledge and tools, deployment channels, and built-in lifecycle management, all within Microsoft Foundry. This includes broad model choice for text-to-speech and speech-to-text models from providers like OpenAI, Microsoft AI and Microsoft Azure. Voice agents support the same development workflows developers already use in Microsoft Foundry Agent Service, making it easy to bring real-time voice capabilities to existing agent experiences.

From voice API to voice-native agents – choosing the right path for building voice experiences

Voice experiences have evolved from standalone speech applications into full agent systems. As a result, developers today can choose from multiple architectures depending on how much of the agent lifecycle they want the platform to manage.

The three approaches shown above reflect different stages of voice application maturity. Some teams only need a real-time voice runtime. Others have already built agents and want to make those experiences conversational. Increasingly, however, organizations are looking for a voice solution that spans development, deployment, monitoring, evaluation, and optimization from a single platform.

The important distinction is not simply how speech is added to an application, but where responsibility for the voice experience lives. Earlier approaches give developers maximum flexibility to assemble their own architecture. As requirements grow, however, teams often need deeper lifecycle capabilities, including observability, evaluation, governance, deployment workflows, and operational tooling. While many developers can build a compelling voice agent demo, operating voice agents in production requires visibility into every interaction, the ability to evaluate agent behavior, diagnose failures, and a system to continuously improve quality, latency, and cost. Those requirements become especially important for customer-facing experiences such as contact centers, employee assistants, and business process automation.

Voice agents in Microsoft Foundry were designed around the needs of production applications. Rather than treating voice as a channel layered onto an existing agent, voice becomes a native part of how agents are built, deployed, observed, evaluated, and optimized. Real-time speech, agent capabilities, deployment workflows, observability, and evaluation are brought together through a single platform and SDK. Teams can tailor experiences to their specific scenarios by choosing the models that best fit their workloads, improving recognition for domain-specific vocabulary through speech customization, creating custom voices, and adding photo or video avatars that reflect their brand. Voice agents also include newly expanded standard talking-head avatars that can be enabled directly in the Foundry playground, making it easy to create a visual agent experience without developing custom avatar assets. Developers can start with these out-of-the-box avatars and evolve to custom voice and avatar experiences as their scenarios grow.

All three approaches remain supported and serve different customer needs. For new enterprise voice workloads, however, Microsoft recommends voice agents in Foundry as the preferred path forward because it provides one of the most complete voice-agent lifecycle, from first conversation to production operations.

Build, test, and deploy voice agents with familiar developer tools

Voice agents are integrated into the developer workflows teams already use. With new support across AZD AI and the Foundry Toolkit for Visual Studio Code (coming soon), developers can create, configure, test, debug, source-control, and deploy voice agents without stitching together separate tools or building custom voice testing applications.

AZD AI introduces a dedicated workflow for voice agents. Developers can initialize a voice agent project, manage agent definitions and deployment configuration in source control, and deploy through the standard AZD lifecycle. Voice agents remain a dedicated application type because their models, runtime requirements, testing workflows, and deployment paths are optimized specifically for spoken interaction.

Inside Visual Studio Code, developers can complete the entire development loop without leaving the editor. Teams can create and configure a voice agent, start a voice session, interact through natural speech, inspect transcripts and runtime events, and debug latency or connection issues from the same environment where they author the agent.

Together, these capabilities reduce the time between writing an agent and having a real conversation with it, helping teams move from experimentation to production more quickly.

Operate voice agents with voice native lifecycle management

Building a great voice experience is only the first step. Once voice agents reach production, teams need to know why conversations succeed or fail, where latency is introduced, how users interact with the agent, and how the experience can be improved over time.

Voice interactions introduce challenges that text agents never encounter. Speech-recognition errors, interruptions, turn-taking issues, and tool delays can all impact the conversation experience. Without observability built into the platform, teams often rely on replaying calls, reviewing logs across multiple systems, and manually diagnosing issues.

Voice agents in Microsoft Foundry are built with observability, evaluation, and optimization by default. They use the same Foundry Observability experience as every other Foundry agents, so there is nothing separate to configure. Each conversation is stored as a trace, and the Foundry evaluation framework runs against those traces in the same way it runs for text agents, making it easier to understand agent behavior, diagnose issues, and measure quality over time.

These capabilities feed the continuous improvement loop that runs across Foundry:

Tracing and monitoring provide visibility into every interaction, helping teams understand user behavior, diagnose failures, and find performance bottlenecks.
Evaluation measures conversation quality against production traces, validates expected behavior, compares versions, and catches regressions before they reach users.
Insights and optimization help teams improve quality, reliability, latency, and cost over time

Voice agents are typically built to complete specific, time-sensitive tasks. Developers need to measure whether the agent did its job and followed the rules, and general-purpose evaluations alone cannot tell them that. Foundry rubric evaluators score conversations against criteria written for a specific agent. You list what matters, such as "confirmed the date, time, and party size" or "did not book a party larger than eight," assign each criterion a weight, and an LLM judge scores every conversation against that list.

Foundry can generate the rubric from the agent's instructions and production traces, and run it continuously against live traffic, so teams know when an agent skips a step or breaks a policy before customers do. Early customers building voice agents in Foundry are already using rubric evaluators this way.

Together, these capabilities help teams move beyond debugging individual conversations and establish a repeatable process for operating and improving voice agents in production.

Voice Agents in Microsoft Foundry bring together real-time speech, knowledge, tools, deployment workflows, observability, and evaluation into a unified experience. Whether you're adding voice to customer support, employee assistants, learning applications, or industry workflows, Voice agents provide the recommended path for building production-grade voice experiences on Microsoft Foundry.

Get started with voice agents in Microsoft Foundry

Getting started only takes a few steps:

Open the Microsoft Foundry portal and create a new agent, select “Voice”.
Choose a voice-optimized model and configure your agent's instructions.
Connect knowledge sources and tools as needed for your scenario.
Test conversations directly in the built-in voice playground.
Deploy to your desired channels and monitor performance using built-in observability and evaluation capabilities.

¹Study done from Jabra and LSE. The Globe and Mail (October 2025)

Insights in Foundry Turns Agent Traces into Action

katelynrothney — Thu, 24 Sep 2026 17:30:00 GMT

If you’ve shipped an agent to production, you’ve probably watched it repeat a mistake that no dashboard flagged and no evaluation caught. A single agent can generate thousands of traces, and predefined evaluations can only detect the problems a team already knew to look for. That leaves developers searching manually for patterns, reconstructing failures, and deciding what deserves attention.

Today, we’re announcing the public preview of Insights in Foundry. Insights analyzes agent traces to surface recurring and previously unknown behavior, explains the likely cause with supporting evidence, and recommends what developers should do next.

Insights builds on the tracing, monitoring, and evaluation capabilities already in Foundry. It helps teams move from observing production activity to understanding where an agent needs attention, and whether the right response is a targeted fix, a new evaluation, or deeper iteration with Agent Optimizer.

Discover behavior you didn’t know to look for

Consider an order-support agent. When an order-status lookup fails, the agent still tells the customer that the order was delivered. The answer sounds confident, and the request appears to have completed. In a single trace, it looks like an isolated mistake. Across hundreds of requests, it becomes a pattern the team needs to understand.

Insights analyzes the agent traces stored in the Azure Monitor Application Insights resource connected to your Foundry project. You don’t need to create an evaluator first. Insights looks across executions for patterns that would be difficult to find by reading traces one at a time.

Depending on the available evidence and configuration, findings can cover tool-call failures, context handling, output quality, reliability, latency, or unnecessary token use. The value isn’t another count of errors. Each finding explains a recurring behavior, shows the evidence behind it, and points to a useful next step.

Insights complements monitoring and evaluation rather than replacing them. Your existing evaluators keep testing the requirements you already know about, while Insights helps identify new behaviors worth investigating and turning into future evaluation coverage.

Understand each finding through its evidence

Each Insight brings a finding and its supporting traces together in one place. It can include a description, category, severity, status, affected agent version, linked traces, representative examples, and a likely cause or recommended action.

In the order-support example, a useful finding goes beyond “the lookup tool failed.” The representative traces show the lookup failure followed by the unsupported delivery claim. A developer can inspect the request, the tool calls and their results, and the final response, then compare them with successful executions.

That distinction matters. An intermediate tool error isn’t automatically a failed task, and a plausible explanation isn’t a confirmed root cause. Insights gives developers a grounded starting point for investigation. The team then validates whether the behavior is real, how it affects the workflow, and who owns the fix.

The quality of the evidence depends on the quality of the telemetry. Missing agent identity, version information, spans, or content can limit the analysis. Infrastructure availability, dependency outages, quotas, and service-level alerts remain with Azure Monitor and the teams that own those systems.

Choose the right action for each finding

A confirmed finding can lead to different responses:

Make a targeted change. Correct an instruction, tool description, code path, or workflow when the evidence points to a specific problem.
Add evaluation coverage. Create or update an evaluator or dataset so the behavior becomes an explicit test before and after a change.
Route the issue to its owner. If a dependency or platform failure is the cause, work with that owner instead of changing an agent that can’t fix the underlying issue.
Explore alternatives. Use an optimization workflow such as Agent Optimizer when improving the behavior means generating and comparing multiple candidates.

For prompt agents and code-based hosted agents, an Insight can include a proposed prompt or code change. When editable instructions or code aren’t available, the recommendation provides investigation or remediation guidance instead.

Insights proposes evidence-backed recommendations. Developers decide what changes and when. Nothing ships without human review.

Insights doesn’t open pull requests, build evaluators, or deploy fixes on its own. Developers review each proposal and apply it through their normal source-control, evaluation, approval, and deployment processes.

Use Agent Optimizer when the problem calls for iteration

Some findings call for a focused correction. Others raise a broader question: which change will improve this behavior without introducing a regression somewhere else?

Agent Optimizer is built for that question. It generates candidate improvements and evaluates them against the goals you set. A confirmed Insight helps a developer define the behavior to improve, select representative evidence, and decide how to measure success before starting an optimization run. The two capabilities are complementary. Insights discovers and helps diagnose, and Agent Optimizer supports iterative improvement. Not every Insight needs an optimization run, and optimization doesn’t need to begin with an Insight. Moving from one to the other is a developer decision, not an automatic handoff.

In the order-support example, the team might test whether an instruction change prevents unsupported delivery claims while preserving correct answers when the lookup succeeds. Whether they write the change directly or explore options with Agent Optimizer, they validate the result before deployment and review fresh production evidence afterward.

Start with one agent and one finding

Pick one finding your team can validate. Inspect its representative traces, choose the right response, and test whether the change improves the behavior. The first scan reviews the last seven days of traces, and you can run analysis again on demand after that. A scan that returns no findings doesn’t mean an agent is free of problems, so keep your existing monitoring and evaluations in place.

Insights is decision support. It doesn’t replace production monitoring, evaluation, security review, or change management. Before you run a scan, check the preview documentation for supported configurations, limitations, and the model and telemetry costs that analysis can incur.

The goal isn’t more findings to read. It’s a repeatable way to turn production evidence into the next improvement decision, and then confirm that decision with new evidence.

What’s next

Learn more

Explore the full set of Microsoft Foundry capabilities announced today for operating agents in production: https://aka.ms/foundrysept2026

Join us for Foundry Friday AMA | September 25, 2026

Bring your questions. The team behind Agent Optimizer and Insights in Foundry is hosting a live AMA this Friday. Post your questions now or ask them live during the session: https://aka.ms/FoundryFriday-Optimizer-Insights

See it live at Microsoft Ignite

Join us at Microsoft Ignite, November 17–20 in San Francisco, to explore the latest Microsoft Foundry innovations and see AI agents in action. Sessions to check out:

DeepSeek-V4.1-Flash is coming to Microsoft Foundry

RashaudSavage — Fri, 25 Sep 2026 15:58:46 GMT

Agentic applications increasingly need to process entire code repositories, large document collections, images, tool results, and long-running conversation histories. As that context grows, the key-value, or KV, cache maintained during inference can become a major constraint on cost, memory, and deployment scale.

DeepSeek-V4.1-Flash is coming to Foundry Models. This multimodal Mixture-of-Experts model supports text and image inputs, a context window of up to one million tokens, and continuously configurable reasoning effort. Developers can deploy the model either Direct from Azure (public preview) or through Fireworks on Foundry (generally available), choosing the consumption path that best fits their performance, operational, and purchasing requirements.

More capability and control for agentic applications

DeepSeek-V4.1-Flash is more than a fine-tuned version of V4-Flash. It introduces a Causal Encoder-Decoder architecture in which a 20-layer causal encoder processes the input and a 20-layer decoder generates the output. The model has a 552-billion-parameter backbone, activating 8 billion parameters during prefill and 16 billion during decoding.

The model also expands beyond the text-only capabilities of V4-Flash. It adds native image understanding and replaces three fixed reasoning modes with a continuous reasoning-effort setting from 1 to 100.

Developers can use lower effort for tasks such as routing, classification, and extraction, then increase effort for planning, coding, debugging, and multi-step tool use, without changing model deployments.

Efficient long-context AI starts with a smaller memory footprint

Long-context agents need to retain code, documents, images, tool results, and conversation history during inference. As that context grows, the KV cache can become a significant memory and scaling constraint.

DeepSeek reports that V4.1-Flash requires 890 bytes of global KV cache per token, approximately one quarter of V4-Flash’s footprint. Its architecture is particularly relevant to workloads that read significantly more information than they generate.

Figure 1. Global KV cache per token across DeepSeek model generations. Values and fold reductions as reported by DeepSeek.

For developers, the practical opportunity is to retain more useful context and support context-heavy workloads more efficiently. Actual latency, concurrency, infrastructure requirements, and cost will depend on the application, serving configuration, traffic profile, and deployment path.

Stronger performance on agentic workloads

DeepSeek-V4.1-Flash pairs its more efficient long-context architecture with improved performance on agentic tasks. On DeepSeek’s published evaluations, V4.1-Flash improves on V4-Flash across the reported agentic benchmarks, including coding, terminal, security, tool-use, and automation scenarios.

Figure 2. DeepSeek-V4-Flash, V4-Pro, and V4.1-Flash on agentic and reasoning benchmarks, all at maximum reasoning effort. Sorted by the gain from V4-Flash to V4.1-Flash.

On DeepSWE v1.1, V4.1-Flash resolves 74.2% of tasks, compared with 54.4% for V4-Flash. DeepSeek also reports improvements on Terminal-Bench 2.1, CyberGym, AutomationBench, and Agent’s Last Exam. Results can vary based on prompts, orchestration framework, context construction, deployment, and workload. Teams should evaluate model quality, latency, and cost using their own application data. Competitor or comparative figures reported by DeepSeek have not been independently verified by Microsoft.

What this means for developers

Put together, the architecture and the benchmark profile point to a specific set of workloads where DeepSeek-V4.1-Flash is a strong fit:

Repository-scale coding agents that load large codebases, run tools in a terminal, and iterate over many steps.
Document-heavy and multimodal analysis, including PDFs, charts, screenshots, and scanned forms, where image inputs and a 1M-token window remove the need for aggressive chunking.
Long-running automation and research agents that accumulate tool outputs and conversation history over hundreds of turns.
Cost-sensitive, high-concurrency deployments where the smaller KV cache translates directly into more sessions per GPU.

The reasoning effort control deserves special attention. Rather than choosing between a fast model and a thinking model, you can run one deployment and set effort per request: low for routing and extraction, high for planning and debugging. That simplifies orchestration compared with the V4 Flash and V4 Pro pairing, and it gives you a single, continuous knob to tune against quality and latency targets in Foundry evaluations.

One model, two deployment paths

As the open model ecosystem evolves, teams increasingly want flexibility not only in which model they use, but also in how they deploy and consume it. With DeepSeek-V4.1-Flash, developers can select between two paths in Microsoft Foundry based on the needs of their application.

Deployment path	Consider this path when you prioritize
Direct from Azure	A native Azure-hosted and Microsoft-supported model offering, billed through your Azure subscription and covered by Azure service-level agreements.
Fireworks on Foundry	Optimized open-model inference, token caching, broader Provisioned Throughput support, or custom-weight deployment capabilities on Fireworks AI's inference stack running natively in Azure.

The right path depends on the workload. Developers should evaluate model quality, latency, throughput, deployment shape, data requirements, and cost using their own application traffic and evaluation datasets.

Other models now available through Fireworks on Foundry

Microsoft Foundry is also expanding model choice through Fireworks with additional global deployment options for leading open models. These updates give developers more flexibility to evaluate and deploy models for agentic, coding, reasoning, and long-context workloads through the same Foundry experience.

DeepSeek-V4.1-Flash is available through Fireworks AI with PayGo Global and Provisioned Throughput deployment options.
GLM-5.3-Flash is available through Fireworks with a Global deployment option.
Kimi K3 now adds a Global deployment option through Fireworks, expanding beyond its previously available Data Zone deployment.

Availability, deployment options, and pricing may vary by model. Review each model card in the Microsoft Foundry model catalog for the latest details before deployment.

Pricing

Fireworks on Foundry	Deployment Region	Input/M tokens	Cache/M tokens	Output/M tokens
GLM 5.3	Standard Global	$1.75	$0.33	$5.50
GLM 5.3	US Data Zone	$2.10	$0.39	$6.60
GLM 5.3 Flash	Standard Global	$0.19	$0.04	$6.25
Kimi K3	Standard Global	$3.00	$0.30	$15.00
Kimi K3	US Data Zone	$3.30	$0.33	$16.50

DeepSeek V4.1 Flash Pricing

Deployment path	Deployment Region	Input/M tokens	Output/M tokens	Cache/M tokens
Direct From Azure	Global Standard	$0.3	$1.2	$0.006
Fireworks on Foundry	Standard Global	$0.37	$1.50	$0.007

A unified Foundry experience

Whichever deployment path teams choose, Microsoft Foundry provides a common environment for discovering models, comparing options, evaluating them against application-specific data, and managing AI development with integrated governance and observability.

This lets developers focus on selecting the right model and deployment path for their scenario instead of assembling fragmented tools and separate operational environments. Model discovery, evaluation, and management stay centered in Foundry, while the consumption path adapts to the team.

Getting started

Explore DeepSeek-V4.1-Flash in the Microsoft Foundry model catalog through either deployment path:

DeepSeek-V4.1-Flash, Direct from Azure: DeepSeek-V4.1-Flash | Model Catalog | Microsoft Foundry
DeepSeek-V4.1-Flash, Fireworks on Foundry: FW-DeepSeek-V4.1-Flash | Model Catalog | Microsoft Foundry

You can then:

Compare the available deployment paths
Evaluate the model using your own datasets and agent scaffolds, sweeping the reasoning effort setting
Select the deployment option aligned with your operational and performance requirements
Begin integrating the model into long-context, multimodal, and agentic applications

For the full architecture description and complete benchmark tables, see the DeepSeek-V4.1-Flash model card on Hugging Face

Claude Opus 5.5 comes to Microsoft Foundry for long-running coding and knowledge work

amar_badal — Tue, 22 Sep 2026 16:47:05 GMT

AI models are increasingly taking on work that extends far beyond a single prompt: building a feature across a codebase, investigating a complex issue, synthesizing hundreds of pages of information, or working through a multi-step business process.

As that work gets longer, raw intelligence is only part of what matters. The model also needs to stay focused, make good decisions along the way, communicate what it is doing, and produce work that people can quickly review and use. Today, Claude Opus 5.5 is available in Microsoft Foundry, bringing Anthropic’s most capable Opus model to developers and enterprises building AI applications and agents.

Claude Opus 5.5 is designed for everyday complex work. It advances Opus 5 across agentic coding, knowledge work, and long-running tasks while making it easier for people to understand what the model did, what it found, and what it needs next. Claude Opus 5.5 also does more with fewer tokens. Lower per-token prices and much cheaper cache reads stack on top of the efficiency gains.

Built for work that takes time

Writing a function is one thing. Building a feature that touches multiple services, tracing a production issue across a large repository, or carrying a task from investigation through implementation and validation is something else entirely. Claude Opus 5.5 is designed for these longer-running workflows.

For software development, it can work through long-running coding tasks such as building features across a codebase, debugging, refactoring, and reviewing code. It finds the root cause before changing anything, checks its work as it goes, and explains its changes in plain language, so engineers can review and trust them quickly.

That combination becomes particularly valuable when developers use models through agentic coding environments, where a session may involve dozens of steps and run for an extended period of time. The same applies beyond software development.

For knowledge workers, Claude Opus 5.5 can bring together information from multiple sources, work through long documents and spreadsheets, perform analysis, and help create artifacts such as memos, reports, and presentations. Compared with Opus 5, it produces outputs that require less editing before they are ready to share.

An AI model that communicates more like a teammate

As agents take on more autonomous work, another challenge emerges: keeping the human in the loop without overwhelming them.

An agent that performs 50 steps should not require someone to inspect 50 steps to understand whether the work was successful. Claude Opus 5.5 introduces improvements to agentic communication designed to make long-running work easier to follow. As it works, the model can surface the information that matters most:

What it did

What it found

What decisions it made

Where it needs input from the user

What happened at the end of a long-running task

The goal is simple: spend less time decoding what the model did and more time using the result. This matters particularly for enterprise agents, where users need to understand not only the final answer but also when an agent needs clarification, encounters a constraint, or reaches a decision point that requires human judgment.

Adpative thinking

Claude Opus 5.5 uses adaptive thinking, automatically determining how much reasoning a task requires. Rather than turning thinking on or off or manually specifying a thinking-token budget, developers use effort to influence how much work the model should put into a request. This allows the model to adapt its reasoning to the task at hand—from relatively straightforward requests to complex problems that require deeper analysis.

For developers building agents, this can reduce the amount of application logic needed to decide when and how a model should reason.

Designed for long-running agent architectures

Long-running agents create challenges beyond model intelligence. Conversations can exceed context limits. Tools available to an agent can change. Applications may need to compact earlier context while preserving the model's understanding of the work already completed.

Alongside Claude Opus 5.5, Anthropic is introducing beta API capabilities designed for these scenarios, including asynchronous compaction, keep-tail compaction, and changing tools during a conversation while preserving thinking and prompt caching. These capabilities can help agent developers maintain continuity across longer tasks without rebuilding the state of the application every time context or available tools change.

Combined with Microsoft Foundry, developers can use Claude Opus 5.5 as part of broader agent systems that connect models with enterprise data, tools, evaluation, and operational workflows.

Expanded safeguards for more capable models

As model capabilities increase, Anthropic is also expanding the safeguards applied to Claude Opus 5.5.

Claude Opus 5.5 is the first Opus model to use safety classifiers like those introduced with Claude Fable 5.1 in areas including cybersecurity, biology, AI development, and distillation.

For common developer, educational, and knowledge-work scenarios, customers can continue using the model for tasks such as identifying software vulnerabilities or learning about biological concepts. Certain requests that Anthropic identifies as higher-risk or dual-use may be handled by another Claude model with the appropriate safeguards.

This reflects an increasingly important part of deploying more capable models: advancing what models can do while applying safeguards appropriate to the capabilities they introduce.

Pricing

Model

Deployment Offers

Input/M Tokens

Output/M Tokens

Availability

Claude Opus 5.5

Global Standard, US DataZone

Cache Hit - $0.20

Cache Write - $5

Cache Write (1 hr) - $8

$20

GA, Hosted on Azure

Claude Opus 5.5 (Long Context)

Global Standard, US DataZone

Cache Hit - $0.20

Cache Write - $5

Cache Write (1 hr) - $8

$20

GA, Hosted on Azure

Build with Claude Opus 5.5 in Microsoft Foundry

Choosing a model is only the beginning of putting AI into production. Microsoft Foundry gives developers a unified place to discover models, build and evaluate AI applications and agents, connect them with enterprise data and tools, and operate those systems in production. As models become capable of taking on more complete units of work, the question is shifting from Can the model answer this prompt? to Can I trust it to carry the work forward? Claude Opus 5.5 represents another step in that direction: stronger performance on complex work, more adaptive reasoning, and clearer communication between people and the AI systems working alongside them.

Claude Opus 5.5 is available today in Microsoft Foundry.

Demystifying Dynamic AI Routing: Exploring the model router in Foundry Models with RouteLab

vikaspandey — Wed, 30 Sep 2026 13:00:00 GMT

As enterprise generative AI applications mature, engineering teams face a continuous balancing act between cost, latency, and response quality. Hard-coding a frontier model for every request ensures high quality but maximizes token costs and latency. Conversely, defaulting to smaller, faster models risks degradation in accuracy for complex reasoning tasks.

To solve this architectural challenge, developers can use the model router in Foundry Models—an intelligent, real-time routing engine. By combining this capability with testing environments like RouteLab, developers can eliminate the guesswork of model selection, visually evaluate routing decisions on custom datasets, and build highly optimized AI systems.

What is the model router in Foundry Models?

The model router is a purpose-built machine-learning model that sits between your application and a configured pool of large language models (LLMs). Rather than relying on rigid, hard-coded rules, the router analyzes the complexity, required reasoning capabilities, and context of each incoming prompt in real time to select the most suitable LLM.

By targeting a single deployment, developers gain access to three distinct routing modes:

Balanced (Default): Evaluates all models within a narrow quality margin and selects the most cost-effective option for general-purpose scenarios.
Cost: Broadens the acceptable quality band to aggressively minimize token costs—ideal for high-volume, budget-sensitive workloads.
Quality: Always routes to the highest-performing model for that specific prompt, ignoring cost implications for complex reasoning tasks.

Getting Started: Running RouteLab Locally

Before diving into the visual evaluations, you need to spin up RouteLab in your local environment. The model-router-playground repository is designed for a frictionless developer experience right out of the box.

Part 1: Setting Up the Environment

Clone the Repository and Open in VS Code: Start by cloning the repository to your local machine. Open the cloned folder directly inside Visual Studio Code (VS Code) so that all project assets, scripts, and dependencies are loaded in your editor workspace.
Launch the Script: Open the integrated terminal in VS Code and execute the setup script by running

git clone https://github.com/Pandey-Vikas/model-router-playground.git cd model-router-playground ./start.ps1.

This kicks off the environment initialization, installs required dependencies, and prepares the backend routing services.

Part 2: Interactive Authentication and Launch

Complete the 6-Step Azure Login and Launch the UI: Once the script executes, it presents a guided, 6-step interactive authentication workflow directly in your terminal. This wizard safely connects your local setup with your Azure credentials, respects enterprise access boundaries, and wires up your designated Microsoft Foundry Model Router resource. Upon completing step 6, RouteLab automatically opens in your default browser, presenting the full testing harness.

Hands-On with RouteLab: The Developer Playground

Understanding how dynamic routing behaves in practice is critical before deploying it to production. The model-router-playground repository provides a practical, UI-driven approach to exploring this architecture through its RouteLab interface.

As seen in the RouteLab dashboard below, the environment is designed to demystify the routing process without requiring custom evaluation scripts. Here is how it empowers development teams:

1. Interactive Manual Evaluation- "Supports Bring your own Dataset"

One of the biggest questions developers have about dynamic routing is, "Which model will it pick for my specific prompt?" RouteLab’s central Manual Evaluation interface allows you to select your desired mode (Balanced, Cost, or Quality) and test the router. Crucially, this is where the "Bring Your Own Data" USP truly shines. Instead of relying solely on generic examples or typing prompts one by one, you can seamlessly import your custom dataset (CSV or JSONL) directly into the Manual UI. By running your proprietary, domain-specific questions through the interface, you gain instant, line-by-line transparency into how the router handles your exact enterprise workloads—immediately displaying the selected underlying model, token usage, and latency for your own data.

To get a comprehensive view of the router's decision-making process, you can execute your imported dataset three separate times—once for each routing mode. After completing these runs, simply navigate to the Analytics tab and launch the Compare button. This powerful feature allows you to evaluate all three executions side-by-side, providing a clear, empirical comparison of cost savings, latency overhead, and model selections across the Balanced, Cost, and Quality modes for your specific prompts.

2. Quick Testing with the Routing Range

To rapidly build intuition on how the router behaves, the UI features a "Test the routing range" capability. With a single click—such as the "Run 15 (quick)" or "Run 30 (full)" buttons—developers can fire off a spectrum of predefined prompts from direct instructions to complex synthesis. This allows you to watch the router actively adapt its model selection across a diverse ladder of complexities.

3. Auto Evaluation at Scale: Validating Custom Workloads

While manual testing builds intuition line-by-line, production readiness requires scale. RouteLab’s Auto Evaluation engine allows developers to run bulk assessments. It comes pre-loaded with bundled datasets (like mixed-prompts.jsonl or java_custom.jsonl) for quick benchmarking, but its ultimate USP is executing automated evaluations on your imported custom datasets.

To make formatting effortless, you can download a sample .jsonl file from the available bundled datasets directly within the UI, providing a perfect template to construct your own custom datasets. By uploading your populated proprietary files, teams can automate the execution of hundreds of their own prompts. This ensures your routing strategy and cost projections are validated against the real-world datasets your application will handle in production, bridging the gap between generic benchmarks and enterprise reality.

Under the Hood: The Code Behind Auto Evaluation Beyond the UI experience, the complete underlying code for this Auto Evaluation engine is available in its own dedicated repository: microsoft-foundry/Model-Router-Auto-Evaluation. By exploring this repository, developers can see exactly how the batch-processing, routing logic, and metric aggregations are programmatically implemented. This open-code approach makes it incredibly easy for engineering teams to lift and shift the evaluation framework directly into their own enterprise CI/CD pipelines or custom testing harnesses.

After successful completion of Auto Evaluation, you click the open button Click on the open button under Past Evaluation to get the dashboard.

4. Deep Dives with Analytics and Logs

Data drives deployment decisions. The top navigation of the RouteLab UI seamlessly transitions users from Evaluation into Analytics and Logs. Instead of parsing raw JSON telemetry, developers can visualize the aggregate results of their custom and bundled datasets:

Cost Distribution: See the exact token savings achieved by routing simpler tasks to smaller models.
Latency Overhead: Measure the actual response times to validate that the router's overhead is negligible.
Model Utilization: View visual breakdowns of how often frontier models are invoked versus specialized models under different routing modes.

The Future is Dynamic

The era of static, one-size-fits-all LLM deployments are ending. By leveraging the model router and visually validating your strategy through RouteLab—using both bundled and custom datasets—your team can confidently deploy AI architectures that are highly responsive, scalable, and remarkably cost-efficient.

Ready to explore intelligent routing? Check out the model-router-playground on GitHub today and start optimizing your enterprise AI workloads!

References & Resources

Microsoft Learn Documentation: Model router in Foundry Models | Learn
RouteLab Developer Playground: model-router-playground on GitHub
Auto Evaluation Source Code: Model-Router-Auto-Evaluation on GitHub

A2A Endpoints and A2A Tool in Microsoft Foundry agents

Christian_Coello — Fri, 18 Sep 2026 19:00:00 GMT

Agent-to-agent collaboration just became much easier to build in Microsoft Foundry. The A2A Tool and incoming A2A endpoints support A2A protocol version 1.0, which is generally available. The earlier a2a_preview tool type and protocol version 0.3 remain available in preview for existing integrations. Hosted Agents can also consume A2A tools through Foundry Toolboxes exposed over MCP.

If you have been following earlier previews, A2A Endpoints were previously known as the A2A API head. The idea is simple:

A2A Endpoint: expose a Foundry agent so an external agent can discover and invoke it.
A2A Tool: let a Foundry agent invoke another A2A-compatible agent.
Hosted Agents: use the A2A Tool through a Foundry Toolbox exposed as an MCP endpoint.

Together, these capabilities make it possible to build multi-agent systems where agents can specialize, publish their skills, and securely collaborate across service boundaries.

Why this matters

Until now, many multi-agent patterns required custom APIs, one-off adapters, or orchestration logic tightly coupled to a specific framework. A2A gives us a standardized protocol for agent-to-agent communication.

That means one agent can ask another agent for help without needing to know its implementation details. The caller discovers the remote agent’s capabilities through an agent card, sends a task through the A2A protocol, and receives a response it can incorporate into the conversation.

For Foundry-hosted A2A endpoints, discovery is authenticated. The agent-card URLs and protocol endpoint require Microsoft Entra ID authentication and are not publicly accessible.

For example:

A support agent can call a billing agent.
A research agent can call a data-analysis agent.
A Foundry-hosted enterprise agent can call a specialized agent running outside Foundry.
A Hosted Agent can call a Foundry Toolbox that wraps an A2A connection.

Architecture 1: External agent calls a Foundry agent through an A2A Endpoint

In this pattern, a Foundry agent is exposed as an A2A endpoint. An external agent discovers the agent card and invokes it using the A2A protocol.

An external agent discovers a Foundry agent through its agent card, invokes it using A2A protocol v1.0, and receives a response generated with Foundry tools, data, and models.

The Foundry agent processes the request using its configured models, instructions, tools, and enterprise data sources, and then returns the response to the calling agent.

A Foundry prompt agent must support the Responses protocol before it can be exposed through an incoming A2A endpoint.

The Foundry agent exposes an A2A base URL in the following format:

https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a

The version-specific agent-card URLs follow these patterns:

https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a/agentCard/v1.0

https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a/agentCard/v0.3

For new integrations, target A2A protocol v1.0, which is GA. Foundry also supports v0.3 for existing preview integrations, but v1.0 is the recommended path forward.

Foundry serves both protocol versions through the same A2A base path. The caller selects the version through agent-card negotiation, the A2A-Version header, or the a2a-version query parameter. Production clients should explicitly negotiate or request version 1.0 rather than relying on the default behavior.

Foundry’s incoming A2A v1.0 endpoint uses JSON-RPC. Incoming A2A v0.3 supports JSON-RPC and HTTP+JSON, but v0.3 remains in preview. Only text modality is supported, and streaming responses are not supported.

Architecture 2: Foundry agent calls another agent using the A2A Tool

The reverse pattern is just as important. A Foundry agent can use the A2A Tool to call another A2A-compatible endpoint.

A RemoteA2A project connection stores the remote A2A base URL and its authentication configuration. The A2A Tool references the connection and specifies the A2A protocol version to use.

Foundry resolves the default agent-card path automatically and negotiates the A2A protocol version. You do not need to configure send_credentials_for_agent_card for a Foundry agent target.

First, set the active Foundry project. Then create the RemoteA2A connection.

PROJECT_ENDPOINT="https://{account}.services.ai.azure.com/api/projects/{project}" azd ai project set "$PROJECT_ENDPOINT" azd ai connection create my-a2a-connection \ --kind remote-a2a \ --target "https://{account}.services.ai.azure.com/api/projects/{project}/agents/{agent}/endpoint/protocols/a2a" \ --auth-type agentic-identity \ --audience "https://ai.azure.com"

The target Foundry project or agent must grant the calling agent identity the Foundry Agent Consumer role, or another role that contains the required endpoint permissions. When assigning the role, use the calling identity’s Microsoft Entra object or principal ID, not its application or client ID.

A role assignment can be created at either the target project scope or the individual agent scope:

az role assignment create \ --assignee-object-id "{calling-agent-principal-object-id}" \ --assignee-principal-type "ServicePrincipal" \ --role "eed3b665-ab3a-47b6-8f48-c9382fb1dad6" \ --scope "{target-project-or-agent-resource-id}"

Agent identities and managed identities are represented as service principals in Microsoft Entra ID, which is why ServicePrincipal is used as the principal type.

Architecture 3: Hosted Agent uses A2A through a Foundry Toolbox

Hosted Agents can use A2A by attaching a Foundry Toolbox. The toolbox contains an A2A Toolbox Tool and exposes it through an MCP-compatible endpoint.

The complete interaction flow is:

The Hosted Agent connects to the Foundry Toolbox over MCP.
The Hosted Agent authenticates to the toolbox using Microsoft Entra ID.
The toolbox invokes its configured A2A Toolbox Tool.
The A2A Toolbox Tool references a RemoteA2A project connection.
The RemoteA2A connection identifies the destination and downstream authentication configuration.
The toolbox invokes the remote agent through the A2A protocol.
The result is returned to the Hosted Agent through the toolbox’s MCP endpoint.

Hosted Agents securely invoke remote A2A agents through a Foundry Toolbox and RemoteA2A project connection.

This creates a two-hop architecture: Hosted Agent to Foundry Toolbox over MCP, followed by Foundry Toolbox to the remote agent over A2A.

The toolbox MCP endpoint used by the Hosted Agent follows this format:

https://{account}.services.ai.azure.com/api/projects/{project}/toolboxes/{toolbox-name}/versions/{version}/mcp?api-version=v1

A version-specific toolbox endpoint can also be used when the Hosted Agent must remain pinned to an immutable toolbox version.

The Hosted Agent authenticates to the toolbox using Microsoft Entra ID and the following scope:

https://ai.azure.com/.default

The Hosted Agent authenticates to the Foundry Toolbox MCP endpoint using Microsoft Entra ID. For direct token acquisition, use the https://ai.azure.com/.default scope.

Downstream credentials, secrets, managed identity settings, and OAuth configuration should remain in the RemoteA2A project connection rather than being embedded in the Hosted Agent’s code.

Authentication is the design decision

The most important architecture choice is authentication. A2A supports different patterns depending on whether the calling agent should act as itself or on behalf of the user.

RemoteA2A connection authentication options

Auth type	Use when
none	The remote endpoint does not require auth. Rare for production.
custom-keys	The endpoint expects an API key, PAT, bearer token, or custom header.
oauth2	Each user authorizes access through an OAuth 2.0 consent flow. Use this when the downstream action must preserve the user’s individual permissions.
user-entra-token	The user’s Microsoft Entra identity should flow to the remote service.
project-managed-identity	All agents in the project should share the project identity.
agentic-identity	For service-to-service calls using an agent identity, service principal, or managed identity, use as the role-assignment principal type.

For incoming A2A Endpoints on Foundry agents, authentication is stricter:

Incoming A2A requires Microsoft Entra ID authentication.
Key-based and unauthenticated incoming access are not supported.
The caller must have Foundry Agent Consumer or another role that grants endpoint access.
Access can be granted at the project scope or individual agent scope.
Calls can use either on-behalf-of user identity or a service identity such as an agent identity, service principal, or managed identity.

Use OAuth or user identity passthrough when the remote action must respect each user’s permissions.

What you configure

To expose a Foundry agent as an A2A Endpoint, you configure two things:

An agent card, describing the agent’s capabilities.
The A2A protocol on the agent endpoint.

The relevant portion of the agent PATCH request body looks like this:

{ "agent_card": { "description": "A specialist agent that answers questions about invoices.", "version": "1.0", "skills": [ { "id": "invoice-lookup", "name": "Invoice lookup", "description": "Finds and summarizes invoice status." } ] }, "agent_endpoint": { "protocol_configuration": { "responses": {}, "a2a": {} } } }

Agent cards for prompt agents in Foundry can also be configured directly in the portal. Then another agent can call it via Tools using the agents A2A endpoint.

A2A Endpoint for an agent in Foundry

Agent card configuration for a Foundry agent

Final thoughts

A2A Endpoints and the A2A Tool make Foundry agents composable. Instead of building one large agent that knows everything, you can build focused agents that publish capabilities and collaborate securely.

Use A2A Endpoints when you want other agents to call your Foundry agent. Use the A2A Tool when your Foundry agent needs to call another agent. For Hosted Agents, use a Foundry Toolbox to bring A2A into the hosted runtime cleanly.

Learn more:

A2A authentication: https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/agent-to-agent-authentication
A2A Tool: https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/agent-to-agent
A2A Endpoint: https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/enable-agent-to-agent-endpoint

Announcing the Playwright Workspaces Remote MCP Server for Agentic Browser Automation

NandiniMuralidharan — Thu, 24 Sep 2026 14:41:55 GMT

An agent can analyze a request, choose the right next step, and call every available API—then stall when the final action exists only in a web interface.

AI agents are increasingly capable of reasoning, planning, and working with APIs. But many real business processes still depend on websites: business portals, admin consoles, support tools, forms, dashboards, and applications without an API for every action. That last mile can stop an otherwise capable agent.

The Playwright Workspaces remote Model Context Protocol (MCP) server, now in preview closes this gap. It gives MCP-compatible agents a managed remote browser and a comprehensive set of browser automation tools optimized for AI agents—without requiring teams to install Playwright, package browser binaries, or operate browser infrastructure in the agent environment.

The remote MCP server uses Streamable HTTP—a transport protocol that allows real-time, bidirectional communication between your agent and the browser session—and is delivered as a Playwright Workspaces feature. Agents can create a browser session, navigate and inspect pages, interact with elements, upload files, review console and network activity, capture screenshots, and close the session when the workflow is complete. We are exposing definitive set of 22 tools which can help you complete browser flows successfully with agent interactions.

MCP config with GitHub Copilot CLI:

MCPTool( server_label="playwright-browser-automation", server_url=mcp_server_url, headers={"x-api-key": playwright_service_access_token}, require_approval="never", )

Configure Tool approval Foundry:

Why this release matters

Enterprise workflows rarely live in one API.

The reality is:

Critical tasks often end in a web interface.
Multi-step workflows require both reasoning and interaction.
Website structure and timing can change between runs.
Agent environments might not allow browser packages or local processes.
Teams need visibility and control when automation reaches an unexpected state.

The Playwright Workspaces remote MCP server closes this gap by exposing browser automation through an open protocol. Your agent uses standard MCP tools while Playwright Workspaces provides the remote browser infrastructure.

This makes it easier to add browser execution to an existing agentic workflow and keep the browser lifecycle explicit, observable, and bounded.

What's new

A hosted remote MCP endpoint : Connect an MCP-compatible client to a workspace-scoped HTTPS endpoint (a URL unique to your Playwright Workspace that handles all MCP communication). The service is hosted and managed, so there is no local MCP server process to install or maintain.

Browser tools designed for agentic workflows : The preview exposes 22 tools across the browser lifecycle. They cover navigation, accessibility-based page inspection, interaction, tabs, file upload, screenshots, browser diagnostics. See the product documentation for the complete tool and parameter reference. Accessibility snapshots give agents a structured way to understand a page and select action targets. For focused tasks, browser_find can locate relevant snapshot nodes without returning the entire page representation.

An observe-act-verify automation loop : Reliable browser agents shouldn't execute a long sequence of clicks without checking the result. The tool set supports a practical loop which helps agents recover when a page rerenders, a selector becomes stale, a dialog opens, or an action completes differently than expected.

Step	Action	Tool Example
1. Observe	Understand the current page state	browser_snapshot or browser_find
2. Act	Execute on a snapshot reference or selector	browser_click, browser_type
3. Wait	Pause for expected page condition	browser_wait
4. Verify	Confirm resulting state before continuing	browser_snapshot
5. Close	End the browser session when complete	browser_close

Support for Microsoft Foundry, GitHub Copilot CLI and other agent builder surfaces : You can connect the remote server to various agent builder surfaces and use cloud browsers for capabilities including session isolation and private website support. VNet injection lets agents access internal websites that aren't exposed to the public internet—useful for enterprise admin consoles and internal tools.

Visibility using LiveView when you need investigation: A created session can return a Live View URL when Live View is available. This gives developers and operators a way to observe browser activity while evaluating or troubleshooting an agentic flow. The server also exposes bounded console-message, network-request, snapshot, and screenshot results. These signals help identify navigation failures, page errors, timing issues, and unexpected application state.

Choose the right Playwright experience

Playwright supports several development and automation models. Choose the experience that matches your workload:

What you need	Recommended experience
Give an MCP-compatible agent a hosted browser through a remote endpoint	Playwright Workspaces Remote MCP Server
Add browser automation to Microsoft Foundry through its integrated toolbox experience	Browser Automation Tool in Microsoft Foundry
Bring your remote CDP browsers with preferred choice of framework	Hosted agent Sample in Foundry
Run an existing Playwright test suite at scale across managed cloud browsers	Playwright Workspaces test execution

The Remote MCP Server is designed for agent-driven, tool-based browser workflows. It complements rather than replaces the Playwright SDK, OSS playwright-cli, or the integrated Browser Automation Tool in Microsoft Foundry.

Get Started

Build your endpoint URL: Take your Playwright Workspaces API endpoint, replace .api with .mcp and append /mcp to create the remote MCP server URL. Example: https://{region}.mcp.playwright.microsoft.com/playwrightworkspaces/{workspaceId}/mcp
Configure authentication: Add the x-api-key header with your Playwright Workspaces access token. Example : set: x-api-key: <your-access-token>
Connect your agent to MCP
TIP : Exercise the Tools approval flow as present in Foundry, Copilot CLI– easier to get started

What this unlocks

The Remote MCP Server enables agents to complete workflows across APIs and websites, combine automation with human approval, inspect dynamic applications, and collect page, console, network, and screenshot evidence. Instead of stopping when a process reaches a browser, an agent can take a controlled action, verify the result, and continue toward the outcome.

Next Steps

Try it now: Follow the MCP Quickstart to connect your first agent in under 10 minutes
Explore the tools: Review the complete tool reference to see all 22 browser automation capabilities
See it in action: Check out our sample agents repository for working examples with Foundry and GitHub Copilot CLI
Get help: Join the conversation on Discord or submit feedback via our feedback channel

Practical Local AI for Enterprise Architects: A Workload Placement Approach

arronhoffer — Wed, 16 Sep 2026 14:00:00 GMT

Enterprise AI discussions often begin with a use case and end with a deployment constraint. A clinical document assistant may have clear value, yet protected health information cannot leave an approved environment. A factory assistant may improve operations, yet the facility cannot depend on continuous connectivity. A financial analysis workload may require dedicated execution, predictable operating costs, or strict data residency.

These conditions do not invalidate the business case. They define the workload execution location and operating pattern.

For enterprise architects, the central question is no longer whether AI belongs in the cloud or on local infrastructure. The more useful question is this: which execution location best satisfies the business, data, security, performance, and operational requirements of each workload?

This article presents a practical workload placement approach for local and contained AI. It also clarifies how Windows ML, Foundry Local, Microsoft Foundry, Azure Machine Learning, Azure Local, and GPU-enabled Azure Kubernetes Service can contribute to one governed architecture.

Architecture constraints are placement requirements

Many AI projects stall between prototype and production. The model may perform well, the user experience may be useful, and the business sponsor may support the investment. The project still stops because the production environment introduces requirements that were absent during experimentation.

Common constraints include:

Regulation: Sensitive information must remain within an approved environment.
Data residency: Inference must occur inside a defined geographic, organizational, or technical boundary.
Security: The workload requires dedicated, private, or isolated execution.
Connectivity: The solution must continue to operate when network access is limited or unavailable.
Performance: The experience depends on consistent, low-latency inference.
Cost: The operating model must be understandable, measurable, and predictable.

Architects should treat these statements as architecture inputs. A constraint can change the execution plane, the infrastructure choice, the governance pattern, or the operational model. It should not automatically eliminate the use case.

This distinction changes the conversation. Instead of asking whether the organization can use AI under restrictive conditions, the architecture team can ask which approved deployment pattern satisfies those conditions.

Think in terms of an AI deployment spectrum

Enterprise AI can execute across a spectrum of locations:

Device: Laptops and workstations can run local inference close to the user and the application.
Edge: Factory systems, field equipment, and other edge environments can support local decisions when latency or connectivity matters.
Private infrastructure: Azure-managed or customer-controlled infrastructure can provide dedicated capacity, isolated networking, and approved data boundaries.
Cloud: Managed models and elastic services can provide broad capability, rapid innovation, and global scale.

These locations are not competing ideologies. They are architecture options. A single enterprise may use all four, from workstation testing and disconnected facilities to governed private infrastructure and elastic cloud services. The appropriate location depends on the workload, not on a universal preference for local or cloud execution.

Local AI expands placement choice. It does not replace cloud AI.

Figure 1. AI can run across device, edge, private infrastructure, and cloud environments. The workload requirements determine the placement.

Clarify the Microsoft AI portfolio

Several Microsoft technologies support this spectrum, but they operate at different architectural layers. Clear role definitions help architecture teams avoid treating related products as interchangeable.

Windows ML: the local execution engine

Windows ML supports efficient AI execution on Windows devices. It provides the execution capability that applications use to execute models on available device hardware. It is not the enterprise governance platform, the private infrastructure layer, or the large-scale GPU orchestration layer.

Foundry Local: local model delivery and management

Foundry Local helps developers discover, manage, and execute models locally. It supports the path from local model selection to application integration and prompt validation. Windows ML executes models, while Foundry Local delivers and manages the local model experience.

Microsoft Foundry: the enterprise control center

Microsoft Foundry supports the enterprise lifecycle for AI applications and agents. Platform teams can use it to build, evaluate, govern, and operate AI assets across a controlled environment. This is where local experimentation can become an enterprise asset.

Azure Machine Learning: custom model lifecycle and MLOps

Azure Machine Learning supports training, deployment, and management for machine learning models. It is appropriate when the organization requires custom MLOps, managed experimentation, model registration, deployment controls, and lifecycle management.

Azure Local: private, Azure-managed infrastructure

Azure Local provides Azure-managed infrastructure within an on-premises or approved private environment. Foundry Local focuses on models and local development. Azure Local focuses on private infrastructure.

GPU-enabled AKS: scalable execution

GPU-enabled Azure Kubernetes Service provides a Kubernetes execution layer for training and inference at scale. Platform engineering teams can use it when a workload requires higher throughput, clustered GPU capacity, or large-scale inference.

The roles are complementary. Windows ML executes. Foundry Local supports local model delivery. Microsoft Foundry governs the enterprise lifecycle. Azure Machine Learning manages custom model operations. Azure Local provides private infrastructure. GPU-enabled AKS supports scalable execution.

Match the workload to the execution pattern

The following framework provides a starting point for architecture discussions.

Workload requirement	Recommended pattern
Fast experimentation	Foundry Local
Lowest latency on Windows or low transaction volume	Windows ML with Foundry Local
Disconnected, sovereign, or private operation	Foundry Local with Azure Local
Enterprise agents and centralized governance	Microsoft Foundry
Custom MLOps and model lifecycle management	Azure Machine Learning
Large-scale inference or high transaction volume	GPU-enabled AKS

This framework does not replace a workload assessment. It provides the assessment with a structured focus.

Architecture teams should evaluate data sensitivity, latency, transaction volume, connectivity, availability, identity, observability, model lifecycle, and operational ownership. They should also decide whether the workload must continue during a network interruption and whether infrastructure must remain within a defined boundary.

The result may be a hybrid pattern. Local inference can serve a device or facility, while centralized services manage identity, policy, evaluation, telemetry, model registration, and approved updates.

Move from a local prototype to a governed asset

Local development can shorten the path to a useful proof of concept. A developer can select a model, run inference, test prompts, refine the user experience, and evaluate output without first building a complete enterprise platform.

Production introduces a different set of operational responsibilities. The organization must package the model and application, register approved assets, define identity and access controls, capture telemetry, manage versions, perform evaluations, monitor health, and establish operational ownership.

The workload does not necessarily need to change during this transition. The operational model does.

Consider an assistant that processes a sensitive document. A small team may begin with local inference on a workstation because the source material must remain local. If the assistant expands from five users to five thousand, the user scenario can remain the same. What changes is the execution plane and the platform around it.

The enterprise version may require Microsoft Foundry, an AI gateway, API management, Microsoft Entra ID, centralized observability, approved telemetry, and GPU-enabled AKS. The architecture adds shared governance, observability, scalability, and operational controls while preserving the original workload requirement.

This is the governed handoff. The prototype becomes a repeatable enterprise asset without losing the placement characteristics that made the use case viable.

Use a contained AI reference architecture

A contained AI platform can be described through four connected layers.

Experience

Applications, copilots, and agents provide the user experience. This layer should remain as independent as practical from the selected execution plane so the organization can change placement without redesigning the entire application.

Local AI orchestration

Windows ML and Foundry Local support execution on devices and at the edge. This layer is useful when processing must remain close to the user, source system, or facility.

Platform and governance

Microsoft Foundry, an AI gateway, API management, and Microsoft Entra ID provide shared controls. This layer can centralize identity, policy, safety practices, routing, evaluation, and lifecycle governance.

Execution and data

Azure Machine Learning, GPU-enabled AKS, Azure Local, and approved data services provide the execution capacity and data access pattern. The selected combination depends on scale, isolation, lifecycle, and infrastructure requirements.

Observability spans every layer. Identity, policy, safety, telemetry, health, logging, and evaluations should not be added after deployment. They are part of the production architecture.

The guiding principle is simple: use one governed pattern, then vary the execution location according to the workload.

Figure 2. A shared governance layer can apply identity, policy, safety, evaluation, and observability controls across several execution environments.

Treat economics as a workload-specific decision

Local AI can support a more predictable operating model when dedicated capacity is appropriate. Cloud services can provide elasticity, managed capabilities, and broad model choice. Neither option produces universal savings.

Economic analysis should reflect the customer workload. Architects should compare utilization, capacity, transaction volume, hardware lifecycle, software operations, support, networking, availability, and governance overhead. A dedicated environment with low utilization may be inefficient. A stable, high-utilization workload may benefit from reserved or dedicated capacity. A variable workload may benefit from consumption-based cloud services.

The right economic question is not whether local AI is cheaper. It is whether the selected placement produces an explainable operating model for the workload.

Revisit use cases that were previously blocked

Local and contained execution patterns can reopen scenarios that were difficult to approve under a cloud-only assumption. Examples include clinical document extraction, factory-floor assistance, disconnected government operations, air-gapped intelligence workloads, offline field guidance, and restricted-data analysis.

The business value in these scenarios may have existed for years. The missing element was an acceptable deployment model.

Architects can now separate the use case from the execution constraint. They can preserve the required boundary while still applying enterprise identity, governance, observability, and lifecycle practices.

A practical next step

Start with one blocked or constrained workload.

Select a use case with clear business value.
Define the required data, network, security, regulatory, and operational boundary.
Identify the appropriate execution location across device, edge, private infrastructure, and cloud.
Map the workload to the Microsoft technologies that serve each architecture layer.
Define the governed handoff from prototype to production.
Validate the operating model, including identity, evaluation, telemetry, support, and cost.

This process turns an abstract local-versus-cloud debate into a concrete architecture decision.

Figure 3. The AI workload can remain consistent as the operating model matures from an individual prototype to a governed enterprise platform.

Conclusion

The future of enterprise AI is workload placement.

Organizations need governed AI systems that can run wherever the business, data, and risk model require. Some workloads belong in elastic cloud services. Others require local inference, private infrastructure, isolated execution, or disconnected operation. Many will combine these patterns.

Enterprise architects can support that diversity without creating separate governance models for every location. Build locally when the workload requires it. Govern centrally across the lifecycle. Scale through the execution plane that fits the demand.

The most useful question is not, “Should this AI run locally or in the cloud?” It is, “What placement allows this workload to operate securely, reliably, and at enterprise scale?”

Your Agent Found the Right Schema. Then Ignored It.

shihyaolin — Tue, 15 Sep 2026 17:35:05 GMT

The assumption nobody tests

Most agent evaluations quietly assume something that never happens in production: that the model is handed exactly the right information, and nothing else.

We changed that one assumption — and watched a model drop from 58% to 18%.

Retrieval was not the problem. The correct information was sitting in the prompt. The model just didn't use it.

TL;DR — Standard agent evaluations hand the model exactly the right schema. Real retrieval does not. Changing only that cost 40 points of accuracy, and neither a larger model nor better prompting recovered it. A $150 fine-tuning job won two-thirds of it back and beat prompted gpt-5 by 28 points at a twenty-fifth of the per-turn cost. Steps 2–4 of the runbook below reproduce the gap on your own registry and need no fine-tuning.

We measured this on schemas, but if you build agents that retrieve tools, functions, or MCP servers into a prompt, the shape of the problem is the same. More on that at the end.

Here is the setup. In schema-guided dialogue agents, the model is given a schema — the list of fields it is allowed to fill — and must return a filled state. Benchmarks put the one correct schema in the prompt and report strong numbers: 86.4% and 74.7% on SGD [2, 3, 4]. But a deployed assistant has no idea which schema is correct. It retrieves candidates from a registry, so the prompt ends up holding the right schema plus whatever else the retriever dragged in.

So we changed exactly one thing: how the schema reaches the prompt. Same dataset, same metric, same model. We call the realistic condition RetDist (retrieval + distractors) — BM25 top-1 plus two random distractor schemas from the same registry, shuffled. Note what this does not guarantee: BM25 ranks the correct schema first on only 65.4% of turns, so on about a third of them the right answer is not in the prompt at all. That matters later.

The score we measure is joint goal accuracy (JGA): the fraction of turns where every field is exactly right. It is unforgiving on purpose — one wrong field fails the turn, which is also how a user experiences it.

So if a turn needs three fields and the model gets two of them right, that turn scores zero. This matters for reading the numbers below: 58 does not mean "58% of fields correct" — it means 58% of turns were completely correct. Slot-level accuracy on the same runs is far higher, and far less honest about what a user would experience.

58.0% with the gold schema. 18.3% under RetDist. Same model — gpt-4.1-mini, same weights, same dialogues, same metric. Only the prompt assembly changed.

Figure 1. The only thing that changes is how the schema reaches the prompt.Standard evaluation (A) hands gpt-4.1-mini the one correct schema, so the task reduces to filling in fields. A deployed assistant (B) retrieves candidates from the 21-service test registry, so the correct schema arrives alongside confusable ones. Same dialogue, same weights, same metric — 58.0 JGA becomes 18.3.

It is not a retrieval problem

The obvious explanation is that retrieval simply missed. It didn't.

We split every turn by whether the correct schema was actually in the prompt. On the turns where it was present — retrieval did its job — gpt-4.1-mini still only reached 24.0%.

Figure 2. Retrieval is not the bottleneck.Restricted to the 27,085 turns where the correct schema was verifiably present in the prompt, the base model still reaches only 24.0 JGA. Fine-tuning on the same distribution lifts the same stratum to 62.7.

So the model isn't failing to find the schema. It's failing to follow it. It falls back on field names memorised during pre-training instead of reading the schema in front of it. We call this Schema-Following Collapse.

This is why better retrieval would not have saved us. Prior robustness work on this benchmark varied the schema's wording [5]; the failure here survives even perfect retrieval.

Bigger models don't fix it

Figure 3. Scale and prompt engineering do not close the gap.Every prompted configuration — Llama-3.3-70B, gpt-5, and gpt-5 again with few-shot examples and strict decoding — lands inside the same 15–19 JGA band, alongside the un-tuned gpt-4.1-mini. Only fine-tuning leaves it. All bars are SGD-200 (n=1,810) under RetDist; the dashed line is the gold-schema ceiling.

Prompt engineering moved gpt-5 by 2.8 points. Everything stayed in a 15–19% band. This is not a capability problem you can scale or prompt your way out of.

Two caveats on that chart, because it names names.

This is a prompting stress test, not a capability ranking. All three models see one fixed deployment prompt. Give gpt-5 a different prompt, a schema-selection tool, or permission to ask a clarifying question and the number changes. The finding is that no prompted configuration escaped the band — not that any particular model is weak. gpt-5 scoring 16.6 here says something about the task, not about gpt-5.

And no, we did not fine-tune them. At the time of these runs gpt-5 exposed no supervised fine-tuning path on Foundry — it is reasoning-mode and fixed at temperature 1.0 — and fine-tuning a 70B model costs orders of magnitude more than the $150 that makes this recipe worth writing about. So this compares a fine-tuned small model against prompted large ones. That is the choice a team on a budget actually faces, but it is not a claim about what those models could do if they were trained.

What did work: train on the noise

RAFT (Retrieval-Augmented Fine-Tuning) [1], originally proposed for document question answering, trains a model with the evidence it should use alongside realistic distractors it should ignore. Unlike RAG, which adds retrieval at inference time, RAFT changes the training distribution, so the model learns how to behave after retrieval has already happened — including how to behave when retrieval brought back the wrong thing.

We adapt that idea from retrieved documents to retrieved schemas. We call the recipe SchemaRAFT: build the fine-tuning examples to look like the deployment prompt, including both the correct schema and confusable alternatives. This is a task-specific application of RAFT, not a new training algorithm.

Figure 4. The recipe.Every training prompt is assembled the way the serving prompt is assembled — the correct schema plus two confusable distractors. One in five examples contains no correct schema at all and is labelled with an empty state, which is what teaches the model to abstain rather than guess.

19,300 chat-format examples (~2,100 prompt tokens, ~40 completion tokens each)
each example: 1 gold schema + 2 random distractors, exactly as at serving time
20% of examples omit the gold schema entirely and are labelled {} — this teaches the model to withhold output rather than invent slots
3 epochs, ~124M training tokens, one fine-tuning job on Microsoft Foundry

Here is one training example, trimmed. The system message carries the schemas, the user message carries the dialogue, and the assistant message is the only part the model is scored on.

system

You are a dialogue state tracker for task-oriented conversations. ... Rules: - Output ONLY a valid JSON object on a single line. - Use only slot names that appear in the schemas below. - If no relevant slots have been mentioned, output: {} Service schemas (one is relevant; others may be distractors): Service: Hotels_4 Accommodation searching and booking portal Slots: - location [free-text]: City or town where the hotel is located - number_of_rooms [categorical] (values: 1, 2, 3): Rooms to reserve -------------------------------------------------- Service: Restaurants_2 Restaurant search and reservation service Slots: - city [free-text]: City where the restaurant is located - cuisine [free-text]: Type of food served - party_size [categorical] (values: 1, 2, 3, 4): Number of seats ... -------------------------------------------------- Service: Buses_3 ...

user

Conversation history: User: I need a table for two in San Francisco. System: What kind of food? User: Italian, somewhere moderate. Around 7 pm tomorrow. Extract the current belief state for service: Restaurants_2

assistant — this is the label, and the only tokens in the loss

{"city": "san francisco", "cuisine": "italian", "party_size": "2", "price_range": "moderate", "time": "19:00"}

Two things are worth noticing.

The model is told which service it needs. The name sits at the end of the user message and matches one of the three schema headers verbatim. Choosing the right schema is string matching. The only real task is to then stay inside it — and that is what collapses. A base model reaches for location from Hotels_4 because "San Francisco" and hotels co-occur constantly in pre-training, even though Restaurants_2 is named right there and has a city field. The gap is not a search failure. It is a grounding failure, and we made the search step as easy as it could possibly be.

Nothing in the loss punishes using a distractor's field. There is no auxiliary objective, no reranker, no constrained decoding. It is ordinary next-token prediction on the assistant message. The model sees ~19,300 examples in which three schemas are present and the answer only ever draws on one of them, and that is the entire training signal.

The inference prompt is the same object; only the assembly changes. At serving time BM25 chooses the schemas instead of the data builder, so the correct one is present 65.4% of the time rather than 80%, and the assistant message is generated instead of given.

Result on gpt-4.1-mini — the same model from Figure 1, now fine-tuned: 18.3 → 45.0 JGA, recovering about two-thirds of the gap to the gold-schema ceiling. On gold-in-prompt turns it goes 24.0 → 62.7.

We re-ran the identical recipe on gpt-4o-mini with no code changes and got +28.0 points (14.5 → 42.5). It's a recipe, not a lucky run.

The economics

Measured on the same deployment, per turn:

	Latency	Relative cost	JGA
gpt-5, prompted	4,401 ms	24.4×	16.6
gpt-4.1-mini + SchemaRAFT	1,095 ms	1.0×	45.0

Four times faster, roughly 25× cheaper per turn, and 2.7× the accuracy. The one-time training job costs about $150 and pays for itself after roughly 18,000 served turns.

Where else this shows up

We measured this on dialogue state tracking, but nothing in the failure is specific to dialogue. The ingredients are generic: several semantically similar capability descriptions retrieved into one prompt, and a model asked to commit to one of them.

It is worth being precise about how close the analogy is. An SGD service schema is a name, a description, and a list of typed parameters with their own descriptions. So is an OpenAI tool definition. So is an MCP tool listing. These are the same object wearing different labels, and the model is doing the same thing with all of them: reading several near-identical interface descriptions and picking one.

That pattern is everywhere in agent systems today — function calling with a large tool catalogue, MCP server selection, tool and plugin routing, multi-agent handoff, enterprise API registries.

What we did not test is whether the collapse reproduces when the decoding target is a tool call rather than a belief state. So treat the extension as a hypothesis, not a result. The cheap move is to measure it on your own registry — which is exactly what the runbook below does, and it needs no fine-tuning.

When this won't apply to you

This gap needs candidates that are genuinely confusable. SGD defines 45 services across 20 domains with heavily overlapping field names — the slot city alone appears in 9 of them, and 15 services carry some variant of it (from_city, to_city, and so on). If the schemas in your registry are clearly distinguishable, a model that retrieves the wrong one tends to notice, and you won't see a collapse this large.

Measure before you fine-tune.

What we haven't answered yet

We are reporting a finding, not a finished agenda. The open questions we think matter most, roughly in order of how cheaply you could settle them:

Does the collapse survive a better retriever? We used BM25 top-1 on character 3–5-grams — deliberately weak, because that is what a registry lookup usually is. A dense retriever with a cross-encoder reranker would reduce how often the wrong schema is in the prompt, but Figure 2 says it cannot fix the turns where the right schema was already there.
Does it reproduce when the output is a tool call? Everything here decodes a belief state. A function-call target has its own constrained decoding and its own schema-validation layer, and either could mask or amplify the effect. Untested.
Can schema selection replace fine-tuning? Making the model pick one schema first and fill it second is a much cheaper intervention than training. It also moves the failure rather than removing it — the picking step is the same discrimination problem. Worth measuring before anyone spends $150.
What is the right abstention ratio? 20% distractor-only examples is a round number that worked, not a tuned one. Abstention also turned out to be a separate skill from schema-following: fine-tuning lifted the gold-present turns enormously and barely moved behaviour on turns where the correct schema was genuinely absent.
Does one fine-tune transfer to a different registry? The recipe transfers across base models — we re-ran it unchanged on a second one for +28.0 points. Whether a model trained on one organisation's schemas follows a different organisation's schemas is a separate question, and the answer decides whether this is a one-time cost or a per-registry cost.

If you run the runbook on your own registry, we would like to know what number you get — especially if you get no gap at all. Negative results on the "when this won't apply" boundary are the most useful thing anyone could send us. Leave a comment on this post with how many schemas are in your registry, how confusable they are, and the two numbers from Steps 2 and 3. That is enough for us to tell whether it is the same failure mode.

Try it yourself (~20 minutes)

The step numbers match the sample's README, so you can move between the two. Steps 2–4 reproduce the gap and need no fine-tuning; they cost only inference on about 400 turns. Step 5 is the fix, written out so you can read the recipe without running it.

Prerequisites: Python 3.10+, a Microsoft Foundry deployment of a small chat model (we used gpt-4.1-mini), and the Foundry User role on that resource. The sample is keyless — it authenticates with DefaultAzureCredential, so there is no API key to paste anywhere. The sample README has the full access walkthrough.

git clone https://github.com/microsoft-foundry/foundry-samples.git cd foundry-samples/samples/python/finetuning/schemaraft-dst python -m venv .venv; .\.venv\Scripts\Activate.ps1 pip install -r requirements.txt Copy-Item .env.example .env # set AOAI_ENDPOINT and DEPLOYMENT az login

Step 0 — confirm the checkout works. Offline and free: no Azure calls, no tokens. If this fails, nothing after it will work.

python selftest.py # 42 offline checks

Step 1 — get the data. The test split only; training data is not needed until Step 5.

python download_sgd.py --splits test

Step 2 — the number papers report.

python eval_jga.py --mode oracle ` --deployment <your-deployment> --n 200 ` --output-dir results/oracle

Step 3 — the number your users get. One flag changes.

python eval_jga.py --mode retrieved_distractor ` --deployment <your-deployment> --n 200 ` --n-schemas 1 --n-distractors 2 ` --output-dir results/retdist

Compare results/*/summary.json. Expect roughly 58 vs 18.

Step 4 — confirm it isn't retrieval. Score only the turns where the correct schema really made it into the prompt:

python goldpresent_strata.py --results results/retdist

They stay far below the oracle number.

Step 5 — the fix, for reference. Don't run this one unless you mean it: it is a real fine-tuning job (~$150, 6–10h in queue). It is here so the recipe is something you can read, not just a link.

python download_sgd.py --splits train python build_sft_data.py --split train --n-distractors 2 --gold-absent-frac 0.2 python submit_finetune.py --train-file data/schemaraft_train.jsonl --epochs 3

The job writes finetune_job.json, so --resume reattaches instead of paying for a second run. Expect it to sit in the queue for a few hours before training starts; you can watch both phases in the Foundry portal's fine-tuning section. Once the status reads succeeded, the new model appears alongside the base models when you deploy. Point --deployment at it and re-run Step 3 — same command, same data, same metric. That is the whole experiment.

What to do with this

If you are shipping an agent that retrieves anything into its prompt — schemas, tools, MCP servers, API definitions — the cheap version of this post is one afternoon:

Re-run your own eval with distractors in the prompt. Not a better retriever, not a bigger model — just stop handing the model the right answer. Steps 2–4 above do exactly this and cost only inference.
If the number holds, you are done. Your registry is not confusable enough for this failure mode and you should not spend money on it.
If it collapses, you now know that prompting won't save you — that is what Figure 3 bought you — and the fix is a supervised fine-tuning job on the same deployment you are already using.

The part we would push back on is skipping step 1. The gap in this post was invisible to a standard evaluation, survived perfect retrieval, and survived gpt-5. It would have shipped.

Appendix — exact setup

Collected here so the narrative above stays readable and the numbers above stay auditable.

Data. Schema-Guided Dialogue (SGD) [2]: 45 services across 20 domains. The test-time retrieval registry is the 21 services in the test split; 15 of them never appear in training, which is the point of the benchmark. Full test split: 39,324 turns.

Retrieval (RetDist). BM25 over character 3–5-grams of the schema text, top-1, plus two distractor schemas sampled uniformly from the same registry. The model never sees a retrieval score — BM25's job is selection, not ranking signal. On the test split the gold schema is top-1 for 65.4% of turns; random distractor sampling surfaces it on some of the rest, leaving 31.1% of turns where the correct schema is genuinely absent from the prompt.

Evaluation populations. Three, and they must not be mixed:

Population	n	Used for
SGD-200	1,810 turns (200 dialogues)	Figure 3, the economics table, all frontier comparisons — frontier models are too expensive to run on the full split
Gold-in-prompt stratum	27,085 turns	Figure 2
Full test split	39,324 turns	Fine-tuning effects only

The gold-in-prompt stratum is every turn where the correct schema reached the prompt — whether BM25 ranked it top-1 (65.4% of turns) or a random distractor draw happened to surface it. The remaining 12,239 turns (31.1%) are the ones where it is genuinely absent.

Models. Base and fine-tuned: gpt-4.1-mini. Prompted baselines: gpt-5, Llama-3.3-70B. All three are served from the same Foundry deployment, which is what makes the latency and relative-cost columns comparable. The prompted baselines were not fine-tuned: gpt-5 exposed no SFT path on Foundry at the time of these runs, and fine-tuning a 70B model is far outside the cost envelope this post is about.

Fine-tuning. ~19,300 chat-format examples, ~2,100 prompt tokens and ~40 completion tokens each; every example carries one gold schema plus two random distractors; 20% are distractor-only and labelled {}. Supervised fine-tuning, 3 epochs, ~124M training tokens, one job on Microsoft Foundry.

Cost basis. Latency and relative cost are measured per turn on that same deployment, not quoted from a price sheet. The ~$150 figure is the one-time training job; the ~18,000-turn break-even is that cost divided by the measured per-turn saving against gpt-5.

Reproducibility. Steps 2–4 of the runbook reproduce the gap and cost only inference. Step 5 reproduces the fix and is a real fine-tuning job.

References

Zhang, T., Patil, S. G., Jain, N., Shen, S., Zaharia, M., Stoica, I., and Gonzalez, J. E. RAFT: Adapting Language Model to Domain Specific RAG. COLM, 2024.
Rastogi, A., Zang, X., Sunkara, S., Gupta, R., and Khaitan, P. Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset. AAAI, 2020. (the SGD benchmark used throughout)
Zhao, J., Gupta, R., Cao, Y., Yu, D., Wang, M., Lee, H., Rastogi, A., Shafran, I., and Wu, Y. Description-Driven Task-Oriented Dialog Modeling. ACL, 2022. (D3ST — 86.4% JGA under an oracle schema)
Bang, N., Lee, J., and Koo, M.-W. Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System. Findings of ACL, 2023. (TOATOD — 74.7% JGA under an oracle schema)
Lee, H., Gupta, R., Rastogi, A., Cao, Y., Zhang, B., and Wu, Y. SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems. AAAI, 2022. (prior robustness probe — stylistic schema variants, still oracle delivery)
Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR, 2024.

Data Meets Agents

Yeliz_Kilinc — Mon, 14 Sep 2026 16:00:00 GMT

Introduction

Enterprise agents become more valuable when they can securely access the data, knowledge and business systems organisations already rely on. And, Data Meets Agents is a way of framing a recurring architecture question we hear from customers: How should an agent access enterprise data that already resides across Microsoft Fabric, Azure Databricks, Snowflake, Google BigQuery, Amazon S3/Redshift, Salesforce and other platforms? How can we make that data and its insights available to agents in a secure and governed way? And are there repeatable architecture patterns and best practices that we can adopt across the organisation?

The challenge is not simply finding a connector. It is deciding how the agent should access that data while preserving the right security, governance, semantics, freshness and system-of-record boundaries.

Microsoft provides several ways to build agents, from Microsoft first-party agents and Microsoft 365 Agent Builder to Copilot Studio and Microsoft Foundry Agent Service, each serving different personas and levels of extensibility.

This umbrella blog briefly sets that broader agent-building landscape, but focuses on Microsoft Foundry Agent Service as a starting point and three complementary patterns for integrating enterprise data:

Direct Tool Access through APIs/MCP – for live access to data and business operations.
OneLake + Fabric IQ – for governed enterprise data, analytics and business semantics.
Foundry IQ knowledge grounding – for reusable enterprise knowledge and retrieval.

Although Azure API Management (APIM) is not explicitly shown in the simplified architecture diagrams, we strongly recommend considering it as a runtime governance layer across these integration patterns, where applicable, to provide consistent controls around API and MCP exposure, authentication, policy enforcement, observability and lifecycle management.

We’ll explore where each pattern fits, the trade-offs involved, and how they can be combined within a multi-pattern enterprise architecture.

Microsoft Agent-Building Platform Landscape

Microsoft provides several approaches to building agents, designed for different personas, use cases and levels of extensibility.

Platform / Approach	Primary audience	Development model	Typical use cases
Microsoft first-party agents	Business users	Ready-to-use / configurable	Purpose-built agent experiences across Microsoft products
Microsoft 365 Agent Builder	Information workers	No-code / declarative	Lightweight agents grounded in Microsoft 365 knowledge
Microsoft Copilot Studio	Makers and developers	Low-code with pro-code extensibility	Enterprise agents, workflows and business process automation
Microsoft Foundry Agent Service	Developers, AI engineers and architects	Pro-code with declarative options	Custom agentic applications, complex orchestration and deep enterprise integration

Across this ecosystem, Agent 365 provides the cross-platform control plane for governing the enterprise agent estate, bringing together agent identity, inventory, security, compliance, lifecycle management and observability.

Microsoft Intelligence Layer

Alongside the agent-building platforms, Microsoft provides intelligence capabilities that help agents understand and reason over different types of context.

Intelligence	Primary role
Work IQ	Organisational context from Microsoft 365, including people, communications, meetings and files
Fabric IQ	Business and analytical context through governed data, semantic models, entities and relationships
Foundry IQ	Enterprise knowledge grounding, retrieval and reusable knowledge across multiple sources
Web IQ	External context and knowledge from the web

For Data Meets Agents, however, the focus is not the IQ capabilities themselves. Our focus is the enterprise data that already resides across different data platforms and operational systems — such as Azure Databricks, Snowflake and other cloud data platforms, as well as CRM, ticketing and line-of-business applications so focus will be Fabric IQ and Foundry IQ from intelligence layer.

For this umbrella blog, we briefly establish broader agent building landscape and Microsoft IQ before focusing on Microsoft Foundry Agent Service and the three Data Meets Agents integration patterns. The next section focuses on Microsoft Foundry Agent Service and three architectural patterns for enabling agents to access that existing enterprise data.

Why Microsoft Foundry Agent Service?

Microsoft Foundry Agent Service is positioned for organisations that need greater control over agent orchestration, enterprise integration and how agents execute and interact with tools, data and other agents. Developers can build sophisticated agents and multi-agent orchestration using Microsoft Agent Framework or other code-based agent frameworks, and deploy that application logic to Foundry as Hosted Agents, retaining control over their agent and orchestration logic. Foundry provides the managed control plane and runtime capabilities around those agents, including deployment and hosting, scaling, identity and access, endpoints, tool integration, tracing, monitoring, evaluation and lifecycle management. This combination of framework flexibility, code-level control and a managed enterprise control plane makes Foundry particularly relevant for complex agentic applications that need to orchestrate across multiple enterprise data platforms, knowledge sources and business systems.

Enterprise Data Integration Patterns with Foundry Agent Service

Enterprise data already resides across cloud data platforms, databases and operational systems such as CRM, ticketing and line-of-business applications. The architectural question is therefore not simply how to connect an agent to that data, but: Where should retrieval and semantic reasoning happen, under whose identity, and where should governance be enforced?

With Microsoft Foundry Agent Service, we frame this through three complementary patterns: Direct Tool Access, OneLake + Fabric IQ, and Foundry IQ knowledge grounding. The right pattern depends on whether the agent needs live operational access, governed analytical semantics, or reusable enterprise knowledge — and a single solution may combine all three.

Pattern 1 — Direct Tool Access: APIs and MCP

The Foundry agent accesses an external platform at runtime through a governed API or MCP capability, keeping the external platform as the system of record.

Prefer when: the agent needs live data, an exact lookup, a platform-native capability, or a controlled business action — for example querying Snowflake or Databricks, retrieving a CRM record, checking a ticket or triggering a workflow.

Architecture focus: identity, least privilege, runtime governance, tool design, latency and operational reliability.

Pattern 2 — Unified Data Platform: OneLake + Fabric IQ

Although Fabric capabilities can also be exposed to Foundry through MCP, we show this as a separate architecture pattern because the data strategy is fundamentally different.

Rather than having the agent directly query each external platform, enterprise data residing in Databricks, Snowflake and other cloud data platforms can be referenced, mirrored or intentionally ingested through OneLake. Fabric then provides a governed analytical and semantic layer through capabilities such as Fabric Data Agents, ontologies and Power BI semantic models, which Foundry can consume.

Fabric is the governed data and semantic layer between agents and heterogeneous enterprise data.

Prefer when: agents need shared analytics, governed business semantics, measures, entities and relationships across enterprise data, rather than direct access to individual source systems.

Architecture focus: reference versus replication, semantic ownership, data freshness, Fabric governance and reuse across AI and analytics.

Pattern 3 — Foundry IQ Knowledge Grounding

Enterprise knowledge is exposed through Foundry IQ knowledge bases, providing reusable retrieval and grounding across indexed and remote knowledge sources.

Prefer when: agents need to discover, retrieve and reason over trusted organisational knowledge with grounding and citations, rather than perform analytical queries or operational transactions.

Architecture focus: knowledge curation, retrieval quality, permissions, citations, reuse and knowledge governance.

Quick Pattern Selection Guide

Decision Criteria	Direct as a Tool	OneLake + Fabric IQ	Foundry IQ
Data location	Remains in source	Replicated or referenced	Indexed or remotely accessed
Freshness	Runtime source result	Depends on integration/query mechanism	Depends on index freshness or remote source
Best for	Live query and controlled operation	Shared analytics and business semantics	Reusable grounding and unstructured knowledge
Main control point	MCP server + source permissions	Source + Fabric governance	Knowledge base + source permissions
Can combine?	Yes	Yes	Yes

Conclusion

The three patterns are complementary. A single Foundry agent — or a multi-agent solution orchestrated using Microsoft Agent Framework or other agent frameworks and deployed to Foundry — may use Foundry IQ for enterprise knowledge, Fabric IQ for governed analytical context, and Direct Tool Access for live queries or business actions. The architectural decision should be made per data source and use case, rather than standardising every enterprise integration on a single approach.

This post establishes the architectural framework; follow-up posts will provide implementation details for each pattern.— Direct Tool Access with APIs/MCP, Unified Data Platform with OneLake + Fabric IQ, and Foundry IQ knowledge grounding — covering architecture, identity, governance and key design trade-offs.

We’ll also explore data-source-specific architectures, looking at how enterprise data already residing in platforms such as Azure Databricks, Snowflake, Microsoft Fabric and other data platforms and business systems can be integrated with Microsoft Foundry Agent Service. Where relevant and supported, we’ll also explore how these approaches apply to Copilot Studio, Microsoft first-party agents and Microsoft 365 Agent Builder, helping architects choose the right platform, integration pattern and data architecture for each use case.

A special thanks to Li Ma and Shilfi Gafur for reviewing and contributing to this work. I’m also looking forward to collaborating with them on the upcoming Data Meets Agents blogs as we take a deeper dive into these patterns and data-source-specific architectures.

Stay Connected - 📝 Coming soon: Deep-dive blogs on each pattern (subscribe to be notified) - 💬 Questions?: Drop a comment below

Get Started - 🚀 ⚡ Try now: Quickstart: Deploy your first hosted agent - Microsoft Foundry | Microsoft Learn

Microsoft Copilot Studio vs. Microsoft Foundry: Building AI Agents and Apps | Microsoft Community Hub

Microsoft Foundry documentation | Microsoft Learn

Agent Factory: Building your first AI agent with the tools to deliver real-world outcomes | Microsoft Azure Blog

Browse all training - Training | Microsoft Learn

OneLake, the unified data lake - Microsoft Fabric | Microsoft Learn

What is Foundry IQ? - Microsoft Foundry | Microsoft Learn

Microsoft Foundry Blog articles

Your Agent Shouldn't Wait for a Prompt: Build an Event-Driven Microsoft Foundry Routine

From conversational agents to event-driven agents

The scenario: triage every new GitHub issue

Prerequisites

Create the GitHub issue routine

Give the agent a clear triage contract

Test before waiting for a real event

Choose the dispatch identity deliberately

Make failure visible

Know the current boundaries

What changes when agents stop waiting?

Try it next

Deploying hosted agents in Foundry Agent Service via Terraform

Background

Why an Infrastructure as Code Approach?

Prerequisites

Deploy the Sample

Process Overview

Technical Details

azapi_data_plane_resource

Conclusion

Foundry IQ in Microsoft Copilot Studio is now generally available

Bring enterprise knowledge into your Copilot Studio agent

Connect through a virtual network (VNet)

1. Configure the private endpoint

2. Enable Power Platform VNet support

3. Connect Foundry IQ in Copilot Studio

4. Validate before publishing

Get started

Build expressive voice experiences with new MAI models in Microsoft Foundry

MAI-Transcribe-2-Streaming: Understand speech while it’s happening

MAI-Voice-2.1: One voice across languages

MAI-Voice-2.1-Flash: Responsive speech at scale

Put the models to work together

Start building

Beyond Agent Scores: Building Consequence-Aware Release Gates with Microsoft Foundry

A better score does not always mean a safer release

The limitation of aggregate evaluation

The scenario: a plausible answer, an unacceptable action

Step 1: Trace the actual execution

Step 2: Define the Consequence-Aware Agent Release Contract

Step 3: Build a boundary-focused evaluation dataset

Step 4: Match evaluation methods to the contract

Step 5: Compare candidates without letting the average decide

Step 6: Produce an auditable Agent Release Decision Record

Step 7: Preserve the failure as release memory

Practical safeguards

Conclusion

References

From "Agent Deployed" to "Agent Ready" - Agent Acceptance Gateway

A Microsoft Foundry handover story

The challenge with poor handover

What a good handover should prove

The better pattern - Agent Acceptance Gateway

Validation flow

What the validator could look like

Configure passwordless identity

Example .env file

Run the validator

Probe prompts for configured services

Closing thought

Cohere Embed V5 brings more relevant retrieval to Microsoft Foundry

Find relevant information across text and images

Choose Fast or Pro to match your workload

Scale retrieval with more control over cost

Get started in Microsoft Foundry

Let the query decide: introducing auto retrieval reasoning effort

Introducing auto: a new retrieval reasoning effort mode that recovers 97.6% of the evidence recall of the highest effort level at 46.9% fewer query planning tokens.

Near-identical retrieval quality, lower cost

Effort that scales with the question

Get started

Appendix

Introducing GPT-6.1 Sol in Microsoft Foundry: Advanced intelligence, optimized for production agents

Claude Sonnet 5.5 is now available in Microsoft Foundry

Introducing voice agents in Microsoft Foundry

From voice API to voice-native agents – choosing the right path for building voice experiences

Build, test, and deploy voice agents with familiar developer tools

Operate voice agents with voice native lifecycle management

Get started with voice agents in Microsoft Foundry

Introducing `auto`: a new retrieval reasoning effort mode that recovers 97.6% of the evidence recall of the highest effort level at 46.9% fewer query planning tokens.