Blog Post

Microsoft Foundry Blog
12 MIN READ

Cross-Region Model Connectivity Options in Microsoft Foundry: Supported Patterns and Tradeoffs

rayankhoury's avatar
rayankhoury
Icon for Microsoft rankMicrosoft
Jun 19, 2026

This article outlines the ways to leverage models across regions in Microsoft Foundry, highlights key tradeoffs, and shows that the APIM-based pattern also works in a VNet-secured cross-region setup

Model availability in Microsoft Foundry is region-dependent. The region approved for your project may not be the one where the model or Foundry Agent Service support you need is available. That creates a common cross-region design choice: connect directly to another Foundry resource, or add a gateway layer for more control and governance. 

This article complements the official Bring Your Own Model guidance by focusing specifically on cross-region scenarios with Foundry resources, including practical architecture tradeoffs and enterprise deployment patterns. 

Executive summary 

Foundry supports more than one way to access models across regions. The simplest option is when you add another Foundry resource as a connection and access models created in the foundry resource. An Azure API Management (APIM) connection adds a customer-controlled gateway for routing, policy, observability, and network isolation. This article explains the tradeoffs and then walks through an APIM-based implementation in a VNet-secured topology and cross-region patterns for Azure AI Foundry, including how to connect Foundry agents (agents built with the Foundry Agent Service) to models in other regions. 

Architecture options at a glance 

Before going deeper on the APIM implementation, it helps to visualize the main options. The diagrams below show different options. The VNet-secured variant is covered in more details later in this article. 

Direct Connection (No gateway)

Use when the goal is to reach models in another Foundry resource with the fewest moving parts.

Azure API Management AI Gateway

Use when the platform team wants a governed model surface and a customer-owned control plane in front of backend model traffic.

VNet-secured APIM variant 

In this variant, APIM ingress is itself fronted by a private endpoint in the project VNet's `pe-subnet`, resolved via the `privatelink.azure-api.net` private DNS zone. Combined with the backend Foundry account's private endpoint, the entire request path — caller → APIM → backend — stays on the Azure backbone with no public hostname in the loop.

Choosing the right cross-region model pattern 

The table below compares different options and their tradeoffs 

Pattern 

Best fit 

Control plane and auth model 

Tradeoffs 

Directly connecting to another Foundry resource  

Best when a team needs the simplest supported path to models in another Foundry resource and does not need an additional gateway tier. 

The model connection points directly to the backend Foundry project endpoint. Authentication can use API key or managed identity, and model exposure is controlled by the connection configuration. 

Lowest architectural overhead, but less room for a platform-owned policy, routing, throttling, or observability layer on every model call. 

Azure API Management model connection 

Best when the platform team wants a stable gateway surface in front of model traffic and expects routing, throttling, centralized policy, or richer telemetry requirements. 

The model connection targets an Azure API Management endpoint that fronts the backend Foundry resource. Azure API Management can normalize auth, apply policies, route across backends, and enforce governance before forwarding the call. 

Adds another service hop and requires Azure API Management design, subnet planning, and operations ownership. 

Azure API Management in front of the agent surface 

Governance and telemetry on agent ingress — throttling, rate-limit-by-key, request/response logging, schema validation, regional egress policy. 

Inbound traffic into the agent endpoint, but Foundry continues to validate the caller's token directly so MCP On-Behalf-Of, customer-data tool authorization, and per-user audit trails still work. The underlying model calls stay inside Foundry-managed surfaces. 

APIM cannot replace identity on the agent surface today — Foundry needs to see the caller (at least for MCP OBO). Treat APIM here as a governance and observability layer, not an auth layer. 

 

In practice, teams have two supported ways to reach models across regions. Direct connections are the simplest starting point. Azure API Management is useful when the platform team wants a governed model surface with centralized routing, policy, or observability. The right choice depends on ownership, control needs, and operational overhead.

What this architecture proves: agents, models, and tracing still work 

This architecture answers a key question: does the experience still hold together when models are reached cross-region through a gateway? In this setup, the answer is yes. Both Foundry agents (state-bearing agent runs in the project Playground) and prompt agents (Responses-API invocations through the same BYO connection) execute end-to-end against models reached through APIM. 

Operational workflows also remain intact. APIM tracing shows the rewritten backend URL and managed-identity token path for each model call. Per-request values such as apim-request-id and apim-trace-id continue to flow through the standard APIM telemetry surfaces that platform teams already use. The routing policy is reusable across model deployments because the /inference/deployments/{deploymentName}/chat/completions path is parameterized. APIM does not need a separate API or policy for every backend model that follows the same inference route. To expose a new model end to end, the model still needs to be deployed on the backend Foundry account and registered in the project-side AI Gateway or connected-model configuration so agents can select it. 

This URL-parameterized pattern works as written for the chat-completions surface, where the deployment name lives on the URL. The Responses API carries the model name in the JSON payload ("model": "<deployment>"), so a single APIM route cannot fan out by URL parameter alone — APIM would need a body-inspection policy that reads model from the request JSON and rewrites the backend per call. The cleaner answer for that surface is Foundry's dynamic model connection, which resolves model → backend on the project side at invocation time. 

Dynamic connection 

Dynamic model connection (Responses API). For payload-based routing — Responses API, any future surface that carries the model name in the body — Foundry's dynamic connection resolves model → backend on the project side at invocation time. The advantage is that APIM never has to inspect the request body and the AI Gateway can stay payload-agnostic. The trade-off is that the routing decision moves back into the project, so the single observable APIM URL no longer covers every model call. Teams that want centralized APIM control should stay with the parameterized chat-completions template; teams adopting the Responses API at scale should pair APIM (for backend access + Managed Identity) with the dynamic connection (for per-call dispatch). 

BYOM feature compatibility at the time of publication 

This table summarizes how current Foundry features interact with BYOM via APIM at the time this post is published. The experience continues to evolve, and teams across Foundry and related platform areas are actively improving coverage and integration over time. 

Foundry feature 

Status with BYOM via APIM 

Notes 

Chat completions / Responses API 

✅ Works 

Both surfaces invoke the BYO connection like any HTTP backend; APIM forwards to the backend Foundry account and the response is returned unchanged. 

Foundry agents (Assistants / Threads / Runs) 

✅ Works 

Agent state lives on the project side; only the model call traverses APIM. The agent runtime, tool calls, and conversation memory remain managed by Foundry. 

Prompt agents (Responses API) 

✅ Works 

Prompt-agent invocations go through the BYO connection on every call. No project-side state is required beyond the prompt definition. 

1P on-behalf-of tools (SharePoint Grounding, Fabric Data Agent, etc.) 

❌ Not supported 

Foundry returns bad_request because forwarding a 1P-token tenant context to a non-Microsoft-managed model endpoint would leak it outside the Microsoft trust boundary. Use a directly-connected Foundry deployment for those tools, or split into two agents (one for 1P-tool calls, one for BYOM calls). 

Evaluations (offline, on a dataset) 

⚠️ Partial 

Eval runs that call the model via the BYO connection execute fine. Built-in eval quality metrics that rely on Foundry-side model introspection (token-level scoring, log-probs, internal model metadata) may degrade because Foundry only sees the request/response surface that APIM exposes. 

Continuous evaluations (production) 

⚠️ Partial 

Same caveat as offline evals. Sampling/scoring works on the request-response pair; deeper telemetry that needs first-party model metadata is reduced. 

Red teaming / adversarial testing 

✅ Works 

Red-team probes are payload-level — they invoke the BYO connection like any other caller and score the response. No first-party hook into the model deployment is required. 

Foundry IQ (model evaluation suggestions) 

⚠️ Reduced fidelity 

IQ recommendations rely on Foundry's view of model usage patterns. BYO traffic shows up as opaque connection-level usage, so recommendations are coarser than for directly-connected deployments. 

Content Safety in Foundry Control Plane (input/output filters) 

⚠️ Configure on APIM 

Foundry-side content filters do not apply to BYO connections automatically. Either attach the Content Safety policy at APIM (recommended — every BYO caller is scanned in one place) or invoke Content Safety as an explicit pre/post step in the agent. 

Prompt Shields (jailbreak / indirect injection) 

⚠️ Configure on APIM 

Same as Content Safety in Foundry Control Plane — attach the Prompt Shields policy at APIM so every request is scanned before the model sees it, regardless of which agent or caller originated it. 

Fine-tuning 

❌ Not applicable to the connection 

Fine-tuning targets a specific deployment on a Foundry account, not a connection. Fine-tune on the backend Foundry account directly; once the new deployment exists, it becomes routable through the same parameterised /inference/deployments/{deploymentName}/chat/completions template with no APIM change. 

Batch API 

⚠️ Direct only 

Batch jobs use long-running deployment-scoped APIs that don't fit the synchronous APIM passthrough model. Submit batch jobs directly against the backend Foundry account; APIM is bypassed for those calls. 

 

Sample implementation topology for APIM in a vnet and core components 

This implementation uses two Foundry resources with distinct roles. The project-side resource hosts the agent-facing project in the approved region and remains VNet-integrated. The backend resource hosts model deployments in the capacity region and is exposed only through a private endpoint. APIM sits between them as the customer-owned gateway, with private DNS resolving backend FQDNs to private IPs. 

Template 16 provides the baseline private-network and APIM setup for Foundry. This extension builds on that template by adding one more subnet for the backend Foundry model resource, so APIM can reach the backend privately in a cross-region setup. 

The request flow is straightforward: the agent runs in the project-side Foundry resource, the project calls APIM through the AI Gateway connection, and APIM forwards the request privately to the backend Foundry account that hosts the model deployment. The table below maps each component in that path to its role in the overall design. 

Component 

Resource 

Notes 

Project resource 

<foundry-account> / <project-name> (project region, e.g., canadaeast) 

Hosts the agent-facing project in the approved region. The project remains VNet-integrated and uses a system managed identity. 

APIM AI Gateway 

 <apim-name> (StandardV2) 

Serves as the customer-owned gateway between the project and the backend model resource. Its system-assigned managed identity is granted Cognitive Services User on the backend account so it can forward model calls without API keys. 

Backend account 

 <backend-account> (backend region, e.g., japaneast) 

Hosts the model deployments in the capacity region. After setup, public network access is disabled so model traffic reaches the account only through the private path. 

Private endpoint 

<backend-account>-pe in the backend-pe subnet  

Provides a private, cross-region path from the project VNet to the backend Foundry account. Private DNS resolves the backend hostname to the private endpoint so APIM can reach it without using a public route. 

AI Gateway connection 

<connection-name> on the project   

The project uses this connection to send model calls to APIM by using managed identity authentication and a defined list of available models. 

Together, these components make the pattern practical for production use. Standard service names resolve privately, public network access is disabled on the backend Foundry account, model calls authenticate through APIM by using managed identity, and every model is reachable through a single observable gateway URL. 

What is actually inside the VNet today 

Today, the project VNet (<vnet-name>) contains four subnets, each with a specific role in the design: 

  • agent-subnet — delegated to Microsoft.App/environments for the Foundry agent runtime. 
  • pe-subnet — private endpoints for the project Foundry account, Storage, Cosmos DB, AI Search. 
  • apim-outbound — APIM SV2 outbound VNet integration, defaultOutboundAccess: false. 
  • backend-pe — the cross-region private endpoint to the backend Foundry account, defaultOutboundAccess: false. 

The Foundry project, the APIM outbound integration, and the cross-region private endpoint to the backend account all sit inside the same VNet. The backend Foundry account itself remains a regional managed resource in the backend region, such as japaneast, but with public network access disabled, the cross-region private endpoint becomes the only route to it. The apim-outbound and backend-pe subnets also use defaultOutboundAccess: false, which means neither subnet has implicit public internet egress. 

 

 

 

Hop 

Auth artefact 

Identity used 

Configured by 

Agent → Project 

None — in-process 

n/a 

Foundry SDK 

Project → APIM 

AAD bearer token 

Foundry Project managed identity 

validate-azure-ad-token policy needs the project MI's app ID (projectMiClientId template param) 

APIM → Backend 

Fresh AAD bearer token 

APIM managed identity 

authentication-managed-identity policy + Cognitive Services User role (Azure RBAC) on the backend account for Azure AI Services acces

Network path 

Private throughout 

n/a 

Backend publicNetworkAccess: Disabled; resolution via privatelink DNS zones; data path via the cross-region PE on the Microsoft backbone 

No API keys are stored anywhere. No APIM subscription key is required (subscriptionRequired: false on the inference API). Token rotation is handled automatically by both managed identities. 

For the full hop-by-hop trace and the source-of-truth auth-chain breakdown, see the extension README ("How the pieces are wired together" section). 

 

Consuming the connection from an agent 

 

Once the connection exists, every Foundry surface that accepts a deployment name — Foundry agents, prompt agents, raw chat-completions and Responses calls from the SDK, the portal Playground — accepts the alias <connection-name>/<deployment-name> in the same slot: 

from azure.ai.projects import AIProjectClient 
from azure.identity import DefaultAzureCredential 
 
project = AIProjectClient( 
    endpoint="https://<foundry-account>.services.ai.azure.com/api/projects/<project-name>", 
    credential=DefaultAzureCredential(), 
) 
 
agent = project.agents.create_agent( 
    model="ai-gateway/gpt-5",        # <connection-name>/<deployment-name> 
    name="cross-region-agent", 
    instructions="You are a helpful assistant.", 
)

What's Next

If you want to keep going, these are the next resources and actions that matter most. 

Deployment walkthrough 

The full implementation lives in the companion repo: foundry-samples/infrastructure/infrastructure-setup-bicep/16-private-network-standard-agent-apim-setup/extensions/byom-cross-region at main · microsoft-foundry/foundry-samples

It contains the phased Bicep modules, a PowerShell orchestrator, a preflight checker, and a sample configuration. The condensed sequence below is enough to stand up the pattern and verify the gateway path works end to end.

Prerequisites 

  • Azure CLI 2.67+ and PowerShell 7+ 
  • An Azure subscription with Contributor on the target resource group 
  • Az PowerShell module (Install-Module Az -Scope CurrentUser) 
  • Owner or Contributor + User Access Administrator on the target resource group (the template creates role assignments). 
  • Foundry Account Owner to create the Foundry account. 
  • Quota for gpt-4o/gpt-5/gpt-5.1 in the backend region, and gpt-4o in the project region. 
  • Quota for APIM StandardV2 in the project region. 
  • The Foundry project's managed-identity client ID (projectMiClientId). Look this up after creating the project; APIM needs it to validate inbound tokens. 

Deploy

git clone https://github.com/azure-ai-foundry/foundry-samples.git 
cd foundry-samples/samples/microsoft/infrastructure-setup/16-private-network-standard-agent-apim-setup/extensions/byom-cross-region 
 
az group create --name <rg> --location <project-region> 
 
az deployment group create \ 
  --resource-group <rg> \ 
  --template-file main.bicep \ 
  --parameters @samples/parameters-cross-region.json \ 
  --parameters projectMiClientId=<paste-client-id> 

After deployment, the gateway URL is in outputs.apimGatewayUrl. The connected models appear in the Foundry portal under Connected resources as <connection-name>/<deployment-name> (e.g. ai-gateway/gpt-5).

Smoke test 

The /inference API has subscriptionRequired: false because authentication is managed-identity end to end. To verify the gateway is reachable from your own session: 

$apim       = '<your-apim-name>' 
$deployment = 'gpt-5' 
 
$token = az account get-access-token --resource 'https://cognitiveservices.azure.com' --query accessToken -o tsv 
$body  = @{ messages = @(@{ role = 'user'; content = 'Say hi in five words.' }); max_tokens = 30 } | ConvertTo-Json -Compress 
 
Invoke-RestMethod -Method POST ` 
  -Uri "https://$apim.azure-api.net/inference/deployments/$deployment/chat/completions?api-version=2024-10-21" ` 
  -Headers @{ Authorization = "Bearer $token"; 'Content-Type' = 'application/json' } ` 
  -Body $body 

 A 200 with x-aigw-backend and x-aigw-region response headers proves the policy chain (MI validation → backend rewrite → cross-region PE) is healthy. For end-to-end validation via a Foundry hosted agent — agent creation, model selection, run loop — see the Use the connected model from a Foundry agent section of the extension README. 

Appendix — what the APIM trace proves 

A live POST against /inference/deployments/gpt-4o/chat/completions confirms the policy chain: APIM acquires a managed-identity token for the Cognitive Services audience, deletes the caller api-key, rewrites the backend to the japaneast Foundry account, forwards the request privately, and returns self-describing gateway headers. 

 

Inbound 

  authentication-managed-identity -> token acquired for https://cognitiveservices.azure.com 

  set-header Authorization        -> Bearer token from APIM SMI, not the caller 

  set-header api-key (delete)     -> removed 

  set-backend-service             -> https://ctcfvce-jpe.openai.azure.com/openai 

  set-header x-aigw-route         -> customer-apim-fvce to japaneast 

 

Backend 

  request-forwarder               -> POST /openai/deployments/gpt-4o/chat/completions 

  forward-request                 -> 200 OK, x-ms-region: Japan East 

 

Outbound 

  set-header x-aigw-backend       -> apim-fvce-aigw / ctcfvce-jpe 

  set-header x-aigw-region        -> canadaeast to japaneast 

  transfer-response               -> response sent to caller in full
Updated Jun 17, 2026
Version 1.0
No CommentsBe the first to comment