Blog Post

Microsoft Foundry Blog
7 MIN READ

An Operational Toolchain for Microsoft Foundry Private Networks

mahyagheini's avatar
mahyagheini
Icon for Microsoft rankMicrosoft
Jun 24, 2026

Enterprise teams deploying Microsoft Foundry into private networks can lose hours to failures caused not by the template itself but by surrounding operational state. Each failed deployment can cost thirty to forty-five minutes to redeploy, and residue from one attempt can block the next. 

This article introduces three tools that make private network deployments easier to operate end to end. Preflight - catches preventable failures before deployment starts. Diagnostic checks - isolate the cause when agent invocation or tool connectivity fails after a successful deployment. Cleanup - restores the environment to a clean state so redeployment can proceed without leftover platform state. 

For enterprise architects and platform teams, the operational problem is not just deployment complexity, but the hidden runtime state that private networking introduces. These tools matter because they reduce time lost to failed redeploys, shorten diagnosis when deployments appear healthy, but calls still fail, and make teardown safe enough to support repeated testing and rollout. 

What a private network Foundry deployment sets up, and where it gets fragile

A private network Foundry deployment brings together several moving parts: a Foundry account and one or more projects, an agent subnet inside a VNet delegated to Microsoft.App/environments, a Service Association Link created on that subnet by the account capability host, private endpoints for the Foundry account and other data-plane dependencies such as AI Search, Cosmos DB, and Storage, private DNS zones linked to the VNet, and role assignments that allow the project managed identity to reach each dependency over the private path. 

The main variation between deployments is which dependencies the customer brings into the architecture. Template 16 adds an existing private APIM instance so API traffic remains inside the VNet. Template 19 takes the full bring-your-own approach with customer-managed Search, Cosmos DB, and Storage resources and lets customers put tools behind vNet. 

This architecture is what makes enterprise adoption possible, but it also makes troubleshooting expensive. Any link in the chain can appear healthy in deployment output while still failing at runtime. The deployment succeeds, the portal shows no errors, and the first agent call that uses a tool fails because the relevant problem lives in runtime state the template cannot inspect. 

Preflight checks to catch preventable failures before deployment runs

The preflight script catches misconfigurations that the deployment itself cannot surface early enough to be useful: issues detectable from the inputs before any resource is created. Most of these failures interrupt deployment partway through, from an unregistered resource provider to a Bring Your Own (BYO) Cosmos account left in its default local-auth-enabled state, which prevents project capability host creation.

Preflight runs the following categories of checks against the inputs to a deployment. 

Check 

What it catches before deployment 

Required resource providers 

Mid-deploy RegistrationRequired ARM errors several minutes in 

Optional resource providers 

Same failure class but scenario specific  

Resource group + location match 

Cross-region private endpoint creation failures, private endpoints use resourceGroup().location 

Existing or soft-deleted AI accounts 

Account-name collisions that block account creation; emits the purge command for the soft-deleted case 

BYO AI Search SKU + AAD auth 

Free SKU has no private endpoint support (mid-deploy fail); no AAD path deploys cleanly and breaks the first agent call. Emits the exact az search service update fix 

BYO Cosmos DB disableLocalAuth 

Mid-deploy failure on project capability host creation. Emits the exact az resource update fix 

BYO Storage kind / APIM / Fabric existence 

Catches BYO resource IDs that don't resolve at deploy time (wrong subscription, typo, deleted resource, or no read access) 

Existing VNet location + subnet shape 

VNet in wrong region; private endpoint subnet missing, or agent-subnet missing or not delegated to Microsoft.App/environments

 

Existing SAL on the injected subnet 

SubnetIsInUseByAnotherResource mid-deploy from a previous deployment's leftover SAL 

PE subnet IP capacity 

Private endpoint creation failure partway through when the subnet runs out of addresses for the 4–6 PEs the template creates 

Private DNS zone conflicts in the resource group 

VNet-link creation failures when a privatelink.* zone already exists from a prior deployment 

Model availability + TPM quota in the region 

"Model not available" or quota-exhausted failures after the rest of the deployment has succeeded 

Cosmos DB total throughput limit 

Agent service needs at least 3000 RU/s per project (3 containers × 1000 RU/s); too low a limit fails project capability host creation 

Regional quotas: AI Search SKU, Storage account count, VNets, PEs 

Quota-exhausted mid-deploy errors (QuotaExceeded, "service quota exceeded for tier") on resources the template creates. Does not cover InsufficientResourcesAvailable, that is physical regional capacity, which Azure exposes no pre-check API for. 

Each check exits with an explicit pass, warn, or fail. For the failures whose fix is a single command, preflight emits the exact az command, so the path from "failure surfaced" to "deployment unblocked" is one paste away. 

Diagnostic script pinpoints cause of failures after a clean deployment

The diagnostic script helps identify the root cause of failures that look identical from the agent's point of view but can stem from different underlying issues, each requiring a different fix. 

Consider an agent that cannot reach Azure AI Search. The user sees a connection that returns no data and an error such as tool_user_error: Invalid endpoint or connection failed. The obvious next step is to inspect the Search resource, but the same symptom can come from several different underlying causes: 

  1. The project's managed identity may be missing Search Index Data Contributor or Search Service Contributor on the Search resource, so the bearer token reaches Search and Search rejects it.  
  2. The project-scoped capability host may not have been created yet, or may be in a Failed provisioning state because the role assignments above had not propagated when it was attempted. 
  3. The private endpoint for Search may exist and be approved, but the privatelink.search.windows.net zone may not be linked to the VNet the agent runs in, so the lookup resolves to the public IP and the network rules reject it. 

All three produce the same agent-side symptoms. Diagnostic's RBAC, capability host, and DNS-zone checks separate them in a single run, so the next action is unambiguous. 

Diagnostic runs fifteen checks against a deployed environment. Each check exits with an explicit pass, warn, or fail. 

Check 

What it Surfaces 

Discovery 

Foundry accounts present in the resource group 

Network Injection 

The account has a networkInjections configuration pointing at an agent subnet, and that subnet exists 

VNet, Subnets, Delegations, SAL 

Agent subnet has the Microsoft.App/environments delegation and the SAL the capability host placed on it 

NSG Rules 

Traffic can flow between the agent subnet and the private endpoint subnet 

DNS Zones 

Each privatelink.* zone exists and is linked to the VNet 

Custom DNS 

Whether the VNet uses custom DNS servers or Azure default DNS; if custom, reports the server IPs as a signal to verify forwarders to `168.63.129.16` are in place 

Private Endpoints + Network Rules 

Each private endpoint is approved and reachable; network ACLs allow the private endpoint subnet 

Projects and Managed Identity 

Each project exists with a managed identity assigned 

Capability Host 

Account and project capability hosts are in an active provisioning state, not Failed or still creating 

Project Connections 

Cosmos, Storage, and Search connections are wired into the project 

RBAC 

Project managed identity has the required role on every dependency it must reach 

Provisioning State 

Every Foundry-touched resource is Succeeded, not Failed or stuck 

Public Network Access + ACLs 

The lockdown is real, public network access is denied where it should be 

Model Deployment 

The model the agent is configured against is deployed and active 

Azure Policy 

No tenant policy is silently blocking what the deployment did 

Cleanup unblocks a stuck redeploy 

A common failure after a teardown is that the next redeploy refuses to run. The deployment fails with SubnetIsInUseByAnotherResource, or the VNet delete fails with InUseSubnetCannotBeDeleted. The Foundry account is gone, the agent subnet shows nothing visible using it, and nothing in the portal explains why the deployment cannot move forward. 

The deployment built a dependency chain bottom-to-top: VNet, agent subnet with Microsoft.App/environments delegation, Foundry account, account capability host plus the Service Association Link it places on the subnet, project, and project capability host. Every link is pinned by the one below it, which means cleanup must unwind in the opposite order. 

The cause of the stuck-redeploy symptom is almost always an orphaned SAL left over because the chain was torn down out of order. The usual path is the team deletes the Foundry account hoping the rest will cascade, but capability host teardown and SAL release happen asynchronously on the platform side, and the soft-deleted account is never purged, so the capability host is never released and the SAL it depends on is never released either. The account is gone from the portal, but the subnet is still pinned. In this case, redeployment will fail, and resource group is blocked and cannot get deleted. 

The cleanup script exists to resolve this in one command, and it follows the exact inverse of the dependency chain created during deployment: 

  1. Delete the project capability hosts: The project capability host depends on the account capability host; must be deleted first because the account capability host can't be deleted while project capability hosts depend on it. 
  2. Delete the account capability hosts: Only safe now that no project capability host depends on them. 
  3. Delete the projects and purge the AI accounts: Includes handling the soft-deleted state Foundry accounts go into, so the same account name can be redeployed cleanly. 
  4. Wait for SAL cleanup on the agent subnet: SAL removal is asynchronous; proceeding to delete or reuse the subnet before the SAL is gone is exactly how the SubnetIsInUseByAnotherResource error is reproduced. The wait polls the subnets it discovered from capability host properties earlier in the run, there is no API to delete a SAL directly, only the owning platform resource can release it. 
  5. Optional resource group delete: Now safe because nothing platform-internal still holds the subnet. 

Each step is idempotent, so rerunning cleanup on a partially cleaned environment completes the remaining work instead of failing. A -DryRun switch prints the full plan without changing anything, which makes it the safest way to inspect the script before using it in an environment you care about. 

Private network deployments become manageable when teams treat preflight checks, deployment, diagnosis, and teardown as one operational workflow rather than isolated tasks. Used together, preflight, diagnostic, and cleanup reduce avoidable failures, make post-deployment issues easier to localize, and prevent teardown residue from blocking the next rollout. 

 

 

Next Steps 

Want to explore the tools in more detail?  

Start with the deployment-tools and the individual documentation for preflight, diagnostic, and cleanup. 

Ready to deploy?  

Start with Template 16: private network with BYO APIM or Template 19: private network with BYO agent tools, and run preflight first.  

Updated Jun 22, 2026
Version 1.0
No CommentsBe the first to comment