Bring Azure AI to your private workloads on Azure VMware Solution

Overview

A global enterprise has migrated thousands of VMware vSphere virtual machines (VMs) to Microsoft Azure as part of their application modernization strategy. Having exited their on-premises data centers and relocated their legacy application VMs to Azure VMware Solution, the next phase of that strategy is no longer about where the workloads run, but about what the organization can do with the data inside them. Those applications hold the enterprise's most valuable data, yet they sit on private networks that Azure AI services cannot reach by default. What should Azure VMware Solution look like when the goal is to make private workloads AI-accessible in place?

Azure VMware Solution is a VMware validated first party Azure service from Microsoft that provides private clouds containing VMware vSphere clusters built from dedicated bare-metal Azure infrastructure. It enables customers to leverage their existing investments in VMware skills and tools, allowing them to focus on developing and running their VMware-based workloads on Azure. That is why organizations move their VMware estates to Azure VMware Solution in the first place: they get to modernize on their own terms, lifting and shifting mission-critical applications into Azure exactly as they run today, with no rewrites, no re-platforming, and no big-bang migration.

Azure AI Foundry is the Azure platform for building, deploying, and governing AI agents, providing managed model deployments, an agent runtime, tool integration, and end-to-end tracing under Microsoft Entra identity. Agents in Foundry reach external systems through the Model Context Protocol (MCP), an open standard that describes a system to an agent as a small, explicit set of tools. MCP matters here because it turns "connect the agent to a workload" into a contract rather than a custom integration: the agent learns one way to ask, and each workload is fronted by a server that translates it.

But the "leave it as it is" strength of Azure VMware Solution has a quiet side effect. The systems holding your most valuable data, such as core databases, file servers, and line-of-business applications, keep running on private networks inside the private cloud. Meanwhile, Azure's AI services live in the Azure control plane and, by default, reach only public endpoints. When the mandate arrives to "add AI everywhere," your Azure VMware Solution workloads are often the one place AI cannot reach. Not because the data isn't valuable, but because it's private by design.

In this post, I will introduce the challenge of reaching private workloads with Azure AI, describe the Azure VMware Solution and Azure AI Foundry architectural components that close the gap, and describe the security and design considerations for doing it safely. The pattern is deliberately workload-agnostic: a managed agent in Azure AI Foundry that reaches private Azure VMware Solution systems through small, purpose-built bridges, one per workload type, while your data stays put, read-only, and inside your network. I will prove it with a working SQL example (two private databases behind a single bridge), then show how the same pattern extends to file shares, APIs, and beyond. The full deployment kit is on GitHub, so you can try it in your own environment.

In the next section, I will introduce the architectural components of Azure VMware Solution.

Architectural Components

The diagram below describes the architectural components of the Azure VMware Solution.

Figure 1 – Azure VMware Solution Architectural Components

Each Azure VMware Solution architectural component has the following function:

• Azure Subscription: Used to provide controlled access, budget, and quota management for the Azure VMware Solution.
• Azure Region: Physical locations around the world where we group data centers into Availability Zones (AZs) and then group AZs into regions.
• Azure Resource Group: Container used to place Azure services and resources into logical groups.
• Azure VMware Solution Private Cloud: Uses VMware software, including vCenter Server, NSX-T Data Center software-defined networking, vSAN software-defined storage, and Azure bare-metal ESXi hosts to provide compute, networking, and storage resources. Azure NetApp Files, Azure Elastic SAN, and Pure Cloud Block Store are also supported.
• Azure VMware Solution Resource Cluster: Uses VMware software, including vSAN software-defined storage, and Azure bare-metal ESXi hosts to provide compute, networking, and storage resources for customer workloads by scaling out the Azure VMware Solution private cloud. Azure NetApp Files, Azure Elastic SAN, and Pure Cloud Block Store are also supported.
• VMware HCX: Provides mobility, migration, and network extension services.
• VMware Site Recovery: Provides Disaster Recovery automation and storage replication services with VMware vSphere Replication. Third party Disaster Recovery solutions Zerto Disaster Recovery and JetStream Software Disaster Recovery are also supported.
• Dedicated Microsoft Enterprise Edge (D-MSEE): Router that provides connectivity between Azure cloud and the Azure VMware Solution private cloud instance.
• Azure Virtual Network (VNet): Private network used to connect Azure services and resources together.
• Azure Route Server: Enables network appliances to exchange dynamic route information with Azure networks.
• Azure Virtual Network Gateway: Cross premises gateway for connecting Azure services and resources to other private networks using IPSec VPN, ExpressRoute, and VNet to VNet.
• Azure ExpressRoute: Provides high-speed private connections between Azure data centers and on-premises or colocation infrastructure.
• Azure Virtual WAN (vWAN): Aggregates networking, security, and routing functions together into a single unified Wide Area Network (WAN).

Those components describe where your workloads run. In the next section, I will introduce the pattern that makes the data inside them reachable by Azure AI, without changing any of it.

The idea in one picture

One managed agent in Azure AI Foundry talks to your private Azure VMware Solution workloads through a set of small Model Context Protocol (MCP) servers, one bridge per workload type. SQL databases are live today; a file-share bridge or an API bridge is the same shape with different tools.

Figure 2: Solution overview

A single managed agent in Azure AI Foundry reaches private Azure VMware Solution workloads through per-workload MCP servers running on VNet-integrated Azure Container Apps, connected to the private cloud over ExpressRoute.

Three ideas carry the design:

• MCP is the universal contract. Every workload is exposed through the same small set of tools. For the SQL bridge, that’s list_sources, get_schema, and run_query. The agent learns one way to ask a question; each bridge translates it for its workload.
• One lightweight bridge per workload type. Each MCP server is a VNet-integrated Azure Container App that can legitimately see both worlds: Azure on one side, the private Azure VMware Solution network over ExpressRoute on the other. Want to make another workload AI-accessible? Add another bridge. The agent doesn’t change.
• Managed identity end to end. No passwords in code or config. The agent presents a Microsoft Entra token; each bridge pulls the credentials it needs from Azure Key Vault at runtime.

One pattern, every workload

The power of the design is that it’s workload-agnostic. The agent, the network path, the Entra identity, and the governance model are built once. Each new workload is just a new MCP server that speaks the same tool contract, so you can light up your estate incrementally, highest-value workloads first, without re-architecting anything.

Every bridge is small, independently deployable, and governed the same way: read-only by default, private over ExpressRoute, and locked to your agent’s managed identity. The pattern is the product; the bridges are how you grow it.

To make it concrete, here’s the first proof of the pattern: a secure, read-only agent over two private SQL databases in Azure VMware Solution.

Reference architecture

The reference deployment runs in a single Azure region, with the Azure resources in one resource group alongside the Azure VMware Solution private cloud.

Figure 3: Reference architecture

A question from the managed agent in Azure AI Foundry crosses into your virtual network, where the MCP server on Azure Container Apps validates the caller's Entra token, reads the database password from Key Vault with its own managed identity, and runs a single read-only SELECT against the SQL Server VMs inside the Azure VMware Solution private cloud.

The data itself lives on VMs inside the Azure VMware Solution private cloud, on a VMware NSX segment (avs-workload-01, 192.168.131.0/24):

The bridge works because the Container Apps subnet routes to the Azure VMware Solution private cloud over ExpressRoute, while Foundry is reachable privately through a private endpoint.

Figure 4: Network topology

The MCP server runs in a delegated aca-subnet and reaches the workload segment over ExpressRoute. Foundry sits behind a private endpoint with privatelink DNS zones, so the portal and jump host reach it privately. The agent’s tool call to the bridge is a single outbound HTTPS request carrying an Entra token.

The data bridge: a small, read-only MCP server

The first bridge is the SQL MCP server, a small Python service (FastMCP over streamable HTTP) that exposes three read-only tools. It’s engine-agnostic (SQLAlchemy) and pulls its credentials from Key Vault at runtime:

@mcp.tool()
def list_sources() -> str:
    """List the databases across the Azure VMware Solution workloads and what each contains."""

@mcp.tool()
def get_schema(source: str) -> str:
    """Return tables and columns for a source (call list_sources first)."""

@mcp.tool()
def run_query(source: str, query: str) -> str:
    """Run ONE read-only SELECT and return up to the source's row cap."""

Guardrails are built in. Only SELECT / WITH statements are allowed, one statement at a time; DML and DDL keywords are blocked; and every result set is row-capped. The agent gets to read and reason, but never to mutate.

Onboarding a database is configuration, not code. A single sources.yaml catalog describes each source and its governance:

sources:
  sales:
    type: mssql
    connection: { host: 192.168.131.55, port: 1433, database: SalesDB }
    auth: { kind: sql, username: agentreader, secret: sql-agentreader-password }
    governance: { allow_schemas: [dbo], max_rows: 200 }
  inventory:
    type: mssql
    connection: { host: 192.168.131.56, port: 1433, database: InventoryDB }
    auth: { kind: sql, username: agentreader, secret: sql-agentreader-password }
    governance: { allow_schemas: [dbo], max_rows: 200 }

The service is containerized (with the ODBC Driver 18 / msodbcsql18), pushed to the container registry, and run on the VNet-integrated Container Apps environment. It has a system-assigned managed identity with Key Vault Secrets User on the vault, so it can fetch the read-only database credential without ever storing it.

The managed agent in Azure AI Foundry

The agent lives entirely in Azure AI Foundry (avs-sql-foundry, project avs-sql-agent), running on a small, cost-efficient model deployment. It’s configured with:

• A schema-first workflow. Discover the available sources, read the schema, write a read-only query, then corroborate the results.
• An MCP tool pointing at the Container App’s /mcp endpoint, authenticated with Microsoft Entra using the project’s managed identity.
• Generic, source-agnostic instructions. The agent discovers structure at runtime rather than hard-coding any schema, so the same agent works against any workload behind an MCP server, not just these two databases.

Figure 5: The agent in Azure AI Foundry

The agent is attached as a tool with the private MCP data server and authenticated via managed identity. Once the tool is attached, answering a question is a single, auditable round trip. The agent discovers the available sources, reads only the schema it needs, issues a read-only query, and corroborates the results, all over private connectivity and with managed-identity authentication end to end.

Figure 6: How a question becomes an answer.

The agent calls list_sources, validates its managed-identity token, reads each schema, and runs read-only SELECTs over ExpressRoute. Key Vault supplies the credentials, and the corroborated answer flows back to the user.

Security and governance, by design

Because these are your crown-jewel systems, security is the starting point rather than a layer on top:

• No secrets in code. Database credentials live in Key Vault; the server fetches them through its managed identity (DefaultAzureCredential to SecretClient).
• Entra managed-identity auth from the agent to the bridge, with no shared keys and automatically rotating tokens.
• Private connectivity. The bridge reaches Azure VMware Solution over ExpressRoute; the SQL VMs are never exposed to the internet; Foundry sits behind a private endpoint.
• Read-only guardrails. Only SELECT is allowed; writes and DDL are blocked; result sets are capped.
• Least-privilege caller lock. The bridge accepts tokens only from the Foundry project’s managed identity.

The payoff: answers no single system can give

The value here isn’t “query a table.” It’s corroboration, the kind a single database can’t do on its own. Our two databases share a ProductID, so the agent queries both and joins them itself. Ask a plain-language question about stock health, and you get:

• Widget A: sold 500, stock 40 (below the reorder point of 50), so REORDER NOW
• Gizmo C: high stock, low sales, so OVERSTOCKED
• Contraption E: present in Sales, missing from Inventory, flagged as a DISCREPANCY

Figure 7: A natural-language question answered from two private Azure VMware Solution databases, with reorder, overstock, and discrepancy flags surfaced directly in the agent’s reply.

Figure 8: The run trace

Every tool call and query is visible and auditable (list_sources, then get_schema, then run_query on each source), so you can see exactly how the answer was produced. The same shape generalizes far beyond retail: claims versus policy, orders versus fulfillment, and tickets versus assets. Anywhere the truth is split across silos, the agent can bring it back together.

Adding the next workload

Coverage grows at two levels: within a bridge, and across bridges.

Within the SQL bridge (another database), it’s configuration, not code. SQLAlchemy gives one code path for SQL Server, PostgreSQL, MySQL, and Oracle; the source type selects the dialect. Adding a PostgreSQL VM, for example, takes no changes to the server code: add the driver, a source block (type: postgresql, allow_schemas: [public]), and a Key Vault secret, then rebuild.

Across workload types (a file share, an API), you add a new bridge that implements the same tool contract with connectors suited to that workload (read_file / search for a document store, call_operation for a REST or SOAP app), then attach it to the same agent. Nothing about the agent, the network, or the identity model changes. Your estate lights up one bridge at a time.

Deploy it yourself

The entire bridge is packaged as an open, reusable kit on GitHub, Azure VMware Solution AI Data Agent, with a Bicep template (and a compiled ARM equivalent), a parameters file, and a one-command deploy script.

# Build and push the MCP image
az acr build --registry <your-acr> --image avs-mcp:v4 .

# Deploy to the VNet-integrated Container Apps environment
az containerapp update -n avs-mcp-server -g avs-sql-rg \
  --image "<your-acr>.azurecr.io/avs-mcp:v4"

# Store the read-only SQL credential in Key Vault
az keyvault secret set --vault-name avs-sql-kv-01 \
  --name sql-agentreader-password --value '<password>'

The script provisions the delegated subnet, the container registry, Key Vault (with the read-only secret), the VNet-integrated Container Apps environment and MCP server, and the managed-identity role assignments, optionally including the Azure AI Foundry account and model deployment. You point it at the VNet already connected to your Azure VMware Solution private cloud and supply a read-only database credential. All that’s left is creating the read-only login and attaching the MCP tool to your agent in the Foundry portal.

What’s next

This SQL example is deliberately just the first bridge. A few directions we’re excited about:

• Meet users where they are. Surface the agent in Microsoft Teams or Microsoft 365 Copilot, so anyone in the organization can ask questions of Azure VMware Solution workloads in natural language.
• One agent, every workload. Grow from databases to file shares, REST/SOAP apps, NoSQL, and message systems, each a new bridge behind the same agent.
• A workload catalog. A library of ready-made bridges, so onboarding a new Azure VMware Solution workload becomes a deploy-and-configure step rather than a project.

Bringing it together

With a managed agent in Azure AI Foundry, lightweight MCP servers on VNet-integrated Azure Container Apps, and private connectivity over ExpressRoute, workloads that used to be “off-limits to AI” become conversational, while staying inside your network, read-only, and credential-less, using the same identity and governance model you already rely on across Azure.

The SQL example is just the beginning. Because every workload is reached the same way, enabling AI across your Azure VMware Solution estate becomes a repeatable motion: pick the next workload, add a small bridge, and it’s live, with no data movement, no re-platforming, and no new trust boundary.

For organizations on Azure VMware Solution, that reframes what Azure VMware Solution is for. Not just where your VMware workloads run, but where they become AI-accessible, one bridge at a time, securely and in place.

Ready to try it? Grab the Azure VMware Solution AI Data Agent kit on GitHub, point it at your Azure VMware Solution-connected VNet, and stand up your first bridge. We’d love to hear what you connect first.

Next Steps

Start by identifying which Azure VMware Solution workloads hold the highest-value data, typically core databases, document stores, and line-of-business applications, and pick one as your first bridge. Deploy the kit into the virtual network already connected to your Azure VMware Solution private cloud, create a dedicated read-only login for the agent, and validate the full path end to end in a non-production private cloud before going further.

With large enterprise solutions for strategic and major customers, an Azure VMware Solution Solutions Architect from Azure, Broadcom, or a trusted VMware Partner should be engaged to review the network path, the identity model, and the data governance boundaries before a production rollout. This should also include a data classification and application dependency assessment, to establish which sources the agent may read, where data gravity lies, and which traffic flows and latency dependencies the ExpressRoute path must support.

Beyond the first bridge, a few directions we are excited about:

• Meet users where they are. Surface the agent in Microsoft Teams or Microsoft 365 Copilot, so anyone in the organization can ask questions of Azure VMware Solution workloads in natural language.
• One agent, every workload. Grow from databases to file shares, REST/SOAP apps, NoSQL, and message systems, each a new bridge behind the same agent.

• A workload catalog. A library of ready-made bridges, so onboarding a new Azure VMware Solution workload becomes a deploy-and-configure step rather than a project.

Summary

In this post, I took a closer look at the challenge of making private Azure VMware Solution workloads reachable by Azure AI, the architectural building blocks that close the gap (a managed agent in Azure AI Foundry, lightweight per-workload MCP servers on VNet-integrated Azure Container Apps, and private connectivity over ExpressRoute), and the security and governance model that keeps your data read-only, credential-less, and inside your network. I also discussed the next steps to continue enabling Azure AI across an Azure VMware Solution estate.

The SQL example is just the beginning. Because every workload is reached the same way, enabling AI across your Azure VMware Solution estate becomes a repeatable motion: pick the next workload, add a small bridge, and it's live, with no data movement, no re-platforming, and no new trust boundary. For organizations on Azure VMware Solution, that reframes what the service is for. Not just where your VMware workloads run, but where they become AI-accessible, one bridge at a time, securely and in place.

Ready to try it? Grab the Azure VMware Solution AI Data Agent kit on GitHub, point it at your Azure VMware Solution-connected virtual network, and stand up your first bridge. I'd love to hear what you connect first.

If you are interested in Azure VMware Solution, please use these resources to learn more about the service and the Azure AI building blocks used in this post:

Azure VMware Solution

• Homepage: Azure VMware Solution
• Documentation: Azure VMware Solution
• SLA: SLA for Azure VMware Solution
• Azure Regions: Azure Products by Region
• Service Limits: Azure VMware Solution subscription limits and quotas
• Well-Architected Framework: Azure VMware Solution workloads
• Cloud Adoption Framework: Introduction to the Azure VMware Solution adoption scenario
• Enterprise Scale Landing Zone: Enterprise-scale for Microsoft Azure VMware Solution
• GitHub repository: Azure/azure-vmware-solution
• Learning Resources: Azure VMware Solution (microsoft.github.io)

Networking and connectivity

• Network connectivity scenarios: Enterprise-scale network topology and connectivity for Azure VMware Solution
• Network Security: Azure VMware Solution Network Security
• Internet Connectivity: Internet connectivity design considerations

Security and governance

• Identity & Access Management: Enterprise-scale identity and access management
• Security Recommendations: Concepts - Security recommendations for Azure VMware Solution
• Security Baseline: Azure security baseline for Azure VMware Solution
• WAF: Security considerations for Azure VMware Solution workloads
• Defender for Cloud: Integrate Microsoft Defender for Cloud with Azure VMware Solution
• Customer-managed keys: Configure customer-managed key encryption at rest

Azure AI and the agent platform

• Azure AI Foundry: Azure AI Foundry
• Foundry Agent Service: What is Azure AI Foundry Agent Service?
• Model Context Protocol: Model Context Protocol specification
• Azure Container Apps: Azure Container Apps overview
• VNet integration: Networking in Azure Container Apps environments
• Managed identities: What are managed identities for Azure resources?
• Azure Key Vault: About Azure Key Vault
• Private endpoints: What is Azure Private Endpoint?
• Deployment kit: Azure VMware Solution AI Data Agent

Tooling and automation

• Azure CLI: Azure Command-Line Interface (CLI) Overview
• PowerShell module: Az.VMware Module
• Azure Resource Manager: Microsoft.AVS/privateClouds
• REST API: Azure VMware Solution REST API
• Terraform provider: azurerm_vmware_private_cloud Terraform Registry
• Enterprise Scale GitHub repository: Azure/Enterprise-Scale-for-AVS

Author Bio

Nivas Narayanasamy is a Senior Software Engineer on the Azure VMware Solution team at Microsoft. He works across the private cloud platform, networking, and automation that customers rely on to run VMware workloads on Azure and is currently focused on secure patterns for bringing Azure AI to private enterprise data.