Today, we are introducing the AI Gateway tier of Azure API Management, now in public preview. It gives platform teams a purpose-built experience built specifically for AI workloads - publishing and governing models and MCP servers. Controls are configured through policy cards rather than XML and expressions, and the portal experience and control plane are structured around models, MCP servers, and tools rather than APIs. (For brevity, we refer to the AI Gateway tier as AI Gateway throughout the rest of this article.)

AI Gateway is built on Azure API Management, bringing proven operational capabilities to AI workloads. The resource runs in your subscription, uses your Entra tenant, and sends telemetry to destinations you control. The operating model will be familiar to existing API Management customers, but the interface is built around AI workloads. The AI Gateway tier is intended for teams that want this focused experience; other API Management tiers remain the right choice when organizations also need general-purpose API management or capabilities not included in the AI Gateway experience.

A practical model for platform teams

The AI Gateway gives platform teams a shared place to manage models, MCP servers, policies, and observability destinations, with access controlled through Azure RBAC.

For example, a central platform group can connect a set of approved models and tools and publish them for application teams. The application teams can test those assets in the test console and build against them without routing every change through the central group. The platform group still owns the shared guardrails and can see how the assets are being used. After an asset is published,  developers can create a named runtime key and begin calling the gateway immediately.

Bring the models and tools you already use

Most organizations don't standardize on a single model provider. Different models are selected based on quality, latency, cost, geography, or specialized capabilities.

The preview supports models from Microsoft Foundry including OpenAI, Anthropic, Mistral, and other Foundry hosted models, as well as models hosted in AWS Bedrock, Google Vertex AI, OpenAI, and Anthropic. A guided wizard simplifies importing models from Microsoft Foundry. Other providers can be added by configuring a connection, with backend authentication configured as part of that connection. All published models are available under the same stable endpoint. Applications continue to use supported API formats such as OpenAI Chat Completions and Responses or Anthropic Messages directly or via SDKs.

The AI Gateway extends governance beyond models to the MCP servers and tools agents use to interact with enterprise systems. You can expose an existing MCP server over SSE or Streamable HTTP, turn all or selected operations from a REST API into an MCP server by uploading its OpenAPI specification, or use more than 1,400 connector-backed tools from the Power Platform and Logic Apps library. You can also federate multiple MCP servers behind a single server, so an agent connects once and sees the tools across those servers. Backend authentication supports an API key, OAuth client credentials, managed identity, or mTLS.

Governance that's built in

Organizations need consistent governance across models and MCP servers without requiring every application team to implement those capabilities independently.

The AI Gateway portal presents governance policies through an intuitive card-based experience rather than requiring policy XML. The same policies are expressed as JSON properties, making them easy to manage as infrastructure as code and to audit and enforce across a fleet with Azure Policy. In the public preview, those cards cover request and token rate limits, token quotas, Azure AI Content Safety, and fallback to a secondary model. Policies are applied per asset, making it clear which controls protect each model or MCP server.

OpenTelemetry-based token metrics

The AI Gateway emits token-usage metrics through OpenTelemetry, with attributes following GenAI and cloud semantic conventions. Metrics can be sent to Application Insights, Datadog, Splunk, Grafana Cloud, or another OTLP endpoint. The portal provides a monitoring view over Application Insights data.

Better together: Microsoft Foundry and AI Gateway

With AI Gateway, teams can extend the same governance controls, for example token rate limits and quotas, across models hosted in Microsoft Foundry and models hosted elsewhere. Foundry and non-Foundry models are published through gateway-managed endpoints, giving applications and agents a consistent way to access governed models regardless of where they are hosted.

Foundry-hosted agents can consume curated sets of tools from Foundry toolboxes, with access to the underlying MCP servers and APIs governed through AI Gateway.

Together, Microsoft Foundry and AI Gateway cover the enterprise application lifecycle: Foundry for building and running AI applications, and AI Gateway for publishing, governing, and observing models, tools, and MCP servers across your AI estate.

The new AI Gateway tier will soon be available through the gateway experience in Microsoft Foundry portal. We are working toward a seamless, integrated AI Gateway experience within Foundry portal and will share more about that work separately.

Available today in public preview

The AI Gateway tier is available today at no cost in public preview in East US 2 and Sweden Central. Pricing will be shared separately.

To provision a resource, add a model or MCP server, and make a first call click this to go to the AI Gateway tier portal and try it.

If you prefer to start from code, use a sample to deploy all the required resources for a Foundry-hosted agent configured to access its model and tools through AI Gateway.

We look forward to your feedback as we continue to rapidly evolve AI Gateway.