AMD Instinct MI350P GPU support will broaden accelerator choice for AI workloads on Windows Server
As enterprises put AI to work across their businesses, they need intelligence that spans the cloud and their own infrastructure. Azure provides access to advanced models and cloud-scale AI services, complemented by on-premises inference that offers an additional choice for workloads with specific data, application, and operational requirements. Together, they give customers flexibility to bring AI to more business processes.
Microsoft and AMD are working together to enable AMD Instinct™ MI350P GPU support on Windows Server, expanding customers’ options for accelerated AI inference within their own datacenters. Built on AMD CDNA 4 architecture, the MI350P is a PCIe accelerator with 144 GB of HBM3E memory designed for memory-intensive AI inference.
Enabling AMD acceleration on Windows Server
AMD has developed Windows Server driver support for the AMD Instinct MI350P GPU—a key step toward enabling customers to run accelerated AI inference alongside their existing enterprise applications. With 128 compute units, 144 GB of HBM3E memory, up to 4.0 TB/s of memory bandwidth, and with 600W PCIe form factor, the AMD Instinct MI350P GPU is designed for cost and power efficient AI deployments including memory-intensive AI inference, agentic workflows and retrieval-augmented generation (RAG) workloads. A single AMD Instinct MI350P GPU PCIe card can support multiple mid-size to small models with concurrent users or a single large model of up to 260 billion parameters delivering up to 4.6 PFLOP of inference throughput at MXFP4.
In our Windows Server announcement, we outlined investments across the hardware, platform, and runtime layers of enterprise inference. Our work with AMD is one part of that program, expanding the accelerator choices available to customers for building inference services on Windows Server.
Building on the platform enterprises trust
Windows Server runs the SQL databases, custom business applications, and critical services that organizations depend on every day. Bringing AI into these environments means more than making models available. It means operating inference services with the security, availability, and management practices these workloads require.
Those requirements vary by application. RAG workloads can involve large volumes of retrieved input text, while coding assistance needs responsive, interactive exchanges. Agentic workflows may make repeated model requests and invoke tools before completing a task. Each places different demands on token processing, concurrency, model capabilities, and infrastructure capacity.
Agentic workflows bring added complexity as enterprises seek greater operational efficiency. Enterprises need to govern agents and control the actions their tools can take. Microsoft’s broader approach addresses these needs through Microsoft Agent 365 as a governance plane for agents and Microsoft Execution Containers (MXC) as a security boundary for agent code and tools. These capabilities bring to AI agent operations the same principles of security, policy-based management, and governance that enterprises expect from Windows Server.
As part of this broader Windows Server investment, Microsoft is working to support open-weight models across technologies including vLLM, Foundry Local, Llama.cpp, and Windows ML. These technologies offer different approaches to running models for workloads such as document search, retrieval-augmented generation (RAG), and coding assistance.
Together, these investments build toward a platform where IT teams can operate inference alongside enterprise applications and choose how to serve each workload, rather than building a separate infrastructure stack for every AI use case.
Put inference to work across enterprise applications
Coding assistance and SQL-connected RAG illustrate how inference can support existing enterprise workflows. These scenarios describe application patterns; availability depends on the selected Windows Server version, hardware, drivers, and inference software. Upcoming MI350P support will add another accelerator option.
Agentic software engineering for development teams
Developers spend time understanding existing code, tracing dependencies, and investigating issues before they can make a change. AI assistance can help with that work when it has access to the relevant code and engineering documentation.
A compatible coding tool can connect to a model hosted on Windows Server, making local model inference seamlessly available within developers' existing workflows. Developers can use authorized internal context to explain a component, suggest a change, or help diagnose a problem without leaving their development tools.
An enterprise token factory can support multiple developers without requiring an accelerator at every desk. Teams can also use models through Microsoft Foundry on Azure when additional capabilities are needed and their data policies permit.
RAG for SQL-connected applications
Business applications need answers grounded in enterprise information. RAG connects a model to relevant records or document passages, giving it context to answer a user's question.
For a SQL-connected application, an embedding model can help retrieve information based on meaning rather than exact keyword matches. The application then supplies the retrieved content to a language model to generate a response. Customers can choose where embedding generation and response generation run, according to their requirements.
See local inference in action at Ignite
At Ignite, AMD and Microsoft's Windows Server team will showcase a proof of concept running an open-weight model on the AMD Instinct MI350P GPU – illustrating how locally hosted inference can support enterprise application scenarios.
Turn datacenter investment into business capability
For business leaders, the value of AI infrastructure is what it enables their organizations to do: help developers work with complex code, make enterprise information more useful, and extend the applications that keep the business running.
Evaluating that investment starts with the workload: the model quality it requires, expected usage and concurrent demand, response-time targets, and data-handling requirements. Those needs determine the inference capacity and operating costs an organization should assess before committing to a deployment.
Windows Server sits at the intersection of those applications and the datacenters that support them. This investment from Microsoft and AMD to enable AMD Instinct MI350P acceleration builds on that foundation, expanding the ways customers can put their infrastructure to work for AI.
For more information on the Windows Server platform's hybrid intelligence investments read our Windows Server 2028 blog. To learn more about AMD's work on the AMD Instinct MI350P GPU for Windows Server, see AMD's blog.