Building Microsoft’s Sovereign AI on Azure Local with NVIDIA RTX PRO and Next Gen NVIDIA Rubin

Across governments, regulated industries, and critical infrastructure, organizations are moving quickly to adopt advanced AI but with a clear requirement: AI systems should be built and operated within Sovereign boundaries. The question is no longer whether to use advanced models, but where those models should run, who controls them, and how they can evolve over time without compromising sovereignty, resilience, or trust.

Many of these customers are building Sovereign Private Clouds: environments designed to run their most sensitive and mission-critical workloads entirely within their own operational boundaries. These environments often span on-premises datacenters, edge locations, and, in some cases, fully disconnected deployments. Customers want the power of modern AI, but available on infrastructure they control.

This is where Azure Local comes in. Azure Local helps bring Azure-consistent infrastructure, lifecycle management, and governance into customer-controlled environments. As AI workloads evolve rapidly, customers are planning for the next generation of AI acceleration and need confidence that their local infrastructure can be ready – without re-architecting their environment.

NVIDIA Rubin: Powering the Next Generation of Sovereign AI on Azure Local

Sovereign AI requirements span a wide range of deployments models – from enterprise inference and development today to large-scale, sustained AI systems in the near future. Azure Local is designed to help support this evolution by providing a consistent deployment, management, and governance model across current NVIDIA Blackwell-based systems and future NVIDIA Rubin platforms.

The NVIDIA Rubin platform represents the next wave of accelerated computing, targeting large-scale workloads that go beyond basic inference. These include frontier model inference, agentic and reasoning‑based systems, and high‑throughput AI pipelines where performance, efficiency, and interconnect matter at scale.

These are the workloads Sovereign Private Cloud customers are planning for:

• National and sector specific AI models trained and operated‑ within sovereign boundaries
• Advanced reasoning and agentic systems used to help customers with defense, intelligence, and critical decision-making‑
• High-performance inference pipelines that operate‑ close to sensitive data, with low latency and strict compliance requirements

Foundry Local is built to help host these workloads directly inside customer environments, providing a modern AI platform for deploying, operating, and scaling advanced models under full customer control while remaining aligned with Azure API, tooling, and governance on Azure Local.

That’s why Azure Local will be adding support for NVIDIA Rubin platform, working closely with NVIDIA to enable customers to adopt this next generation of accelerated computing as part of their Sovereign Private Cloud strategy.

Enterprise AI Acceleration Available Today on Azure Local

While Rubin represents the next wave of large-scale AI acceleration, Azure Local also supports enterprise-grade GPUs available today to help customers begin building sovereign AI systems now.

Built on the NVIDIA Blackwell architecture, NVIDIA RTX PRO™ 6000 Blackwell Server Edition delivers powerful AI and visual computing performance for on-premises and sovereign environments. Azure Local harnesses the power of the RTX PRO 6000 for scenarios such as AI inferencing, code development, and data adjacent workloads, whether environments are connected to Azure or fully disconnected.

Support for the NVIDIA RTX PRO 6000 Blackwell Server Edition is generally available starting with Azure Local 2603 release. And coming soon, Azure Local will be adding support for the upcoming models of the Blackwell series, including the NVIDIA RTX PRO™ 4500 Blackwell Server Edition. This allows customers to start building and operating sovereign AI workloads today, while preparing for next generation platforms such as NVIDIA Rubin as they become available.

From Infrastructure to Models: Bringing AI Closer to the Data

With Foundry Local services on Azure Local, customers can deploy and operate advanced AI models directly inside their sovereign environments. This includes access to modern models and APIs from the Foundry catalog, alongside NVIDIA-optimized models and frameworks, enabling high-performance AI execution close to sensitive data.

Using Azure Kubernetes Services (AKS) on Azure Local, customers can containerize and deploy AI workloads consistently across public and sovereign environments. This supports scenarios where models are built in the cloud and operated locally, or built and run entirely within customer-controlled boundaries, while maintaining a consistent application and operations model.

Together, this enables customers to continuously evolve AI solutions over time without sacrificing control, compliance, or operational consistency.

A Platform That Evolves With Customers

Azure Local is more than infrastructure. Through Azure Arc, it helps deliver consistent management, policy enforcement, and lifecycle operations across sovereign environments, with integrated security and monitoring capabilities.

This platform approach allows Sovereign Private Cloud customers to adopt new GPU architectures, AI frameworks, and models over time, without re-architecting their environments, while maintaining a strong security posture and compliance with regulatory requirements.

Azure Local and NVIDIA will work closely with OEMs including Dell, HPE, and Lenovo to enable validated hardware platforms and GPU support over time. This helps ensure customers can adopt new AI infrastructure with confidence as part of their Sovereign Private Cloud strategy.

As AI workloads continue to advance, Azure Local remains focused on one simple goal: enabling customers to run demanding AI workloads securely, compliantly, and under their control, today and into the future.

To learn more: https://aka.ms/azurelocal