From Silicon to Systems: Bending the Cost and Complexity Curves of AI

The next era of AI infrastructure will be defined not by a single breakthrough, but by how well we co-design every layer—from models and software to silicon, servers, racks and datacenters. At the Open Compute Project Asia Pacific Summit, we are proud to share the latest community innovations that exist to solve the variety of infrastructure demands up and down the stack that have been brought forth by the diverse workloads powering agentic AI today.

That same full-stack philosophy guides how Microsoft builds purpose-built AI infrastructure for cloud scale: co-optimizing silicon, software and systems to improve efficiency, performance and security. In our Maia 200 accelerator, we applied this approach to Microsoft’s production inference workloads, co-designing chip, model, and system to improve performance per dollar and performance per watt. The Azure Cobalt 200 CPU extends it to cloud-native and agentic computing, with an architecture designed for concurrency, orchestration and faster agent call patterns. The Azure Boost DPU brings the same discipline to storage and networking by offloading infrastructure services from host CPUs to purpose-built silicon, improving performance consistency and efficiency while reducing power, thermal footprint, and overall cost. Together, these platforms show the value of engineering hardware and software as one system—and establish the foundation for innovation that extends beyond the chip to the rack and datacenter.

Innovation beyond the chip

Across Azure, we are advancing rack-level architectures, liquid cooling and highly optimized compute and storage services alongside silicon. In our Fairwater datacenter, infrastructure is designed around the density and operating characteristics of modern AI systems. At the component level, next-generation heat exchanger units help transfer heat efficiently from high-density racks, while microfluidics points toward more direct cooling at the chip itself. Across the power path, Mt. Diablo disaggregates rack-scale power from compute, solid-state transformers simplify conversion, high-temperature superconductors offer a path to move greater amounts of power with lower resistive loss, and high-temperature fuel cells create opportunities to generate power aligned with future 800 VDC datacenter designs. These efforts are complementary: each removes a different system bottleneck, and together they create room for denser, more reliable and more sustainable AI infrastructure.

Innovation beyond the chip

Across Azure, we are advancing rack-level architectures, liquid cooling and highly optimized compute and storage services alongside silicon. In our Fairwater datacenter, infrastructure is designed around the density and operating characteristics of modern AI systems. At the component level, next-generation heat exchanger units help transfer heat efficiently from high-density racks, while microfluidics points toward more direct cooling at the chip itself. Across the power path, Mt. Diablo disaggregates rack-scale power from compute, solid-state transformers simplify conversion, high-temperature superconductors offer a path to move greater amounts of power with lower resistive loss, and high-temperature fuel cells create opportunities to generate power aligned with future 800 VDC datacenter designs. These efforts are complementary: each removes a different system bottleneck, and together they create room for denser, more reliable and more sustainable AI infrastructure.

Rearchitecting power from grid to chip

Power architecture is one of the clearest examples of why open, system-level collaboration matters. Microsoft, NVIDIA and Meta are contributing to the industry’s transition toward 800 VDC through OCP work that helps connect facility power, conversion, distribution and rack design into a practical roadmap. Moving to higher-voltage direct current can reduce conversion stages between the grid and the accelerator, helping more available power reach compute as rack densities rise. The broader architecture also supports multiple adoption paths: hybrid-compatible power racks can bring 800 VDC into facilities built around existing AC infrastructure; row-level systems can distribute power across multiple racks; and future native designs can convert medium-voltage power directly to 800 VDC. Alongside the evolving Diablo specification and the solid state transformer specification providing a medium voltage to 800VDC power conversion platform, these joint contributions give equipment providers common interfaces around which to design, test and build. Alongside the evolving Diablo specification and the LVDC solid-state transformer specification, these joint contributions give equipment providers common interfaces around which to design, test and build. That openness is essential to creating a resilient supply chain and giving operators a credible path from today’s facilities to the AI factories of the next decade.

Cooling innovation is equally important because the traditional tradeoff between density and thermals has reached its limits. Rather than treating cooling as a facility afterthought, we are co-designing it with chips, modules, servers and racks. Closed-loop liquid systems can support sustained performance while minimizing water use. Heat exchanger units can improve the efficiency and serviceability of rack-level cooling. Microfluidic approaches can bring coolant closer to where heat is generated, opening a path to lower peak temperatures and higher density. The goal is not a collection of disconnected technologies; it is an end-to-end thermal architecture that can evolve with successive generations of compute.

Architecting the next generation of AI networks

Networking must evolve with the same end-to-end discipline. Within a scale-up domain, AI systems need high bandwidth, low latency, low power and predictable reliability across tightly coupled accelerators. Our work to drive Ethernet Scale-Up Networking, or ESUN, advances open Ethernet as a common scale-up fabric, incorporating input from a broad ecosystem so designs can scale without locking the industry into a closed approach. Beyond the pod, scale-out fabrics must connect massive clusters while maintaining real-time visibility, path-level resilience and efficient utilization. Multipath Reliable Connection, or MRC, is designed for that environment: it enables traffic to use multiple paths, adapt automatically at machine timescales and support resilient, predictable training across systems exceeding 100,000 GPUs. Optics completes this story by extending scale efficiently. Through work such as the OCI-MSA, tighter integration between optical connectivity, compute and networking silicon can increase bandwidth density and system scalability. Together, MRC, Ethernet scale-up networking and scale-out optics form a continuum—from accelerator to pod, from pod to datacenter, and from datacenter to a distributed AI supercomputer.

The value of this full-stack approach compounds. Better silicon raises system efficiency. Better system design increases infrastructure utilization. Better power, cooling and networking lower the cost and complexity of training and serving models. And improvements at every layer translate into more capable, reliable and accessible AI services for customers. This is how the industry can move beyond incremental optimization and begin bending the cost, power and complexity curves together.

Accelerating together as one community

No company can solve these challenges alone. Progress depends on earlier collaboration across silicon providers, packaging partners, manufacturers, networking companies, cloud providers, datacenter operators, researchers and standards organizations. We’re proud to be part of the broader Open Compute Project Global Foundation community, driving progress in an open forum to turn individual breakthroughs into shared progress. Microsoft welcomes deeper partnership across power, cooling, networking, security, sustainability and systems design—especially within the APAC ecosystem, where so much of the world's advanced computing infrastructure comes to life.