By Reynold D’Sa, Ricardo Bianchini, Pulkit Misra, and Anant Deval
As cloud services and AI workloads continue to grow, power has become a key constraint in expanding compute capacity. Using available power more efficiently enables Azure to deploy more servers in each datacenter, helping meet growing customer demand while improving sustainability. To address this challenge, Microsoft has taken a vertically integrated approach to infrastructure, co-designing hardware and software to optimize performance and efficiency across the stack.
With Azure Cobalt CPUs, Microsoft introduced an industry-first per-virtual machine (VM) power monitoring and capping capability, developed from the ground up and deployed in Azure. Bringing this capability into production required hardware/software co-design across the Azure stack, spanning silicon and firmware, virtualization, server agents, VM placement systems, and datacenter power management infrastructure. By managing power at the granularity of individual VMs, Azure can preserve performance for critical workloads while improving overall infrastructure efficiency.
The Need for Fine-Grained Power Management to Enhance Efficiency
A widely adopted strategy for improving datacenter power efficiency is oversubscription — adding more servers within the same power budget to harvest the unutilized power and relying on power capping when power draw exceeds the limit. However, conventional hardware-based capping applies uniform CPU throttling across all cores. While this ensures power safety, it degrades performance for all VMs, including critical customer-facing workloads. This limitation restricts how aggressively Azure can oversubscribe power, since higher oversubscription levels cause more frequent and severe throttling, thereby degrading customer workload performance.
To address these limitations, Azure initially introduced software-only per-VM power capping. This system uses VM metadata and workload predictions to selectively throttle non-critical VMs (e.g., non-production workloads, batch jobs) through a userspace Azure agent, helping protect the performance of customer-facing production applications; the figure below shows the criteria used for determining each VM’s priority. However, software-based throttling is slow since it relies on fixed frequency change steps in a feedback loop that struggles to converge quickly.
Hardware/Software Co-design for Per-VM Power Management on Cobalt
Building our own CPUs enabled Microsoft to extend software-only per-VM power capping into a more effective hardware/software co-design, with per-VM power monitoring and management built directly into Cobalt from the ground up. This industry-first capability, now deployed in Azure, preserves performance for critical customer workloads while enabling up to 20% more power oversubscription on Cobalt 200.
In our implementation, the software provides VM identifiers (IDs) and each VM’s priority to the hardware. The hypervisor communicates a VM’s ID and priority to the hardware on VM core scheduling. CPU firmware uses VM priorities to maximize performance within a power budget. When power consumption exceeds the limit, it reduces power draw in priority order by lowering the voltage and frequency (DVFS) of each VM's cores. Since VM to core assignments can change due to context switches (at milliseconds scale), the firmware samples VM priorities of cores and re-evaluates power and core frequency every millisecond. The below figure illustrates the evolution of Azure's power management approach, from conventional hardware capping to per-VM (workload-aware) power management on Cobalt.
Furthermore, the firmware uses the programmed IDs to monitor per-VM power and frequency. This telemetry is used to understand the throttling experienced by VMs, build VM power models, and support energy/carbon reporting.
Real-World Impact
- Co-designed per-VM power capping on Cobalt provides up to 24% higher performance than software-only capping under the same power budget, based on SPEC CPU Integer benchmarks. Cobalt provides the best (100%) performance for the top 5 priorities while the software-based approach can only do so for the top priority. Overall, Cobalt provides 2 – 24% higher performance.
- Prioritized capping on Cobalt 200 provides up to 20% more power oversubscription for the evaluated scenarios while preserving performance of critical VMs. It allows a 35% reduction in power with no performance impact to the critical VMs whereas no prioritization only allows a reduction of 15%. Azure can choose power savings between 15 to 35% based on tolerable performance impact to the non-critical VMs.
These results demonstrate that per-VM power management on Cobalt 200 helps preserve performance for critical workloads while improving efficiency. By applying power limits more selectively, Azure can use power more effectively — deploying more servers within its existing datacenter footprint to meet growing customer compute demand and improve sustainability.
End-to-End Integration across the Azure Stack
The figure below shows the end-to-end architecture for per-VM power management on Cobalt and highlights the hardware and software enhancements across the Azure stack. The VM scheduler places VMs in racks with available power and live migrates VMs from constrained racks using workload criticality and measured VM power. The node agent provides each VM’s ID and priority to the hypervisor, which tags the VM’s cores during scheduling. Each VM's ID enables per-VM telemetry, while its priority guides power management decisions. The per-VM telemetry is sent to Resource Central for building and refining prediction models. Finally, the power manager monitors power consumption across server groups, such as racks and rows. When a power limit is reached, it signals servers to reduce usage. The Cobalt CPU on each server then uses VM priorities to selectively throttle lower-priority workloads, preserving performance for critical applications while maintaining power safety.
Looking Ahead
This work exemplifies the power of hardware/software co-design in cloud infrastructure and is a consequence of deep collaboration across many hardware and software teams within Microsoft. As cloud workloads continue to evolve, innovations like co-designed per-VM power management will be essential to balance performance, efficiency, and sustainability.