The OpenVMM Project
What is OpenVMM? Virtualization platforms are made up of three key components: Firstly, there is the hypervisor – the privileged software that interfaces directly with the hardware to provide the low-level primitives required to execute virtual machines (VMs). Secondly, there is the Virtual Machine Manager (VMM) – the software that uses the hypervisor interfaces to provide the general capabilities required to run VMs and manage their lifecycle. Lastly, there is the host operating system on which the VMM runs, which provides networking and storage access. OpenVMM is an open-source, cross-platform VMM project written in Rust. Released under the MIT license and developed openly in the OpenVMM GitHub repository , it can serve as a key building block for all types of virtualization scenarios. OpenVMM is used in the OpenHCL paravisor, an execution environment that runs within the guest VM, and it can also be used as a traditional VMM on a host operating system. How Has OpenVMM Evolved Since Being Open Source? Since we open soured OpenVMM in 2024, our active developer community has grown to over 100 contributors with over 200 forks. The OpenVMM project has benefited from strong ongoing contributions from industry partners like Intel and Arm, which have helped the project evolve and mature in several aspects, including cross-platform virtualization, device emulation, testing infrastructure and confidential computing. Advancing Cross-Platform Virtualization Contributions have enhanced OpenVMM's support across architectures, host operating systems, and virtualization backends. Key enhancements include aarch64 support for the MSHV backend and correctness improvements for KVM-based virtualization. Expanding Device Model and Device Emulation Support Device support has grown to now include: - Emulated SMMUv3 for Arm-based VMs - Compute Express Link (CXL) emulation - PCIe NUMA and ACS configuration exposure - Modern VFIO device assignment interfaces for improved hardware passthrough - Full VLAN support for NetVSP networking Strengthening Testing and Engineering Infrastructure Several improvements have been made to the Petri testing framework based on community feedback. These include physical NVMe test plumbing, CXL coverage, transparent-huge-page enablement, and richer diagnostic reporting like halt reason error messages for easier debugging. In parallel, improvements in CI and build systems and release engineering have streamlined development workflows, including new build support for Azure Linux. Innovations in Confidential Computing One of the most significant areas of growth has been confidential computing. Thanks to contributions from Intel, we developed General Availability paravisor support for Intel TDX Confidential VMs in Azure in the open via over 50 GitHub issues and then focused on changes that have led to substantial performance gains, like reducing Confidential VM performance overhead from roughly 3% to about 1% in CPU/memory intensive workloads. Thanks to contributions from Arm, support for an entirely new confidential architecture, Arm CCA Realm is underway, with initial Arm CCA foundational enablement for OpenHCL and TMK test infrastructure already completed, and GICv3 support in OpenHCL and support for booting Linux guests using tmk_vmm almost complete. Collaboration with AMD continues to enhance support for AMD SEV-SNP Confidential VMs through paravisor performance optimizations and improved debugging capabilities like virtual NMI support and paravisor boot serial logging. We also continue growing paravisor attestation and sealing capabilities, like adding support for stateless Confidential VMs with hardware sealing policy. Growing Beyond the Core Project OpenVMM has also shown signs of broader ecosystem/product integration. Community members are exploring how OpenVMM can power adjacent technologies and products. For example, contributors within the Windows Subsystem for Linux (WSL) ecosystem are exploring using OpenVMM as VMM backend option for WSL VMs. As we develop new OpenVMM functionality in the open, we would love your feedback. You can learn more about this project on and how to use it on https://openvmm.dev. Why OpenVMM? The motivation behind building OpenVMM comes from three of its core characteristics: cross-platform, distributed and flexible, and its modern security architecture. Cross-Platform There is a need for a VMM that works consistently across Windows and Linux and across multiple hypervisors. Broad cross-platform support is one of the design principles of, as well as one of the key motivations for developing OpenVMM. OpenVMM supports a variety of host operating systems including Linux, Windows, and macOS, and a variety of virtualization backends including MSHV, WHP, KVM and hypervisor.framework. It also supports both x64 and AArch64 architectures. Distributed and Flexible OpenVMM is designed to be a collection of VMM components that can be combined to build specialized VMMs custom tailored to power specific scenarios. Although OpenVMM can run as a traditional VMM on the host, it is designed so that its runtime functionality can be split across multiple environments, like running in a paravisor inside a guest VM. In fact, our goal of running OpenVMM inside a paravisor environment, and this scenario’s unique threading model requirements, was one of the reasons OpenVMM was created. The ability to use OpenVMM in a variety of environments simultaneously is enabled by OpenVMM’s modular design and internal message-passing architecture. Components can be reused in different scenarios without being tied to the broader “block diagram”. OpenVMM is also flexible in terms of the wide range of scenarios in can support. It can optimize modern workloads while also supporting legacy workloads. For instance, today OpenVMM supports different boot modes: UEFI, BIOS and Linux Direct Boot, and it has wide device support across direct assigned devices, para-virtualized devices (both Virtio and VMBus), and emulated devices: vTPM, NVMe, serial UARTs and legacy x86 (i440BX + PIIX4 chipset, IDE HDD/Optical, Floppy, PCI, VGA graphics). In fact, Azure compatibility has also been a goal, which is why OpenVMM has support for the Microsoft Hypervisor ABI and all Hyper-V emulated and para-virtualized devices. Going back to OpenVMM’s modular design, one can choose a subset of the Hyper-V guest interface, or an alternate interface altogether, for new scenarios with different requirements. Modern Security Architecture OpenVMM aims to have a modern, secure design. It does this, firstly, by implementing as much virtualization functionality (emulation, instruction completion, paravirtualized devices, and memory management) in user mode as possible. This can improve security via hardware memory protection and enhanced exploit mitigations, and it also makes it easier to service the code, diagnose issues, and mitigate problems in the field, enabling faster patching of security vulnerabilities. Secondly, OpenVMM uses powerful process sandboxing, allowing unrelated VMM functionality to run in separate processes. On Windows, this sandboxing is backed by least-privileged app containers (LPAC), while on Linux it’s supported by kernel namespaces and (in the future) SeLinux. Lastly, like with many newer VMMs, OpenVMM is written in Rust, which reduces the opportunity for programming error leading to guest escape. With Safe Rust, programmers do not have to worry about type-safety or memory-safety... about the possibility of a dangling pointer, a use-after-free, or other kinds of Undefined Behaviors (which became known as nasal demons in 1992 for good reason). The OpenVMM community has been actively involved in Rust since the early days, driving the compiler and libraries forward. OpenVMM was one of the five case studies used by the Rust async working group to evaluate the utility of the stabilized MVP version of async functions in traits - OpenVMM is the Microsoft tool cited in Rust’s blog Stabilizing async in traits in 2023! There are many other great security initiatives in Rust that OpenVMM benefits from: ongoing efforts are regularly creating new open-source security tools for Rust developers, conducting audits and threat models and developing a collaborative team of Rust security experts. Conclusion Today, we covered why the OpenVMM project started and how it has grown since being open source. Microsoft is committed to working across open-source projects and supporting multiple VMMs for customers, alongside OpenVMM. Our goal is to give customers choices to find the solution that best meets their requirements and their workloads as well as to provide the strong performance and isolation capabilities of the Microsoft Hypervisor - which is used in Azure, on-prem, and Client scenarios - to a wide range of VMMs in various virtualization topologies and operating systems. For example, we are working on VMMs like Cloud-hypervisor and Hyperlight which power Azure workloads, and we are also investing in broad compatibility, getting a wide range of VMMs including QEMU to run with the Microsoft Hypervisor to support diverse Linux workloads in Azure. - Core OS Team.Announcing the open sourcing of OpenHCL
From the beginning of the cloud computing era, virtualization technology has enabled compute workloads to run as virtual machines (VMs) in a server environment. As hardware has evolved, and new functionality has become available, the software stack has kept VMs running seamlessly, thanks to sophisticated advances in the hypervisor and virtualization software. Confidential Computing is a powerful technology for significantly improving the security of VMs running in the cloud. However, the trust boundary of a Confidential VM imposes a barrier that prevents the hypervisor from offering the rich virtualization services that VMs normally expect. Customers desiring the benefits of Confidential VMs are therefore forced to update the operating systems of their VMs to newer versions, which must be continually revised as Confidential VM technology maintains its path of rapid evolution. Microsoft has embraced a different approach that offers much more flexibility to customers through the use of a “paravisor”. A paravisor executes within the confidential trust boundary and provides the virtualization and device services needed by a general-purpose operating system (OS), enabling existing VM workloads to execute securely without requiring continual update of the OS to take advantage of innovative advances in Confidential Computing. As Confidential Computing evolves, the software stack can keep VMs running seamlessly thanks to the paravisor, in much the same way other advances in virtualization software kept VMs running seamlessly on ever evolving hardware. Introducing OpenHCL Microsoft developed the first paravisor in the industry, and for years, we have been enhancing the paravisor offered to Azure customers. This effort now culminates in the release of a new, open source paravisor, called OpenHCL. We plan to develop OpenHCL in the open here: microsoft/openvmm: Home of OpenVMM and OpenHCL (github.com). Today, OpenHCL can run on both x86-64 and ARM64 platforms, it has support for Intel TDX and AMD SEV-SNP, and it offers a rich set of powerful services to VMs including device emulation via standard device interfaces, device translation via standard device interfaces, diagnostics and support for guests that are not fully enlightened - such as Windows and older versions of Linux - to run via standard architectural interfaces. OpenHCL is used in Azure in new Azure Boost SKUs, and it will be used in future Confidential VM SKUs. In the past month alone, over 1.5 Million VMs were running with OpenHCL in Azure. Open for collaboration We have much more we plan to do with OpenHCL, and as we develop new functionality in the open, we would love to collaborate with you. You can learn more about OpenHCL in our extended blog OpenHCL: the new, open source paravisor - Microsoft Community Hub as well as in https://openvmm.dev. - Core OS team.
