First published on MSDN on Jul 10, 2009
Hi Cluster and Hyper-V fans,
With a standalone Windows Server 2008 R2 Hyper-V server machine (any non-cluster Hyper-V host) the amount of memory that is reserved for a VM host (physical machine) is the memory not allocated for the VM guest (virtual machine). Therefore, the user who creates the VMs can determine the amount of memory that will be reserve for the host. The user can place as many VM on a given host as long as both the host and the VM guests can operate effectively.
Similar to a standalone server, the amount of physical memory not reserved for the VM guest machines in a failover cluster node is reserve by the host. However, as part of the cluster, a host can often receive VM guests from other nodes in the cluster to keep them highly-available. This can either be a user initiated move such as live migration of a VM from another node or a resource or hardware failure that would cause the failover of VMs. As a result, the user does not have as much control over the memory that is reserve for the host. VM guests from another node can easily move to a particular node and overwhelm its memory. This would leave insufficient memory for clustering service that runs on the host to provide VM fail over and migration operations. As a result, the cluster environment variable
RootMemoryReserved
was introduced to ensure that clustered VM hosts have a minimum amount of physical memory reserve for the host.
RootMemoryReserve
Despite its name, the variable RootMemoryReserved
does not literally ensure that the root partition would have a certain amount of physical memory reserved for itself. Rather, it specifies a memory size for the host OS to compare with when the host OS is about to start a VM (which moved to that node either by user action or failover). If by starting a VM, the remaining host OS physical memory would fall below the limit specified by RootMemoryReserved plus the memory that the host currently uses, then the VM start operation would fail.
For example, on a cluster node with 16 GB of physical memory and the RootMemoryReserved is set to 1024 MB (1 GB) and the host uses 1 GB, and if each VM take up 1 GB each, the maximum number of VM that can be online is 14 VM because 1 GB of memory is already being use by the host OS and another 1 GB on top of that is reserved by the host OS. This reserves 2 GB of memory for the host. Attempts to start the 15
th
VM would bring the physical memory usage of the VMs above 15 GB, which would cause the physical memory reserve for the host OS to fall below 1 GB beyond what host OS is currently using. Thus, the VM start operation for the 15
th
VM would fail.
The RootMemoryReserved is by default set at 512 MB. This should be sufficient for the host VM that is not performing any operation other than manage the VMs. This variable can be view by the PowerShell cmdlet.
(get-cluster <cluster name>).RootMemoryReserved
To change the RootMemoryReserved, the desired reserved memory size is assigned to the PowerShell cmdlet above. Use the following PowerShell cmdlet to set RootMemoryReserved to 1024 MB:
(get-cluster <cluster name>).RootMemoryReserved=1024
Changing the RootMemoryReserved does not affect any VMs that are already running. For example, on a node with 16 GB of physical memory, if RootMemoryReserved is set to 512 MB, the host uses 480 MB at the time and the VMs are taking up 15 GB of memory, this would leave leaving the host with little free memory. If for some reason, such as another application running on the host, causes the system to slow down, changing RootMemoryReserved to 2048 MB (2 GB) would not automatically free up physical memory for the host. In this scenario, the only way to free up physical memory for the host is to offline VMs. It is therefore recommended that the desired RootMemoryReserved be set appropriately before brining any VMs online.
The maximum value for the RootMemoryReserved is 4096 MB (4 GB). Any changes to the value higher than 4 GB would be ignored and the previous value would be used. Also, RootMemoryReserved, as a cluster parameter, applies to all the nodes in the cluster. The value in RootMemoryReserved would be use to reserve the VM host memory on all the nodes in the cluster.
The RootMemoryReserved variable does not limit the amount of memory the host can use. The purpose of this variable is to ensure that the host would have a minimum amount of physical memory reserve beyond what it is currently using to control the VMs. The host can definitely use much more memory than the value set aside by RootMemoryReserved. Therefore, the amount of physical memory available for the VMs would be equal or less than the amount of memory not reserve by RootMemoryReserved.
Thanks,
Peter Huang
Software Development Engineer in Test
Clustering & High-Availability
Microsoft
