Configuring How VMs Are Moved when a Node is Drained
Published Mar 15 2019 02:34 PM 4,927 Views
First published on MSDN on Mar 21, 2013

Introducing some concepts:


Windows Server 2012 introduced the ability to drain a node, which is sometimes also referred to as Node Maintenance Mode.  When you drain a node, the Windows Server Failover Cluster will Pause the node to prevent any new VMs from moving to that node, then it will automatically move virtual machines (VMs) and other workloads off of the node to other nodes in the cluster.  I like to call the moves that aren’t initiated by a user or an external management tool (like System Center Virtual Machine Manager), “cluster initiated moves.”

Windows Server 2012 also introduced the concept of priorities for cluster roles.  You can set the priority to High (3000), Medium (2000) or Low (1000).  The failover cluster uses this priority setting for a number of things, such as the order in which VMs are started.  It is also used to define how VMs should be moved when a node is drained.

Live migration moves VMs to other nodes without clients losing connection. However, it can use significant network bandwidth.

Quick migration involves putting a VM into saved state, moving it to another node, and then resuming from saved state.  Quick migrations are usually faster than live migrations and use less network bandwidth.  However, quick migration does cause clients to be disconnected during the move.

Virtual machines and cluster initiated moves


The default behavior for VMs is to live migrate the high and medium priority VMs, and quick migrate the low priority VMs.  The logic is that additional time and resources will be spent to move important VMs with no downtime and that it is ok for non-critical VMs to have downtime during the move.

However this behavior is fully configurable, if you wish for non-critical VMs to also be live migrated you can change the default behavior so that low priority VMs are also live migrated during cluster initiated moves. Or, you can change it so that medium, or both medium and high priority VMs use quick migration.

The “Virtual Machine” resource type has a parameter (sometimes called a private property) that is named MoveTypeThreshold.  Resource type parameters are settings that affect all cluster resources of that type.  Changing this parameter value changes how all VMs on the cluster are moved during automatic moves.

The default value for the MoveTypeThreshold parameter is 2000, this means that medium priority and any higher priorities than medium will use live migration for cluster initiated moves (like node drain).  If you set the value to 1000, this means that low priority and any priorities higher than low will use live migration.  If you set it to 3000, only high priority VMs will be live migrated, and the medium and low priority VMs will be quick migrated.  If you want to only use quick migration, set it to 3001 or higher.

Here is an example of using Windows PowerShell to get the setting and to set it:
PS C:\Windows\system32> Get-ClusterResourceType "Virtual Machine" | Get-ClusterParameter MoveTypeThreshold | fl *

ClusterObject : Virtual Machine
Name          : MoveTypeThreshold
IsReadOnly    : False
ParameterType : UInt32
Value         : 2000

PS C:\Windows\system32> Get-ClusterResourceType "Virtual Machine" | Set-ClusterParameter MoveTypeThreshold 1000

PS C:\Windows\system32> Get-ClusterResourceType "Virtual Machine" | Get-ClusterParameter MoveTypeThreshold | fl *

ClusterObject : Virtual Machine
Name          : MoveTypeThreshold
IsReadOnly    : False
ParameterType : UInt32
Value         : 1000

MoveTypeThreshold values and move types



MoveTypeThreshold


Value



VM Priority



Low



Medium



High


1000 Live Live Live
2000 Quick Live Live
3000 Quick Quick Live
3001 Quick Quick Quick


Changing the behavior for specific virtual machines


The MoveTypeThreshold parameter of the Virtual Machine resource type affects the behavior for all of the VMs in the failover cluster.  If you wish to have different behavior for different VMs that can also be accomplished, for example you want all low priority VMs to be quick migrated but there is one VM that you want to be live migrated.  Each Virtual Machine resource has a DefaultMoveType private property which by default it is set to a value of -1 (This shows as 4294967295 if you look at the value of the parameter in Windows PowerShell).  The value of -1 tells the individual Virtual Machine resource that it has no unique setting and that it should inherit its settings from the Virtual Machine resource type.

Note: DefaultMoveType is a parameter of each virtual machine’s Virtual Machine resource.  Each VM will have its own Virtual Machine resource.

DefaultMoveType parameter values and their behavior:


Value Behavior
-1 (4294967295) Use global setting (MoveTypeThreshold)
0 Turn off VM
1 Save VM (quick migration)
2 Shut down VM
3 Shut down VM (forced)
4 Live migrate VM



*Note: Value 2 and 3 have the same behavior.

Here is an example of using Windows PowerShell to get the setting and to set it:

Find the resource name:
PS C:\Windows\system32> Get-ClusterResource | ft Name,ResourceType

Name                                                        ResourceType
----                                                        ------------
Cluster Disk 1                                              Physical Disk
Cluster IP Address                                          IP Address
Cluster IP Address                                          IPv6 Address
Cluster Name                                                Network Name
Virtual Machine Configuration test1                         Virtual Machine Configuration
Virtual Machine test1                                       Virtual Machine


Get the resources’ private properties using the Get-ClusterParameter cmdlet:
PS C:\Windows\system32> Get-ClusterResource "Virtual Machine Test1" | Get-ClusterParameter | ft Name,Value

Name                                                        Value
----                                                        -----
VmID                                                        76138d6e-ff1d-45da-bce3-d6ddc46a9bae
OfflineAction                                               1
ShutdownAction                                              0
DefaultMoveType                                             4294967295
CheckHeartbeat                                              1
MigrationState                                              0
MigrationProgress                                           0
VmState                                                     3
MigrationFailureReason                                      0
StartMemory                                                 512
VirtualNumaCount                                            1


Set the private property using the Set-ClusterParameter cmdlet:
PS C:\Windows\system32> Get-ClusterResource "Virtual Machine Test1" | Set-ClusterParameter DefaultMoveType 1


Check that the private property was changed:
PS C:\Windows\system32> Get-ClusterResource "Virtual Machine Test1" | Get-ClusterParameter | ft Name,Value

Name                                                        Value
----                                                        -----
VmID                                                        76138d6e-ff1d-45da-bce3-d6ddc46a9bae
OfflineAction                                               1
ShutdownAction                                              0
DefaultMoveType                                             1
CheckHeartbeat                                              1
MigrationState                                              0
MigrationProgress                                           0
VmState                                                     3
MigrationFailureReason                                      0
StartMemory                                                 512
VirtualNumaCount                                            1
Version history
Last update:
‎Mar 15 2019 02:34 PM
Updated by: