How to Properly Shutdown a Failover Cluster or a Node

[Elden Christensen]

This blog will discuss the proper process for shutting down an individual node in a Windows Server 2012 R2 Failover Cluster or the entire cluster with all the nodes.  Note:  While the steps outlined will be specific to Windows Server 2012 R2, the process applies to a cluster of any OS version.

Shutting Down a Node

When shutting down or rebooting a node in a Failover Cluster, you first want to drain (move off) any roles running on that server (such as a virtual machine).  This ensures that the shutting down of a node is graceful to any applications running on that node. 

1. Open Failover Cluster Manager (CluAdmin.msc)
2. Click on “ Nodes ”
3. Right-click on the node name and under ‘ Pause ’ click on ‘ Drain Roles ’
4. Under Status the node will appear as ‘Paused’.  At the bottom of the center pane click on the ‘Roles’ tab.  Once all roles have moved off this node, it is safe to shut down or reboot the node.

To resume the node after it has been restarted…

When the node is once again powered on and ready to be put back into service, use the Resume action to re-enable the node to host roles. 

1. Open Failover Cluster Manager (CluAdmin.msc)
2. Click on “ Nodes ”
3. Right-click on the node name and select ‘ Resume ’, then select either:
4. ‘ Fail Roles Back ’ – This will resume the node and move any roles which were running on the node prior to the node back.  Caution:  This could incur downtime based on the role
5. ‘ Do Not Fail Roles Back ’ – This will resume the node and not move any roles back.

Shutting Down a Node with Windows PowerShell®

1. Open a PowerShell window as Administrator and type:
   Suspend-ClusterNode -Drain
2. Verify that there are no roles listed under “OwnerNode” for that node by running:
   Get-ClusterGroup
3. This could be scripted with the following syntax:
   PS C:\> (Get-ClusterGroup).OwnerNode –eq "NodeBeingDrained"

1. Shutdown or restart the computer by typing either:
   Stop-Computer
   Restart-Computer

To resume the node after it has been restarted…

1. Open a PowerShell window as Administrator and type:
   Resume-ClusterNode

Or if you wish to fail back the roles which were previously running on this node type:
Resume-ClusterNode –Failback Immediate

Shutting Down a Cluster

Shutting down the entire cluster involves stopping all roles and then stopping the Cluster Service on all nodes.  While you can shut down each node in the cluster individually, using the cluster UI will ensure the shutdown is done gracefully. 

1. Open Failover Cluster Manager (CluAdmin.msc)
2. Right-click on the cluster name, select ‘ More Actions ’, then “ Shut Down Cluster… ”
3. When prompted if you are sure you want to shut down the cluster, click “ Yes ”

Shutting Down a Cluster with PowerShell

1. Open a PowerShell window as Administrator and type:
   Stop-Cluster

Controlling VM Behavior on Shutdown

When the cluster is shut down, the VMs will be placed in a Saved state.  This can be controlled using the OfflineAction property.

To configure the shut down action for an individual VM (where "Virtual Machine" is the name of the VM):

Get-ClusterResource "Virtual Machine" | Set-ClusterParameter OfflineAction 1

Value	Effect
0	The VM is turned off
1 (default)	The VM is saved
2	The guest OS is shut down
3	The guest OS is shut down forcibly

To start the cluster after it has been shut down

1. To start the cluster type:
   Start-Cluster

Thanks!
Elden Christensen
Principal PM Manager
Clustering & High-Availability
Microsoft

[Ekw_1981]

For a file server cluster you could do something like this, which has the advantage of not needing to restart the cluster service once the nodes have been rebooted.

#Get Nodes of FSCluster

[ValidateNotNullOrEmpty()]$FSNodes = (Get-ClusterNode -Cluster $FSCluster).Name

$PrimaryFSNode = $FSNodes[0]

$AuxiliaryFSNodes = $FSNodes[1..($FSNodes.Count -1)]

#Turn off AuxiliaryFSNodes

Get-VM -Name $AuxiliaryFSNodes.Name -ComputerName $HVNodes | Stop-VM -Verbose:$Verbose

#Wait for FSCluster to failover to Primary Node

While ((Get-ClusterGroup -Cluster $FSCluster).OwnerNode -ne $PrimaryFSNode) {

Write-Verbose "Waiting for Cluster $FSCluster to failover to $PrimaryFSNode"

Start-Sleep -Seconds 10

}

#Turn off PrimaryFSNode

Get-VM -Name $PrimaryFSNode.Name -ComputerName $HVNodes | Stop-VM -Verbose:$Verbose

[rajuegypt]

Hi Elden,

 I read your important notes on Hyper-V best practises to shut down/ restart.

I need your advise, second thoughts on shutting down my infrastructure for cleaning my rack system.

1- Three Physical Servers HP Proliant

2- Hyper-V 2016 Fail Over Cluster roles

3- All VM's on three physical server

4- One HP Proliant repository for dedicated storage, not related to Three Physical server.

5- One HP Prolaint repository Storage for dedicated VEEAM BACKUP STORAGE.

Now, I have notifications on restart required due to critical updates on all three Physical Servers, including VMs where several host are running.

How can I achieve this proper shut down and restart. 6 months ago I blindly restarted servers as per notifications to restart and **bleep** my VM printer server got corrupted and unable to start.( I restored it via VEEAM).

Should I shut down all VMs first , then physical server in intervals like each servers after 5 minutes?

Please tell me the sequence or order.

Thanks and appreciate your help in advance.

[Tomaszw335]

When I do it, and shutdown the second node, the whole cluster fails

[Karl_Wester-Ebbinghaus]

Thanks Elden, if you have time, please include Windows Admin Center in your guide. 

[SystemEngineer]

Hi @Elden Christensen ,
Is this procedure still relevant and won't cause any outage when the Passive /Auxiliary node is rebooted due to Windows Update? 

[bestelle]

When following this procedure my Server 2019 cluster would crash until I found that I needed to put the server that I am going to reboot into Maintenance mode after draining the roles.   

If the server is running Windows Server 2019 or Azure Stack HCI, version 20H2, run the following cmdlet as administrator:

Get-StorageScaleUnit -FriendlyName "Server1" | Enable-StorageMaintenanceMode

If the server is running Windows Server 2016, use the following syntax instead:

Get-StorageFaultDomain -Type StorageScaleUnit | Where-Object {$_.FriendlyName -eq "Server1"} | Enable-StorageMaintenanceMode

After the server reboots immediately take it out of maintenance mode

If the server is running Windows Server 2019 or Azure Stack HCI, version 20H2, run the following cmdlet as administrator to disable storage maintenance mode:

Get-StorageScaleUnit -FriendlyName "Server1" | Disable-StorageMaintenanceMode

If the server is running Windows Server 2016, use the following syntax instead:

Get-StorageFaultDomain -Type StorageScaleUnit | Where-Object {$_.FriendlyName -eq "Server1"} | Disable-StorageMaintenanceMode

Then Resume your server into the Cluster.