In some circumstances, Azure Batch might set the node state to unusable due to different reasons. When a node is unusable, tasks can't be scheduled to the node, but it still charges. If it’s not needed to find the root cause of nodes becoming unusable, we can locate these unusable batch nodes and delete them timely to avoid the unexpected cost. This article provides a PowerShell script that can detect all unusable nodes in a batch account/pool and delete them.
Please download the script sample from this link.
Some Tips:
1.If the auto-scale feature is enabled in a batch pool, it’s needed to disable the auto-scale before removing the nodes.
2.It’s needed to make sure the batch pool is in steady status before enabling auto-scale. Otherwise, there could be some issues. For example, the auto-scale can be enabled without any errors or warnings but it won’t work as expected, because the actual samples of metric data can’t be obtained.
3.When an unusable node is deleted, it would be very hard to troubleshoot the root cause of nodes becoming unusable. If the root cause is needed, you need to disable the auto-scale so that the problematic node can be reserved.
4.This PowerShell script can be configured in an Azure Function App with Timer triggers to detect and delete the unusable nodes periodically.
Reference links:
1.Details of unusable batch node: Node in unusable state.
2.Install the latest Batch PowerShell module: Batch PowerShell module.
Microsoft
