Cluster Shared Volumes (CSV) Cache is a feature which allows you to allocate system memory (RAM) as a write-through cache. The CSV Cache provides caching of read-only unbuffered I/O. This can improve performance for applications such as Hyper-V, which conducts unbuffered I/O when accessing a VHD or VHDX file. Unbuffered I/O’s are operations which are not cached by the Windows Cache Manager. What CSV Block Cache delivers is caching which can boost the performance of read requests, with write-through for no caching of write requests.
CSV Cache delivers caching at the block level, which enables it to perform caching of pieces of data being accessed within the VHD file. The primary difference from caching solutions in the form of a PCI card with flash which you add to the server, is that CSV Block Cache reserves its cache from system memory. The CSV Cache also tracks VM mobility and invalidates the cache when it moves from host to host, this removes the need to replicate and keep the cache coherent on all nodes in the cluster. This improves efficiency by not having to cache all VMs on all nodes, as well as reduces the performance overhead of pushing the data between nodes.
CSV Cache is completely integrated into the Failover Clustering feature and handles orchestration across the sets of nodes in the cluster.
Deployment Planning
CSV Cache will deliver the most value in scenarios where VMs are heavy read requests, and are less write intensive. Scenarios such as Pooled VDI VMs or also for reducing VM boot storms. Because the applicability of CSV Cache depends on the workload and your specific deployment considerations, it is disabled by default. The customer feedback on CSV Cache has been overwhelmingly positive and we generally recommend turning it on for all scenarios, including both Hyper-V Clusters using CSV and Scale-out File Servers using CSV.
You can allocate up to 80% of the total physical RAM for CSV write-through cache, which will be consumed from non-paged pool memory.
- Hyper-V – Our preliminary testing has found 512 MB to deliver excellent gain at minimal cost, and is the recommend starting point / minimal value if enabled. Then based on your specific deployment and the I/O characteristics of the workloads in the VMs you may wish to increase the amount of memory allocated. Since system memory is a contended resource on a Hyper-V cluster, it is recommended to keep the CSV Cache size moderate. Such as 512 MB, 1 GB, or 2 GB
- Scale-out File Server – It is recommended to allocate a significantly larger CSV Cache on a SoFS as physical memory is typically not a contended resource, you may want to allocate 4 GB, 6 GB, or even more...
There are two configuration settings that allow you to control CSV Cache.
- BlockCacheSize – This is a cluster common property that allows you to define how much memory (in megabytes) you wish to reserve for the CSV Cache on each node in the cluster. If a value of 512 is defined, then 512 MB of system memory will be reserved on each node in the Failover Cluster. Configuring a value of 0 disables CSV Block Cache.
- EnableBlockCache – This is a private property of the cluster Physical Disk resource. It allows you to enable/disable caching on an individual disk. This gives the flexibility to configure cache for read intensive VMs running on some disks, while allowing to disable and prevent random I/O on other disks from purging the cache. For example parent VHD’s with high reads you would enable caching on Disk1, and high writes for differencing disks the CSV cache could be disabled on Disk2. The default setting is 1 for enabled.
Configuring CSV Cache
The CSV Cache is disabled by default, you only need to define how much memory you want to allocate to enable it with the following process:
- Open an elevated Windows PowerShell prompt
- Define the size of the size of the cache to be reserved (example of setting to 1 GB)
(Get-Cluster). BlockCacheSize = 1024
Disabling on a per disk basis
Once the CSV Cache is enabled, all disks on all nodes will be cached by default. You have the flexibility to disable the CSV Cache on an individual disk using the following process:
Get-ClusterSharedVolume “Cluster Disk 1” | Set-ClusterParameter EnableBlockCache 0
Optimizing CSV Cache
The CSV Cache also provides a set of counters you can use to monitor the performance of the cache. You can leverage the Performance Monitor tool (PerfMon.msc) to add the following counts to monitor different aspects of the CSV Cache.
Open Performance Monitor, and under Add Counters you will find “Cluster CSV Volume Cache” with the following counters.
I/O satisfied from cache:
- Cache IO Read-Bytes
- Cache IO Read-Bytes/Sec
- Cache Read
- Cache Read/Sec
I/O satisfied from disk:
- Disk IO Read-Bytes
- Disk IO Read-Bytes/Sec
- Disk Read
- Disk Read/Sec
Total I/O:
- IO Read-Bytes
- IO Read-Bytes/Sec
- IO Read
- IO Read/Sec
Considerations:
The CSV Cache was introduced in Windows Server 2012 and the above applies to all releases following. The CSV Cache has evolved over the releases, and below are a list of considerations with previous releases .
Windows Server 2012 R2:
- Enabling CSV Cache on an individual disk requires that the Physical Disk resource be recycled (taken Offline / Online) for it to take effect. This can be done with no downtime by simply moving ownership of the Physical Disk resource from one node to another.
- Recommended not to exceed allocating 64 GB on Windows Server 2012 R2
- CSV Cache will be disabled on:
- ReFS volume when integrity streams is enabled
- Tiered Storage Space with heat map tracking enabled
- Deduplicated files using in-box Windows Server Data Deduplication feature (Note: Data will instead be cached by the dedup cache)
Windows Server 2012
- Maximum of 20% of the total physical RAM can be allocated for the CSV write-through cache with Windows Server 2012
- The cache size can be modified with no downtime, however for the Hyper-V root memory reserve in the parent partition to be modified to accommodate the memory allocated to the CSV cache it does require a server reboot with Windows Server 2012. To ensure resource contention is avoided, it is recommended to reboot each node in the cluster after modifying the memory allocated to the CSV cache.
- Enabling CSV Cache on an individual disk requires that the Physical Disk resource be recycled (taken Offline / Online) for it to take effect. This can be done with no downtime by simply moving ownership of the Physical Disk resource from one node to another.
- The EnableBlockCache private property is named CsvEnableBlockCache in Windows Server 2012
- The BlockCacheSize common property is named SharedVolumeBlockCacheSizeInMB in Windows Server 2012
- The way it is enabled is also slightly different, here is the process:
- Open an elevated Windows PowerShell prompt
- Define the size of the size of the cache to be reserved (example of setting to 1 GB)
(Get-Cluster). SharedVolumeBlockCacheSizeInMB = 1024
- Enable CSV Cache on an individual disk (must be executed for every disk you wish to enable caching)
Get-ClusterSharedVolume “Cluster Disk 1” | Set-ClusterParameter CsvEnableBlockCache 1
Thanks!
Elden Christensen
Principal PM Manager
Clustering & High-Availability
Microsoft