Forum Discussion
Cache drive reconfiguration in Server 2025 Storage Spaces Direct cluster
We have a three node S2D cluster running Server 2025, with the storage in a 3 way mirror, running Hyper-V VMs.
Each node has 4 x NVMe drives that are currently being used as cache drives, but which are connected to a RAID controller (in HBA mode), so in the S2D configuration they appear as SSD drives rather than NVMe drives.
We've purchased the required cables and drive bays to be able to reconfigure the NVMe drives so that they're attached directly to the PCIe bus, so they'll show up as NVMe drives and hopefully give us a performance boost, so I'm just trying to plan the reconfiguration.
I was hoping it would be a relatively simple process of shutting everything down, reconfiguring the storage and bringing everything back online, but ChatGPT suggests things won't be that easy and that a complete reconfiguration of the storage would be required.
So in a nutshell, can the cache drives be reconfigured without a complete rebuild of the S2D storage ?
Cheers,
Rob
3 Replies
To eventually answer my question, yes it is possible to reconfigure the cache drives without completely rebuilding the S2D storage.
Prior to doing the work on our production cluster, I built a small test cluster to replicate the setup.
This was set up with all disks originally on a RAID controller in HBA mode.
I then swapped the RAID controller for a HBA in one node, to simulate what I planned to do on the production cluster in changing the bus type. In this case from RAID to SAS. This all worked without issues. The disks were still recognised correctly in S2D, with the cache/journal disks working as expected.
Yesterday, I reconfigured the production cluster, moving the NVMe cache disks from the RAID card to be directly connected to the PCIe bus on the system board, changing the bus type in S2D from SAS to NVMe.
Everything came up as expected without issues. This was done one node at a time, keeping the cluster up throughout. All disks were still correctly recognised in S2D and the cache working correctly.
If you're doing something other than changing the bus type, then I suspect doing some of the steps mentioned by Darryl van der Peijl would be required.
The Get-CacheDiskStatus powershell scripts from his link https://www.darrylvanderpeijl.com/storage-spaces-direct-cache-disk-status/ was very useful to prove that the cache disks were still being used correctly.
Cheers
Rob
Hello Rob,
Yes, you can replace the cache drives but it does require some shuffling.
When your node boots, it binds the capacity drives to (new) cache drives automatically. See a guide here:
How To replace S2D Cache device from SSD to NVME - jtpedersen.com IT made simple
and S2D Manual Cache Rebind--
Darryl van der Peijl
SplitbrainThanks Darryl.
I had read both of those articles already and it wasn't overly clear whether they'd apply to my scenario.
I'm hoping that as the signatures and metadata on the disks should remain the same when moving them from a SAS to NVMe connection, everything will just carry on working, but I'm struggling to find any definitive documentation that will back that up.
Cheers,
Rob
