Capabilities of NVMe-oF so far?
Started playing around with NVMe-oF last night and got a successful connection to TrueNAS. I'm wondering what the current capabilities/limitations are. As far as I can tell you can't cluster the disk which is what I was primarily interested in. I was able to connect the same NQN endpoint to two servers but the WSFC cluster wouldn't detect it as a cluster available disk. Also the disks seems to disappear on every reboot and has to be set up again, so is there no auto-mount yet? I understand this is early preview so I don't have many expectations, just would like some clarity. Thank you.Updated Failover Clustering API Documentation
Does anyone know when the Failover Cluster API documentation will be updated? Currently, in learn it only shows Server 2008 and 2012. https://learn.microsoft.com/en-us/previous-versions/windows/desktop/mscs/failover-cluster-apis-portal The same goes for storage spaces. Is there any updated documentation?
CSV Auto-Pause on Windows Server 2025 Hyper-V Cluster
Hi everyone, i'm facing a very strange behavior with a newly created HyperV Clsuter running on Windows Server 2025. One of the two nodes keep calling for autopause on the CSV during the I/O peak. Does anyone have experienced this ? Here are the details : Environment Cluster: 2-node Failover Cluster Nodes: HV1 & HV2 (HPE ProLiant DL360 Gen11) OS: Windows Server 2025 Datacenter, Build 26100.32370 (KB5075899 installed Feb 21, 2026) Storage: HPE MSA 2070 full SSD, iSCSI point-to-point (4×25 Gbps per node, 4 MPIO paths) CSV: Single volume "Clsuter Disk 2" (~14 TB, NTFS, CSVFS_NTFS) Quorum: Disk Witness (Node and Disk Majority) Networking: 4×10 Gbps NIC Teaming for management/cluster/VMs traffic, dedicated iSCSI NICs Problem Description The cluster experiences CSV auto-pause events daily during a peak I/O period (~10:00-11:30), caused by database VMs generating ~600-800 MB/s (not that much). The auto-pause is triggered by HV2's CsvFs driver, even though HV2 hosts no VMs. All VMs run on HV1, which is the CSV coordinator/owner. Comparative Testing (Feb 23-26, 2026) Date HV2 Status Event 5120 SMB Slowdowns (1054) Auto-pause Cycles VM Impact Feb 23 Active 1 44 1 cycle (237ms recovery) None Feb 24 Active 0 8 0 None Feb 25 Drained (still in cluster) 4 ~60 (86,400,000ms max!) 3 cascade cycles Severe - all VMs affected Feb 26 Powered off 0 0 0 None Key finding: Draining HV2 does NOT prevent the issue. Only fully powering off HV2 eliminates all auto-pause events and SMB slowdowns during the I/O peak. Root Cause Analysis 1. CsvFs Driver on HV2 Maintains Persistent SMB Sessions to CSV SMB Client Connectivity log (Event 30833) on HV2 shows ~130 new SMB connections per hour to the CSV share, continuously, constant since boot: Share: \\xxxx::xxx:xxx:xxx:xxx\xxxxxxxx-...-xxxxxxx$ (HV1 cluster virtual adapter) All connections from PID 4 (System/kernel) — CsvFs driver 5,649 connections in 43.6 hours = ~130/hour Each connection has a different Session ID (not persistent) This behavior continues even when HV2 is drained 2. HV2 Opens Handles on ALL VM Files During the I/O peak on Feb 25, SMB Server Operational log (Event 1054) on HV1 showed HV2 blocking on files from every VM directory, including powered-off VMs and templates: .vmgs, .VMRS, .vmcx, .xml — VM configuration and state files .rct, .mrt — RCT/CBT tracking files Affected VMs: almost all Also affected: powered-off VMs And templates: winsrv2025-template 3. Catastrophic Block Durations On Feb 25 (HV2 drained but still in cluster): Operations blocked for 86,400,000 ms (exactly 24 hours) — handles accumulated since previous day These all expired simultaneously at 10:13:52, triggering cascade auto-pause Post-autopause: big VM freeze/lag for additional 2,324 seconds (39 minutes) On Feb 24 (HV2 active): Operations blocked for 1,150,968 ms (19 minutes) on one of the VM files Despite this extreme duration, no auto-pause was triggered that day 4. Auto-pause Trigger Mechanism HV2 Diagnostic log at auto-pause time: CsvFs Listener: CsvFsVolumeStateChangeFromIO->CsvFsVolumeStateDraining, status 0xc0000001 OnVolumeEventFromCsvFs: reported VolumeEventAutopause to node 1 Error status 0xc0000001 (STATUS_UNSUCCESSFUL) on I/O operation from HV2 CsvFsVolumeStateChangeFromIO = I/O failure triggered the auto-pause HV2 has no VMs running — this is purely CsvFs metadata/redirected access 5. SMB Connection Loss During Auto-pause SMB Client Connectivity on HV2 at auto-pause time: Event 30807: Share connection lost - "Le nom réseau a été supprimé" Event 30808: Share connection re-established What Has Been Done KB5075899 installed (Feb 21) — Maybe improved recovery from multi-cycle loop to single cycle a little, but did not prevent the auto-pause Disabled ms_server binding on iSCSI NICs (both nodes) Tuned MPIO: PathVerification Enabled, PDORemovePeriod 120, RetryCount 6, DiskTimeout 100 Drained HV2 — no effect Powered off HV2 — Completely eliminated the problem I'm currently running mad with this problem, i've deployed a lot of HyperV clusters and it's the first time i'm experiencing such a strange behavior, the only workaround i found is to take the second nodes off to be sure he is not putting locks on CSV files. The cluster is only running well with one node turned on. Why does the CsvFs driver on a non-coordinator node (HV2) maintain ~130 new SMB connections per hour to the CSV, even when it hosts no VMs and is drained?Why do these connections block for up to 24 hours during I/O peaks on the coordinator node? Why does draining the node not prevent CsvFs from accessing the CSV? Is this a known issue with the CsvFs driver in Windows Server 2025 Build 26100.32370? Are there any registry parameters to limit or disable CsvFs metadata scanning on non-coordinator nodes ? If someone sees somthing that i am missing i would be so grateful ! Have a great day.Migrating from VMware to Hyper-v
Hi, I've recently deployed a new 3x node Hyper-v cluster running Windows Server 2025. I have an existing VMware cluster running exsi 7.x. What tools or approach have you guys used to migrate from VMware to Hyper-v? I can see there are many 3rd party tools available, and now the Windows Admin Center appears to also support this. Having never done this before (vmware to hyper-v) I'm not sure what the best method is, does anyone here have any experience and recommendations pls?
PS script for moving clustered VMs to another node
Windows Server 2022, Hyper-V, Failover cluster We have a Hyper-V cluster where the hosts reboot once a month. If the host being rebooted has any number of VMs running on it the reboot can take hours. I've proven this by manually moving VM roles off of the host prior to reboot and the host reboots in less than an hour, usually around 15 minutes. Does anyone know of a powershell script that will detect clustered VMs running on the host and move them to another host within the cluster? I'd rather not reinvent this if someone's already done it.
Cache drive reconfiguration in Server 2025 Storage Spaces Direct cluster
We have a three node S2D cluster running Server 2025, with the storage in a 3 way mirror, running Hyper-V VMs. Each node has 4 x NVMe drives that are currently being used as cache drives, but which are connected to a RAID controller (in HBA mode), so in the S2D configuration they appear as SSD drives rather than NVMe drives. We've purchased the required cables and drive bays to be able to reconfigure the NVMe drives so that they're attached directly to the PCIe bus, so they'll show up as NVMe drives and hopefully give us a performance boost, so I'm just trying to plan the reconfiguration. I was hoping it would be a relatively simple process of shutting everything down, reconfiguring the storage and bringing everything back online, but ChatGPT suggests things won't be that easy and that a complete reconfiguration of the storage would be required. So in a nutshell, can the cache drives be reconfigured without a complete rebuild of the S2D storage ? Cheers, RobBLOG: Determine and modernize Filesystem Deduplication
Version history - 1.6 Added references / links - 1.5 Added insights from Steven Ekren. Many thanks! / Added ReFS Docs link and added clarification about drawbacks. - 1.4 revised script so ReFS volumes with classic dedup will be identified, added more eligibly checks and error handling - 1.3 added point #4 in migration guidance - 1.2 revised script - 1.1 formatting This blog explains the two Windows deduplication modes classic Windows Data Deduplication (ReFS or NTFS) and ReFS Deduplication (ReFS). It covers how they differ, why you should consider upgrading to Windows Server 2025 to leverage the new ReFS dedup engine, and clear warnings about scenarios where ReFS is not recommended. Practical migration guidance and detection commands are included. Differences between classic dedup and ReFS dedup File system: Classic dedup runs on NTFS or ReFS; ReFS dedup runs on ReFS and Windows Server 2025 or later, only. Implementation: They are separate engines with different metadata formats and management cmdlets. Management: Classic dedup uses the Dedup PowerShell module (Get‑DedupVolume, Start‑DedupJob, Disable‑DedupVolume). ReFS dedup uses its own ReFS dedup cmdlets (Get‑ReFSDedupStatus, Enable‑ReFSDedup). Conversion: There is no in‑place conversion between the two; metadata and chunk formats are incompatible. Improvements: the new in-line ReFS Deduplication leverages the advantages of ReFS files system. This makes deduplication more efficient and less CPU intensive. The new ReFS Deduplication can also compress data in-line using L1Z algorithm. This makes it up to par with enterprise solutions, often found in SAN storage or Linux appliances. Compression needs to be set per volume, and optional. Edit: Steven Ekren, a former Senior Product Manager for Hyper-V shared valuable insights on how both engines operate in a comment on LinkedIn: [...] the basic conceptual difference between WS Deduplication and ReFS deduplication is that the Windows Server [dedup] version takes the duplicate file data and moves it to a repository and puts a reparse point in the file system from each point that references the data. This involves data movement and therefore not recommended for workloads that are changing it's data often, but best for more static data like documents and picture/videos. ReFS is a file system that uses links natively for all the objects so leaving the data in place and managing the links is much more efficient and doesn't involve the data copy and managing a repository. Effectively it's built into the file system. As the blog notes, there are some situations not recommended for this version of dedupe, but generally it's lower performance and storage I/O impact. Why upgrade to Windows Server 2025 Improved version of ReFS Filesystem Improved ReFS in-line deduplication + optional L1Z compression: Server 2025 includes enhancements to ReFS dedup performance, scalability, and integration with modern storage features. Support and fixes: Windows Server 2016 and 2019 are past mainstream support, increasing the likelihood of costly support cases and delayed fixes; upgrading reduces operational risk and ensures access to ongoing improvements. Future compatibility: Newer OS releases receive optimizations and bug fixes for ReFS and dedup scenarios that older releases will not. SMB compression: for reasonably faster data transfer at minimal CPU when transferring data through the networks. Feature and security related improvements refer to availabile Microsoft Windows Server 2025 Summit content on techcommunity.microsoft.com Scenarios where ReFS is not recommended ReFS on SAN in clustered CSV environments: Avoid placing ReFS dedup on top of SAN‑backed Cluster Shared Volumes (CSVFS) in production clusters; clustered SAN/CSV scenarios causing severe performance issues in practice. Please refer to the ReFS documentation. (personal opinion and experience, not endorsed by Microsoft): Many small, fast‑changing files: Workloads with frequent small writes, such as user profiles, folder redirection of AppData folders, or applications that churn small config files (for example, Lotus Notes config files) can cause locks, performance degradation, or unexpected behavior on ReFS. Exclude these disks from dedup or keep them on NTFS. Note: Restrictions on high churn rate, like lockups or high RAM consumption, deadlocks / BSOD might have been addressed in Windows Server 2025 and the ReFS Dedup, see comment of Steven Ekren. Improving reliability and performance is a top goal for ReFS, to improve the adoption and feature parity with NTFS. For information about feature parity please refer to the ReFS documentation. Migration guidance The following instructions describe a high level and supported migration path from Windows deduplication using the NTFS file system to native ReFS Deduplication. Note: Step #3, data migration is not required when already using ReFS with Data Deduplication. In this case it's enough to execute step #1 and #2. Note: Validate on non‑production data first. Plan for rehydration time and network/storage throughput. Ensure backups are current before starting. Make sure to have a full backup before upgrading Server OS or making changes. 1. Disable classic dedup on the NTFS source: Disable-DedupVolume -Volume YourDriveLetter: 2. Rehydrate (un‑deduplicate) the data: Start-DedupJob -Volume YourDriveLetter: -Type Unoptimization 3. Copy or move data to a ReFS volume (new target): For straightforward NTFS→ReFS copies, robocopy is recommended. A GUI and job based alternative to this is the File Server Migration Feature (uses robocopy) in Windows Admin Center. For complex scenarios, open files long path names very large datasets (< 5 TB) or many small files restructuring, GUI (including Windows Server Core) automation, improved logging cloud/hybrid migrations I recommend the usage of GS RichCopy Enterprise by GuruSquad for higher speed (up to 40%) and reliability, compared to robocopy. 4. Optionally remove the Windows Server feature When there is no old deduplication in use consider to remove the feature. Your advantages of doing so: removes an unneccessary service. removes the file system filter driver for dedup, which causes performance impacts, even when not in use. removes the PowerShell commandlets for the old dedup, so they cannot mistakenly used by existing scripts, unaware admins etc. When migrating files over network: SMB compression: consider both source and target run Windows Server 2025 and leverage SMB compression. SMB Compression is available in Microsoft xcopy, Microsoft robocopy and Gurusquad GScopy Enterprise. Balancing and Teaming with SMB: SMB does not require LFBO or SET Teaming. It automagically detects network links and actively balances on its own on Windows Server 2016 and later. Using teaming, depending the configuration, can negatively affect transfer speed. Quick detection and diagnostic commands Check file systems: Get-Volume | Select DriveLetter, FileSystem Check classic dedup feature: Get-WindowsFeature -Name FS-Data-Deduplication Get-DedupVolume Get-DedupStatus Check ReFS dedup: Get-Command -Module Microsoft.ReFsDedup.Commands Get-ReFSDedupStatus -Volume YourDriveLetter: Diagnostic script to detect both: <# .SYNOPSIS Detects classic NTFS Data Deduplication and ReFS Deduplication across local volumes. .DESCRIPTION - Reports NTFS volumes with classic Data Dedup enabled. - Lists ReFS volumes present on the host. - If the ReFS dedup cmdlet exists AND OS build >= 26100, checks ReFS dedup status per ReFS volume. - Color coding: * Classic dedup enabled → Yellow * Classic dedup not enabled → Cyan * ReFS dedup enabled → Green * ReFS dedup not enabled → Cyan .NOTES Version: 1.7 Author: Karl Wester-Ebbinghaus + Copilot Requirements: Elevated PowerShell session, PowerShell 5.1 or newer Supported OS: Windows Server 2025, Azure Stack HCI 24H2 or newer Unsupported OS: Windows 10, Windows 11 (script terminates) #> #region Initialization Write-Verbose "Initializing variables and environment..." $Volumes = $null $Volume = $null $DedupVolumesList = $null $DedupReFSVolumesList = $null $DedupReFSVolumesListLetters = $null $DedupReFSStatus = $null $refsCmd = $null $OSBuild = $null $runReFSDedupChecks = $null #endregion Initialization #region Volume Discovery Clear-Host Write-Verbose "Querying NTFS and ReFS volumes..." $Volumes = Get-Volume | Where-Object FileSystem -in 'NTFS','ReFS' #endregion Volume Discovery #region ReFS Dedup Cmdlet, OS Build and OS SKU Detection Write-Verbose "Checking for ReFS deduplication cmdlet..." $refsCmd = Get-Command -Name Get-ReFSDedupStatus -ErrorAction SilentlyContinue Write-Verbose "Reading OS build number..." try { $OSBuild = [int](Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' -Name CurrentBuildNumber).CurrentBuildNumber } catch { Write-Verbose "Registry read for OS build failed. Falling back to Environment OSVersion." $OSBuild = [int][Environment]::OSVersion.Version.Build } # end try/catch for OS build detection Write-Verbose "Checking OS InstallationType and EditionID..." $CurrentVersionKey = Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' $InstallationType = $CurrentVersionKey.InstallationType # "Client" or "Server" $EditionID = $CurrentVersionKey.EditionID # e.g. "AzureStackHCI", "ServerStandard", etc. Write-Verbose "Detected InstallationType: $InstallationType" Write-Verbose "Detected EditionID: $EditionID" Write-Verbose "Detected OSBuild: $OSBuild" # Block Windows 10/11 (Client OS) if ($InstallationType -eq 'Client') { Write-Error "Unsupported OS detected: Windows Client (Windows 10/11). Only Windows Server or Azure Stack HCI are supported. Script will terminate." exit } # Allow Azure Stack HCI explicitly if ($EditionID -eq 'AzureStackHCI') { Write-Verbose "Azure Stack HCI detected. Supported platform." } else { # Must be Windows Server if ($InstallationType -ne 'Server') { Write-Error "Unsupported OS detected. Only Windows Server or Azure Stack HCI are supported. Script will terminate." exit } Write-Verbose "Windows Server detected (EditionID: $EditionID). Supported platform." } Write-Verbose "Evaluating ReFS dedup eligibility based on cmdlet presence and build >= 26100..." $runReFSDedupChecks = $false if ($refsCmd -and ($OSBuild -ge 26100)) { $runReFSDedupChecks = $true Write-Verbose "ReFS dedup checks ENABLED (cmdlet present and OS build >= 26100)." } else { Write-Verbose "ReFS dedup checks DISABLED (cmdlet missing or OS build < 26100)." } #endregion ReFS Dedup Cmdlet, OS Build and OS SKU Detection #region Main Loop foreach ($Volume in $Volumes) { # begin foreach volume loop Write-Host "Volume $($Volume.DriveLetter): ($($Volume.FileSystem))" Write-Verbose "Processing volume $($Volume.DriveLetter)..." #region Classic Dedup + ReFS Volume Listing if ($Volume.FileSystem -eq 'NTFS' -or $Volume.FileSystem -eq 'ReFS') { Write-Verbose "Checking classic deduplication status for volume $($Volume.DriveLetter)..." $DedupVolumesList = Get-DedupVolume -Volume $Volume.DriveLetter -ErrorAction SilentlyContinue if ($DedupVolumesList) { Write-Host " → Classic Data Dedup ENABLED on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Yellow } else { Write-Host " → Classic Data Dedup NOT enabled on $($Volume.DriveLetter),$($Volume.FileSystem)" -ForegroundColor Cyan } # end if classic dedup enabled Write-Verbose "Listing ReFS volumes on host..." $DedupReFSVolumesList = Get-Volume | Where-Object FileSystem -eq 'ReFS' if ($DedupReFSVolumesList) { $DedupReFSVolumesListLetters = ($DedupReFSVolumesList | ForEach-Object { $_.DriveLetter }) -join ',' Write-Host " → ReFS volumes present on host: $DedupReFSVolumesListLetters" } else { Write-Host " → No ReFS volumes detected on host" } # end if ReFS volumes present } # end NTFS/ReFS block #endregion Classic Dedup + ReFS Volume Listing #region ReFS Dedup Status if ($Volume.FileSystem -eq 'ReFS') { if ($runReFSDedupChecks) { Write-Verbose "Checking ReFS deduplication status for volume $($Volume.DriveLetter)..." $DedupReFSStatus = Get-ReFSDedupStatus -Volume $Volume.DriveLetter -ErrorAction SilentlyContinue if ($DedupReFSStatus) { Write-Host " → ReFS Dedup ENABLED on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Green } else { Write-Host " → ReFS Dedup NOT enabled on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Cyan } # end if ReFS dedup enabled } else { if (-not $refsCmd) { Write-Error " → Skipping ReFS dedup check: Get-ReFSDedupStatus cmdlet not present" -ForegroundColor Cyan } else { Write-Error " → Skipping ReFS dedup check: OS build $OSBuild < required 26100" -ForegroundColor Cyan } # end reason for skipping ReFS dedup check } # end if runReFSDedupChecks } # end if ReFS filesystem block #endregion ReFS Dedup Status Write-Host "" } # end foreach volume loop #endregion Main Loop #region End Write-Verbose "Script completed." #endregion End Recommendations and next steps Inventory: Identify volumes using NTFS dedup and ReFS dedup, and map workloads that create many small or rapidly changing files. Plan: Schedule rehydration and migration windows; test ReFS dedup on representative datasets. Upgrade: Prioritize upgrading servers still on 2016/2019 (End of Mainstream Support) to reduce support risk and gain the latest ReFS dedup improvements. Kindly consider reading my Windows Server Installation Guidance and Windows Server Upgrade Guidance Exclude: Keep user profiles, AppData, and other high‑churn small‑file paths off ReFS dedup or on NTFS. Consider ReFS Dedup with Compression: Enable compression optionally. Mind ReFS dedup compression is not the same as compress files integration in File Explorer or File Explorer properties (Windows 9x). It's transparent to the application Make smart decisions: Avoid using dedup when the dataset is changing fast or your dedup + compression rate is below 20%. Usually you can expect 40% or more savings, and up to 80% in specific use cases like VDI VHDX with ReFS Dedup + Compression. Plan your dedup jobs: Ensure of making use of the planning features for dedup jobs through PowerShell or Windows Admin Center (WAC) when using ReFS dedup on more than one volume per Server. Otherwise they might all run at the same time and impact your storage performance (esp. spinning rust) and consumption of RAM and CPU. Share and Educate: Inform your infrastructure team about the changes so they avoid using the traditional dedup on ReFS. Related blogposts: https://splitbrain.com/windows-data-deduplication-vs-refs-deduplication/ , Thanks Darryl van der Peijl and team. https://www.veeam.com/kb2023 Veeam best practices about Windows Deduplication and ReFS Deduplication.Get-ClusterExcludedAdapter cmdlet
Following link https://learn.microsoft.com/en-us/powershell/module/failoverclusters/get-clusterexcludedadapter?view=windowsserver2025-ps when execute Get-ClusterExcludedAdapter cmdlet with error below Get-ClusterExcludedAdapter : The term 'Get-ClusterExcludedAdapter' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + Get-ClusterExcludedAdapter + ~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (Get-ClusterExcludedAdapter:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException same for cmdlet Add-ClusterExcludedAdapter (https://learn.microsoft.com/en-us/powershell/module/failoverclusters/add-clusterexcludedadapter?view=windowsserver2025-ps) Does anyone know why these commands are not available?
S2D FaultDomainAwareness
We're setting up a 2 Node windows 2025 cluster with storage spaces direct After creating the pool we created two virtual disk but see the following output PS C:\WINDOWS\system32> Get-VirtualDisk | Format-List FriendlyName, Size, FaultDomainAwareness FriendlyName : ClusterPerformanceHistory Size : 25769803776 FaultDomainAwareness : StorageScaleUnit FriendlyName : S2DVOL01 Size : 10995116277760 FaultDomainAwareness : FriendlyName : S2DVOL02 Size : 10995116277760 FaultDomainAwareness : The FaultDomainAwareness is empty for the two virtual disk created on the storage pool which is configured like this PS C:\WINDOWS\system32> Get-StoragePool –FriendlyName S2D-CLHV-001-Pool | Format-List FriendlyName, Size, FaultDomainAwarenessDefault FriendlyName : S2D-CLHV-001-Pool Size : 57592038555648 FaultDomainAwarenessDefault : StorageScaleUnit is there something wrong ?
