BLOG: Determine and modernize Filesystem Deduplication
Version history - 1.4 revised script so ReFS volumes with classic dedup will be identified, added more eligibly checks and error handling - 1.3 added point #4 in migration guidance - 1.2 revised script - 1.1 formatting This blog explains the two Windows deduplication modes classic Windows Data Deduplication (ReFS or NTFS) and ReFS Deduplication (ReFS). It covers how they differ, why you should consider upgrading to Windows Server 2025 to leverage the new ReFS dedup engine, and clear warnings about scenarios where ReFS is not recommended. Practical migration guidance and detection commands are included. Differences between classic dedup and ReFS dedup File system: Classic dedup runs on NTFS or ReFS; ReFS dedup runs on ReFS and Windows Server 2025 or later, only. Implementation: They are separate engines with different metadata formats and management cmdlets. Management: Classic dedup uses the Dedup PowerShell module (Get‑DedupVolume, Start‑DedupJob, Disable‑DedupVolume). ReFS dedup uses its own ReFS dedup cmdlets (Get‑ReFSDedupStatus, Enable‑ReFSDedup). Conversion: There is no in‑place conversion between the two; metadata and chunk formats are incompatible. Improvements: the new in-line ReFS Deduplication leverages the advantages of ReFS files system. This makes deduplication more efficient and less CPU intensive. The new ReFS Deduplication can also compress data in-line using L1Z algorithm. This makes it up to par with enterprise solutions, often found in SAN storage or Linux appliances. Compression needs to be set per volume, and optional. Why upgrade to Windows Server 2025 Improved version of ReFS Filesystem Improved ReFS in-line deduplication + optional L1Z compression: Server 2025 includes enhancements to ReFS dedup performance, scalability, and integration with modern storage features. Support and fixes: Windows Server 2016 and 2019 are past mainstream support, increasing the likelihood of costly support cases and delayed fixes; upgrading reduces operational risk and ensures access to ongoing improvements. Future compatibility: Newer OS releases receive optimizations and bug fixes for ReFS and dedup scenarios that older releases will not. SMB compression: for reasonably faster data transfer at minimal CPU when transferring data through the networks. Feature and security related improvements refer to availabile Microsoft Windows Server 2025 Summit content on techcommunity.microsoft.com Scenarios where ReFS is not recommended ReFS on SAN in clustered CSV environments: Avoid placing ReFS dedup on top of SAN‑backed Cluster Shared Volumes (CSVFS) in production clusters; clustered SAN/CSV scenarios causing severe performance issues in practice. Please refer to the ReFS documentation. (personal opinion and experience, not endorsed by Microsoft): Many small, fast‑changing files: Workloads with frequent small writes—user profiles, redirected AppData, or applications that churn small config files (for example, Lotus Notes config files)—can cause locks, performance degradation, or unexpected behavior on ReFS. Exclude these paths from dedup or keep them on NTFS. Restrictions, like lockups or high RAM consumption, deadlocks / BSOD might have been addressed in Windows Server 2025. Improving reliability and performance is a top goal for ReFS, to improve the adoption and feature parity with NTFS. For information about feature parity please refer to the ReFS documentation. Migration guidance The following instructions describe a high level and supported migration path from Windows deduplication using the NTFS file system to native ReFS Deduplication. Note: Step #3, data migration is not required when already using ReFS with Data Deduplication. In this case it's enough to execute step #1 and #2. Note: Validate on non‑production data first. Plan for rehydration time and network/storage throughput. Ensure backups are current before starting. Make sure to have a full backup before upgrading Server OS or making changes. 1. Disable classic dedup on the NTFS source: Disable-DedupVolume -Volume YourDriveLetter: 2. Rehydrate (un‑deduplicate) the data: Start-DedupJob -Volume YourDriveLetter: -Type Unoptimization 3. Copy or move data to a ReFS volume (new target): For straightforward NTFS→ReFS copies, robocopy is recommended. Another alternative is the Windows Admin Center File Server Migration Feature For complex scenarios, open files long path names very large datasets (< 5 TB) or many small files restructuring, GUI (including Windows Server Core) automation, improved logging cloud/hybrid migrations I recommend the usage of GScopy Enterprise by GuruSquad for higher speed (up to 40%) and reliability. 4. Optionally remove the Windows Server feature When there is no old deduplication in use consider to remove the feature. Your advantages of doing so: removes an unneccessary service. removes the file system filter driver for dedup, which causes performance impacts, even when not in use. removes the PowerShell commandlets for the old dedup, so they cannot mistakenly used by existing scripts, unaware admins etc. When migrating files over network: SMB compression: consider both source and target run Windows Server 2025 and leverage SMB compression. SMB Compression is available in Microsoft xcopy, Microsoft robocopy and Gurusquad GScopy Enterprise. Balancing and Teaming with SMB: SMB does not require LFBO or SET Teaming. It automagically detects network links and actively balances on its own on Windows Server 2016 and later. Using teaming, depending the configuration, can negatively affect transfer speed. Quick detection and diagnostic commands Check file systems: Get-Volume | Select DriveLetter, FileSystem Check classic dedup feature: Get-WindowsFeature -Name FS-Data-Deduplication Get-DedupVolume Get-DedupStatus Check ReFS dedup: Get-Command -Module Microsoft.ReFsDedup.Commands Get-ReFSDedupStatus -Volume YourDriveLetter: Diagnostic script to detect both: <# .SYNOPSIS Detects classic NTFS Data Deduplication and ReFS Deduplication across local volumes. .DESCRIPTION - Reports NTFS volumes with classic Data Dedup enabled. - Lists ReFS volumes present on the host. - If the ReFS dedup cmdlet exists AND OS build >= 26100, checks ReFS dedup status per ReFS volume. - Color coding: * Classic dedup enabled → Yellow * Classic dedup not enabled → Cyan * ReFS dedup enabled → Green * ReFS dedup not enabled → Cyan .NOTES Version: 1.7 Author: Karl Wester-Ebbinghaus + Copilot Requirements: Elevated PowerShell session, PowerShell 5.1 or newer Supported OS: Windows Server 2025, Azure Stack HCI 24H2 or newer Unsupported OS: Windows 10, Windows 11 (script terminates) #> #region Initialization Write-Verbose "Initializing variables and environment..." $Volumes = $null $Volume = $null $DedupVolumesList = $null $DedupReFSVolumesList = $null $DedupReFSVolumesListLetters = $null $DedupReFSStatus = $null $refsCmd = $null $OSBuild = $null $runReFSDedupChecks = $null #endregion Initialization #region Volume Discovery Clear-Host Write-Verbose "Querying NTFS and ReFS volumes..." $Volumes = Get-Volume | Where-Object FileSystem -in 'NTFS','ReFS' #endregion Volume Discovery #region ReFS Dedup Cmdlet, OS Build and OS SKU Detection Write-Verbose "Checking for ReFS deduplication cmdlet..." $refsCmd = Get-Command -Name Get-ReFSDedupStatus -ErrorAction SilentlyContinue Write-Verbose "Reading OS build number..." try { $OSBuild = [int](Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' -Name CurrentBuildNumber).CurrentBuildNumber } catch { Write-Verbose "Registry read for OS build failed. Falling back to Environment OSVersion." $OSBuild = [int][Environment]::OSVersion.Version.Build } # end try/catch for OS build detection Write-Verbose "Checking OS InstallationType and EditionID..." $CurrentVersionKey = Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' $InstallationType = $CurrentVersionKey.InstallationType # "Client" or "Server" $EditionID = $CurrentVersionKey.EditionID # e.g. "AzureStackHCI", "ServerStandard", etc. Write-Verbose "Detected InstallationType: $InstallationType" Write-Verbose "Detected EditionID: $EditionID" Write-Verbose "Detected OSBuild: $OSBuild" # Block Windows 10/11 (Client OS) if ($InstallationType -eq 'Client') { Write-Error "Unsupported OS detected: Windows Client (Windows 10/11). Only Windows Server or Azure Stack HCI are supported. Script will terminate." exit } # Allow Azure Stack HCI explicitly if ($EditionID -eq 'AzureStackHCI') { Write-Verbose "Azure Stack HCI detected. Supported platform." } else { # Must be Windows Server if ($InstallationType -ne 'Server') { Write-Error "Unsupported OS detected. Only Windows Server or Azure Stack HCI are supported. Script will terminate." exit } Write-Verbose "Windows Server detected (EditionID: $EditionID). Supported platform." } Write-Verbose "Evaluating ReFS dedup eligibility based on cmdlet presence and build >= 26100..." $runReFSDedupChecks = $false if ($refsCmd -and ($OSBuild -ge 26100)) { $runReFSDedupChecks = $true Write-Verbose "ReFS dedup checks ENABLED (cmdlet present and OS build >= 26100)." } else { Write-Verbose "ReFS dedup checks DISABLED (cmdlet missing or OS build < 26100)." } #endregion ReFS Dedup Cmdlet, OS Build and OS SKU Detection #region Main Loop foreach ($Volume in $Volumes) { # begin foreach volume loop Write-Host "Volume $($Volume.DriveLetter): ($($Volume.FileSystem))" Write-Verbose "Processing volume $($Volume.DriveLetter)..." #region Classic Dedup + ReFS Volume Listing if ($Volume.FileSystem -eq 'NTFS' -or $Volume.FileSystem -eq 'ReFS') { Write-Verbose "Checking classic deduplication status for volume $($Volume.DriveLetter)..." $DedupVolumesList = Get-DedupVolume -Volume $Volume.DriveLetter -ErrorAction SilentlyContinue if ($DedupVolumesList) { Write-Host " → Classic Data Dedup ENABLED on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Yellow } else { Write-Host " → Classic Data Dedup NOT enabled on $($Volume.DriveLetter),$($Volume.FileSystem)" -ForegroundColor Cyan } # end if classic dedup enabled Write-Verbose "Listing ReFS volumes on host..." $DedupReFSVolumesList = Get-Volume | Where-Object FileSystem -eq 'ReFS' if ($DedupReFSVolumesList) { $DedupReFSVolumesListLetters = ($DedupReFSVolumesList | ForEach-Object { $_.DriveLetter }) -join ',' Write-Host " → ReFS volumes present on host: $DedupReFSVolumesListLetters" } else { Write-Host " → No ReFS volumes detected on host" } # end if ReFS volumes present } # end NTFS/ReFS block #endregion Classic Dedup + ReFS Volume Listing #region ReFS Dedup Status if ($Volume.FileSystem -eq 'ReFS') { if ($runReFSDedupChecks) { Write-Verbose "Checking ReFS deduplication status for volume $($Volume.DriveLetter)..." $DedupReFSStatus = Get-ReFSDedupStatus -Volume $Volume.DriveLetter -ErrorAction SilentlyContinue if ($DedupReFSStatus) { Write-Host " → ReFS Dedup ENABLED on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Green } else { Write-Host " → ReFS Dedup NOT enabled on $($Volume.DriveLetter), $($Volume.FileSystem)" -ForegroundColor Cyan } # end if ReFS dedup enabled } else { if (-not $refsCmd) { Write-Error " → Skipping ReFS dedup check: Get-ReFSDedupStatus cmdlet not present" -ForegroundColor Cyan } else { Write-Error " → Skipping ReFS dedup check: OS build $OSBuild < required 26100" -ForegroundColor Cyan } # end reason for skipping ReFS dedup check } # end if runReFSDedupChecks } # end if ReFS filesystem block #endregion ReFS Dedup Status Write-Host "" } # end foreach volume loop #endregion Main Loop #region End Write-Verbose "Script completed." #endregion End Recommendations and next steps Inventory: Identify volumes using NTFS dedup and ReFS dedup, and map workloads that create many small or rapidly changing files. Plan: Schedule rehydration and migration windows; test ReFS dedup on representative datasets. Upgrade: Prioritize upgrading servers still on 2016/2019 (End of Mainstream Support) to reduce support risk and gain the latest ReFS dedup improvements. Kindly consider reading my Windows Server Installation Guidance and Windows Server Upgrade Guidance Exclude: Keep user profiles, AppData, and other high‑churn small‑file paths off ReFS dedup or on NTFS. Consider Dedup and compression: Enable compression optionally. Mind ReFS dedup compression is not the same as compress files integration in File Explorer or File Explorer properties (Windows 9x). It's transparent to the application Make smart decisions: Avoid using dedup when the dataset is changing fast or your dedup + compression rate is below 20%. Usually you can expect 40% or more savings, and up to 80% in specific use cases like VDI VHDX Plan your dedup jobs: Ensure making use planning features for dedup jobs through PowerShell or Windows Admin Center (WAC) when using ReFS dedup on more than one volume per Server. Otherwise they might all run at the same time and impact your storage performance (esp. spinning rust) and consumption of RAM and CPU. Share and Educate: Inform your infrastructure team about the changes so they avoid using the traditional dedup on ReFS.
WSUS changing Update Source on its own
We have 2 WSUS Servers and ConfigMgr. A week ago, one of the WSUS servers began changing the Update Source on its own, no changes had been made. It began pointing to the ConfigMgr and when changed back to use MS Update, shortly after checking again it reverted back to use ConfigMgr. Checked all Events, checked the SQL SUSDB for the WSUS server however there was no information related to this action. Any ideas where I can look next ? Thank youMicrosoft Clearinghouse server connection issues + phone line is dead end
We see a rising number of customers having issues installing or reinstalling their RDS Licenses via automatic connection. According to volume licensing support reinstalling licensing should not work this way any longer and requires customers to contact the RDS activation hotline which is +49 800 5077777 for Germany. If you dial through the menu and reach the RDS Licensing support, for weeks it is not possible to get through speaking with any agent. Instead the phone computer asks for your phone number and to enter it via your phone (DTMF). Whatever the way you enter a number like +492212343434 or 02212343434 it ends up that the voice computer says the number cannot be recognized. I guess that the Microsoft Clearinghouse server has issues with TLS 1.2 and some ciphers but we cannot pin it down even with the networking guys. Here are some possible messages: Customer A: Cannot install licenses, the server is correctly activated and can also be reactived successfully Customer B: Cannot install licenses, server ist correctly activated but can only be reactivated via web. Interestingly activating or reactivating the RDS CAL Server itself works fine on some customers On Other customers even this is not successful anymore due to connection issues. I began to see this last year when customers began to use Windows Server 2019 RDS CAL Servers, while Windows Server 2016 and 2012 R2 were unaffected. We have tried to setup a fresh Windows Server but no help. So 3 things causing a combined issue and blocker: - the RDS CAL phone support is unavailable in Germany - Automatic activation installing new licenses does not work (but is required for RDS via CSP) - Automatic activation re-installing already activated licenses does no longer work according to VL Support as - Automatic activation for Activation / Reactivation of a Server does not work anymore for some customers due to connection issues
Volume Activation role questions
We have a DC, running Server 2016 to decommission (call it old server). One of the roles it had was Volume Activation (VA). This is Active Directory based and the keys AD holds are both for clients (Win11) and servers (2016/19/22/25). I have removed the VA role from the server and tested with a server which I added to the domain and the OS activated successfully, so it looks like it is working. I noticed the _vlmcs SRV DNS record was not deleted and is still pointing to the old server. Since the old server is no longer having the VA role, is it safe to delete the DNS record for the _vlmcs SRV record? What else do I need to take into account? Thanks in advancePublic network on NIC instead of domain network
On a Windows Server 2025 Standard Edition, I have 2 NICs: a 10Gbps NIC and 1Gbps NIC. Both NICs have a static IP address, but only the faster 10Gbps has a default gateway. The faster NIC correctly identifies as the domain network but the slower NIC says it is on a public network. The DNS settings and suffixes are the same for both cards. I have another Windows 2025 server with the same setup but both NICs identify as the domain network. Any ideas on how to fix this? Thanks.Deploying Multiple NPS Servers
I have been working on ditching our password-based WiFi with WPA2-Enterprise. On DC1 I deployed internal CA, NPS, and group policies that auto-request certs and deploy wireless network settings. Cisco AP is pointed to DC1 as the radius server. NPS has been registered in AD and wireless network policy has been created. Test laptops get their cert and connect just fine. It's working. For redundancy, I installed NPS on DC2. This NPS instance has also been registered in AD, and I imported the NPS config from DC1 to DC2 NPS. Cisco AP has DC1 as first radius server and DC2 as second radius server. If I stop NPS on DC1 to force the Cisco AP to authenticate against DC2, test laptops won't authenticate and connect. What am I missing? They're configured exactly the same (except DC1 hosts the CA...I was under the assumption the CA is AD integrated).Hyper-V live mig failing from 2025 host to 2022 host
I'm having an issue with my one hyper-v cluster. I have 4 Server 2022 hosts, and 1 Server 2025 host. Live migration works INBOUND to the 2025 host, but live migrations fail OUTBOUND from the 2025 host. I have tried everything I can think of to make sure settings are ok. I have tried both setting it to use all networks, and the specific network. I've matched the live migration NIC settings to mirror what the 2022 hosts have. I've made sure the Hyper-V Live Migration settings all match. They use CredSSP and compression. Nothing seems to be working. Is there some really subtle 2025 bug that doesn't allow for live migrations to lower OS version hosts?
Windows Server 2016 and internet issues
Hello everyone, I'm relatively new here and hoping for some help. I'm IT at a school in Manhattan that has an old Dell tower running Windows Server 2016 in the basement that is not in use any more with the school as they have transferred over everything to digital. This was set up before my time working here and the person who set it up unfortunately did not leave any notes. We are ready to disconnect the machine as all of it's functions have been moved elsewhere. But, every time it is turned off or disconnected from our network patch, the internet in the entire building goes dead. I do not have previous experience with these kinds of servers and am trying to figure out what could possibly be causing this. I am concerned because the machine is old and feels like it's being held together by duct tape, and if it goes down, I hoping internet doesn't go with it. Looking for any advice or knowledge about these servers and what I can try to do to disconnect it.
Noob needs help with RDP Services
I am new to Windows server management. I setup a 2019 Server in a VM (Hyper-V). I installed the licenses we got for RDP from MS after installing the Remote Desktop Services. I am getting an error about Remote Desktop Licensing Mode is not configured. Tells me to use Server Manger to specify RD Connection Broker. Either I neglected to install it or configure it, not sure. Articles I find say go to Server Manager -> Remote Desktop Services -> Overview... BUT, that tells me I am logged in with a local account but must use a domain account to manage servers and collections. Again, not using a DC. This server is not part of a domain. We do not run AD internally only AzureAD online. We have 1 program we still run internally and users RDP to it. Should I remove the service and reinstall? What about the licenses I added already? How to I keep them? Any assistance will be greatly appreciated... J
