ReFS volume appears RAW (version doesn't match expected value) after Windows Update

[MikeLabatt]

After Windows Update last night, Windows Server 2019 wouldn't mount a storage space volume as ReFS (it appears as RAW). The error in the ReFS event log is "ReFS failed to mount the volume. Version 1.2 doesn't match expected value 3.4" No issues that I can see at the storage space level (it is a mirrored disk). The volume was working fine before Windows Update and the reboot. Another ReFS volume still works fine after the update.

Any clues? I could not find this error mentioned anywhere else. Thanks.

[#2 · Accepted solution]

I solved this by uninstalling KB5009557. The ReFS volume came back working as it should, instead of appearing as RAW.

Update: since even the February 2022 Windows Update bricks ReFS in the same way, and hints from Microsoft are that ReFS 1.x is no longer supported, we copied everything to new disks, upgrading ReFS from 1.2 to 3.4 in the process. Such a (manual) ReFS upgrade should be the solution that everyone needs, allowing to re-enable Windows Update.

[#3]

I will tag along in the hope if @Pernille-Eskebo will acknowledge or come with a mitigation for it.

[#4]

It seems to be a known issue for the 2022-01 patches.

[#5]

We ran into this on a Server 2012 server and had to uninstall both KB5009595 and KB5009624 before our drives came back.

[#6]

@robertkennington 

Same problem here on Windows 2012R2. We have to uninstall both, KB5009595 and KB5009624. After uninstallation our ReFS File Sytem came back!

[#7]

Microsoft just released a series of out-of-band updates (KB5010790 for Windows Server 2016, KB5010793 for Windows 10, KB5010795 for Windows 11, KB5010796 for Windows Server 2022, KB5010794 for older systems, etc.) to address the ReFS issue. For some reason they still claim the bug to be limited to removable media, even if that was not the case for me and others.

Alas, I could not find an out-of-band update for Windows Server 2019. So far, no KB page and no Windows Update showing up. ReFS is still broken by re-applying the current (January 18) KB5009557.

[#8]

Thanks for the follow-up much appreciated!

[#9]

A day after the other out-of-band updates, KB5010791 was made available for Windows Server 2019 as well. However, it did not resolve the ReFS issue for me. I uninstalled KB5010791 and then KB5009557, and the broken ReFS volume mounted fine again (it is on a storage space, not removable media).

https://support.microsoft.com/en-us/topic/january-18-2022-kb5010791-os-build-17763-2458-out-of-band-...

[#10]

Can definitely confirm that Server 2019 Datacenter is still experiencing the ReFS issue after updating, and the only other patch that appears in Updates is KB5010791.  It doesn't solve the issue either. 

Can't believe it's been this long and there still isn't a fix for this on 2019 when it's obviously an issue (whether MS wants to admit it or not). 

The only fix so far for 2019 is to uninstall both KB5010791 and KB5009557. And then not run Windows Updates again until it's finally acknowledged and fixed.

[#11]

just installed KB5010791 to fix ReFS issue. Did not solve the problem on Server 2019. Volume still is not available.

Seems like a joke that "The Resilient File System (ReFS) is Microsoft's newest file system, designed to maximize data availability, scale efficiently to large data sets across diverse workloads, and provide data integrity with resiliency to corruption" is getting useless due to an until now unfixed MS Bug.

[#12]

Possible solution to fix this issue:

I wondered about the fact that only "removable" devices were affected.
So i changed the "devices.hotplug" value in VM config to "false".
This is dissabling the hotplug for the VM.
After starting the server 2019 my ReFS HD was back.
Simple to do and i do not see big side effects.
Hope it helps somebody.

[#13]

@MikeLabatt @Pernille-Eskebo 

The Issues is caused by the kb5005959 on Windows 2012(r2) Server with ReFS-Formatted disks.

Uninstalling the patch and rebooting fixes the problem.

Best,

Noël 

[#14]

Updated three Server 2019 Datacenter with direct attached ReFS Volumes (VMware) two days ago with the "fixed" updates. Two servers went fine. Third not. ReFS volume RAW. Uninstalled KB5010791 on this server. Took round about 20 minutes. After uninstall ReFS volume was OK again. Don't know what to say....

Edit:

On all three servers hotplug isn't disabled!

Will try to disable hotplug on the affected server next weekend and try to install update KB5010791 again.

But why the issue does not occure on the two other servers with hotplug enabled too?

[#15]

@GKrem 

I can confirm - this worked for me too.  Followed this, adding devices.hotplug and a value of false. to our VM configurations in VMWare: 

https://kb.vmware.com/s/article/1012225

17 Windows 2019 servers with ReFS volumes.  All now updated through latest windows updates and stable. 

[#16]

The main issue is there are various versions of REFS
ReFS v1 as used on Server 2012R2 and ReFS v3 used on later OS's
The Updated fix, for Refs on removable drives, only addresses the ReFS v3 case and does not (and never will) address the refsv1 case.
Note that some disks, even on later OS's might be using ReFS v1 if they were originally set up on earlier systems
And so the fix will not help on those disks on those systems
Since its mainly VMWare VM's that are affected, as they consider hotplug disks as removable, suggest using the vmware devices.hotplug solution described earlier

You can check the Refs version with
fsutil fsinfo refsinfo x:
(although cant do this if the drive is currently showing as RAW)

[#17]

Thank you for the explanation. Indeed the ReFS volume that turns into RAW is 1.2, and the one that keeps working is 3.4. Same machine, both are storage spaces (no VMWare or removable media involved). So what is the recommended solution for the older version?

[#18]

Even though your post was helpful I have to shake my head at how sad it is how Microsoft treats its customers now. Do you not care about your onprem customers now? Who do you think knows what version of ReFS they were running? Should I worry about what version of NTFS or FAT32 i'm running or maybe Microsoft will say sorry your SOL version 3.2 of NTFS is no longer supported and our update broke it and we aren't going to fix it. So next time I upgrade my OS do I have to reformat every drive I have or worry this will happen again?

This screw-up almost cost me to lose data (almost formatted the drive because I thought it was corrupt) and you should be appologizing rather than pretty. much publically telling everyone sorry your SOL. At least you should have a utility to upgrade your ReFS version (if possible) or maybe at least warn users? This is really inexcusable. The only thing good about it I guess is it's lit a fire under me to get every piece of Microsoft out of our infrastructure that I can.

[#19]

@stephc_msft Thanks for the info. If am understanding what you are saying correctly, then my Exchange 2016 Server running on Windows Server 2012 R2 will never be able to patched beyond the 2022-01 security update for Windows Server because both the security patch and the out of band update do not account for the older version of REFS?

My only options are to try the VMware workaround or migrate to a newer OS? Is there a way to upgrade REFS to a version that is supported by the patch? Thanks again for taking the time to post in the thread.

[#20]

Unfortunately the only way to upgrade the ReFS version (if using refsv1 disks on WS2016 or later) is to create new disks or reformat old disks (and confirm refsv3.x) and copy/restore data.
For those still on WS2012R2 (which doesnt support refsv3), then the best (and only) soln if on VMWare is the vmware disable hotplug soln .

If WS2012R2 and NOT on VMWare then shouldnt be affected by the issues  (except in much rarer

circumstances such as really using removable USB drives etc)

KB5010691: ReFS-formatted removable media may fail to mount or mounts as RAW after installing the Ja...
But only briefly touches on the refs version and WS2012R2 aspect