Nested-V Dedup corruption 26100.1742 and Insider 26296.5001 (and Server 2022) is still there.

Question

Nested-V, host can be Server 2019, Server 2022, Windows 10, Windows 11. Can be Intel or AMD. SSD recommended, 32 GB minimum.

1st level guest is Server 2022, 2025 or the newest vNext, dedup on drive D : active where the nested VMs reside. And they experience data corruption, and in some rare cases even on the D : drive of the 1st level guest. Does not happen if 1st level guest is Server 2019, rock solid then. (I have to use D : with a space, because that over-eagerness of office products to force auto-guess and auto-correct things crept into this forum too, converting it to a smiley!)

The video, in that case Server 2022 on Win11 as host: https://www.joumxyzptlk.de/tmp/microsoft/S2022-Nested_Deduplication_VDI-Hyperv_profile_kills_filesystem_v2.mp4

See here my previous posts:

https://techcommunity.microsoft.com/t5/windows-server-insiders/server-vnext-26040-and-server-2022-deduplication-data-corruption/m-p/4047321

Here:

https://techcommunity.microsoft.com/t5/windows-server-insiders/server-2025-hyper-v-deduplication-corruption-is-still-there/m-p/4136286

Here:

https://techcommunity.microsoft.com/t5/windows-server-insiders/26063-deduplication-data-corruption-is-still-there/m-p/4067827

Still perfectly reproducible, in this case the host was just upgraded to Win11 24h2. What is new and different? I know you (i.e. Microsoft) not only does have a bug entry, it got reproduced, and in one case I even know the CPU type to be an E5-1650 where this bug got reproduced (though with ~50% chance there since that CPU is old and slow). I even know it got sorted into "existing features fixed" category this bug belongs since deduplication is not a new feature.

Here the two screens from todays re-test:

Happy fixing! I am lucky I never used deduplication on a CSV in a Hyper-V Cluster - I was just about to do in 2023, but then this happened.

kind regards,

Joachim Otahal

msswrahman · Answer

Just one more thing. You may be knowing this already, but when you enable full dump on the L2 guests, please check that pagefile is enabled on C: and is big enough to accommodate the RAM size on those VMs.After you get the repro, do share the logs mentioned in points c), d), e) and f) of my initial post.Thanks

msswrahman · Answer

Hello&nbsp;Joachim_Otahal&nbsp;
I have a few clarification questions.
While configuring dedup on 😧 of L1 guest, did you use VDI profile?And did you set "deduplicate file older than" to 0 days?On L1 Hyper-V, when you import the L2 VMs, what import type option did you use?
&nbsp;
We have not been able to repro the issue internally with the VDI profile on either WS2022 or WS2025 as the L1 OS.
&nbsp;
Given that it is reliably reproducible for you, could you please do the following?
a) enable full dump on both the L2 guest VMs by running:reg add "hklm\SYSTEM\CurrentControlSet\Control\CrashControl" /v CrashDumpEnabled /t reg_dword /d 1 /f
b) Do the repro. If you can, save the output of chkdsk C: /f or chkdsk C: /scan run on the L2 guest VMs
c) Share the system.evtx, Application.evtx and Microsoft-Windows-Ntfs%4Operational.evtx from the L2 guest VMs. They are under C:\windows\system32\winevt\Logs folder.
d) Run the command: fsutil repair enumerate C: on both the L2 guest VMs and share the output.
e) Share the system.evtx, Application.evtx and Microsoft-Windows-Ntfs%4Operational.evtx from the L1 guest VM
f) If either of the L2 VMs bluescreen, please share the dump.
Greatly appreciate your input in figuring out this issue.
&nbsp;
Thanks

msswrahman · Answer

Thank you for the clarifications, Joachim.
Please use VDI profile because that is the recommended one in this scenario. For the L1 guest please use server 2025, because that is where dedup is being enabled. For the L2 guest, feel free to use server 2012. I guess L2 could be any OS; I was using server 2022.

Also, my attempts have been on Intel hosts so far. I will try your tip of deactivating and reactivating Hyper-V in the L1 guest (before starting the dedup job i guess).

joachim_otahal · Answer

MSSWRahman&nbsp;&nbsp;Thank you for your reply!&nbsp;Which profile: Does not matter, happens with all three (as described in previous. postings..)."Deduplicacte older than 0 days" is only needed if the L2 VMs are fresh set up, before starting any Windows Update. If you import existing VMs without starting them, which have not been touched for &gt; 3 days, you don't need "0 days".&nbsp;How to import the L2 VMs: Attach the VHDX, open Hyper-V GUI, import "register directly". Exactly as in the video.&nbsp;As for sharing the crashed: I will follow your additional preparation instructions, and then provide you with the complete VM package "right before dedup", "right before the final repo step", and "after L2 VMs died", which includes the L1 OS and the (crashed) L2 VMs. Host will be Win11 24h2, Ryzen 5950x, ECC RAM, which is important since the L2 VMs does not work on Intel hosts in my tests unless you deactivate and reactivate Hyper-V in the L1 guest.&nbsp;Do you have a special request regarding what OS the 1st or 2nd level guests have to be? Else I'll use "Server 2025 with updates" for L1 guest, and "Server 2012 R2 German and English" for L2 since the latter are already there.&nbsp;regards,&nbsp;Joachim Otahal

joachim_otahal · Answer

MSSWRahman&nbsp;I redid it fresh.Host is Windows 11 24H2, Ryzen 5950x ECC RAM. L1 VM fresh Server 26100.2033, and the two L2 VMs fresh too, though I accidently used build 26296 there :D. The L2 guest VMs are identical, the second is a clone of the first.One big difference: Instead of running Windows Update a powershell script "Server2025-and-Server2022-dedup-corruption-repo-test.ps1", used in the L2 guests, is enough to trigger it. Script is at https://github.com/Joachim-Otahal/PowerShell . Has a little menu, create a 4 GB file, modify it, check whether the CRC is the expected value. In some cases running Step 1 in one of the L2 VMs alone is enough o crash both, on other cases the "step 1, step 2, step 3" had to be repeated a few times in both L2 VMs.There is no memory.DMP, it did not seem to survive the chkdsk.exe run. Even cmd.exe, dir c:\*.dmp /b /s /a shows no .DMP file anywhere.&nbsp;The chkdsk output and events of L2 guests, as far as I could capture them:https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-Events-CHKDSK-etc.7z&nbsp;The whole package, L1 guest and the L2 guests BEFORE activating deduplication. I had it on the second D :&nbsp;😧 SSD, not the boot SSD. (Can anybody tell me how to turn of that over-eager auto smiley stuff? Feels like over-eager MS office autocorrect, correcting even when turned off...)https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-BEFORE-actviating-dedup.7zThe whole package, L1 guest and the L2 guests AFTER they crashed, which should include "everything".https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-AFTER-the-crash.7z&nbsp;If you import those in Server 2025 or Windows 11 24H2 on an Intel machine: You may have to deactivate and reactivate Hyper-V on the L1 guest, else it may not work. There seems to be some differences on that regard, which are only taken into account when activating Hyper-V.The culprit, from my point of view, is still the L1 guest, either Server 2022 or Server 2025, not the host OS below or the second level guests. Using Server 2019 as L1 guest with the nested VMs inside works fine, whether the real hardware host is Server 2019, Windows 10, Server 2022, Windows 11 or Server 2025 does not make a difference.&nbsp;If I could get access to .ISOs from vNext between Server 2019 and Server 2022 I could narrow it down at bit when that bug was introduced.&nbsp;For the "fun": A few screenshots.&nbsp;&nbsp;This last one is with the chkdsk run in both VMs when booted from Server 2022 ISO into recovery "repair computer" "cmd.exe" mode.&nbsp;

Forum Discussion

Nested-V Dedup corruption 26100.1742 and Insider 26296.5001 (and Server 2022) is still there.

5 Replies

Resources