User Profile
Joachim_Otahal
Iron Contributor
Joined 5 years ago
User Widgets
Recent Discussions
Re: Nested-V Dedup corruption 26100.1742 and Insider 26296.5001 (and Server 2022) is still there.
MSSWRahman I redid it fresh. Host is Windows 11 24H2, Ryzen 5950x ECC RAM. L1 VM fresh Server 26100.2033, and the two L2 VMs fresh too, though I accidently used build 26296 there :D. The L2 guest VMs are identical, the second is a clone of the first. One big difference: Instead of running Windows Update a powershell script "Server2025-and-Server2022-dedup-corruption-repo-test.ps1", used in the L2 guests, is enough to trigger it. Script is at https://github.com/Joachim-Otahal/PowerShell . Has a little menu, create a 4 GB file, modify it, check whether the CRC is the expected value. In some cases running Step 1 in one of the L2 VMs alone is enough o crash both, on other cases the "step 1, step 2, step 3" had to be repeated a few times in both L2 VMs. There is no memory.DMP, it did not seem to survive the chkdsk.exe run. Even cmd.exe, dir c:\*.dmp /b /s /a shows no .DMP file anywhere. The chkdsk output and events of L2 guests, as far as I could capture them: https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-Events-CHKDSK-etc.7z The whole package, L1 guest and the L2 guests BEFORE activating deduplication. I had it on the second D : 😧 SSD, not the boot SSD. (Can anybody tell me how to turn of that over-eager auto smiley stuff? Feels like over-eager MS office autocorrect, correcting even when turned off...) https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-BEFORE-actviating-dedup.7z The whole package, L1 guest and the L2 guests AFTER they crashed, which should include "everything". https://joumxyzptlk.de/tmp/microsoft/Server2025-dedup-corruption-AFTER-the-crash.7z If you import those in Server 2025 or Windows 11 24H2 on an Intel machine: You may have to deactivate and reactivate Hyper-V on the L1 guest, else it may not work. There seems to be some differences on that regard, which are only taken into account when activating Hyper-V. The culprit, from my point of view, is still the L1 guest, either Server 2022 or Server 2025, not the host OS below or the second level guests. Using Server 2019 as L1 guest with the nested VMs inside works fine, whether the real hardware host is Server 2019, Windows 10, Server 2022, Windows 11 or Server 2025 does not make a difference. If I could get access to .ISOs from vNext between Server 2019 and Server 2022 I could narrow it down at bit when that bug was introduced. For the "fun": A few screenshots. This last one is with the chkdsk run in both VMs when booted from Server 2022 ISO into recovery "repair computer" "cmd.exe" mode.If only MS would take more care of details...
Currently setting up the "deduplication-corruption repo with newest pre-release Server 2025 as L1 and L2 VMs". And then I see this: It is still the problem a lot of "Client only, never needed or wanted on a server by default" stuff creeps into the Server UI. Another example of today is this bad default setting: Microsoft could do so much better if it would take more care of details, reduce the Marketing/Public-Relations (previously known as Propaganda Departement) budget and invest more in actual quality. Please Dave Cutler, you have to rescue Windows - AGAIN, like you did when XP was released (i.e. when it was SP0, I remember how bad it was at first)...Re: Nested-V Dedup corruption 26100.1742 and Insider 26296.5001 (and Server 2022) is still there.
MSSWRahman Thank you for your reply! Which profile: Does not matter, happens with all three (as described in previous. postings..). "Deduplicacte older than 0 days" is only needed if the L2 VMs are fresh set up, before starting any Windows Update. If you import existing VMs without starting them, which have not been touched for > 3 days, you don't need "0 days". How to import the L2 VMs: Attach the VHDX, open Hyper-V GUI, import "register directly". Exactly as in the video. As for sharing the crashed: I will follow your additional preparation instructions, and then provide you with the complete VM package "right before dedup", "right before the final repo step", and "after L2 VMs died", which includes the L1 OS and the (crashed) L2 VMs. Host will be Win11 24h2, Ryzen 5950x, ECC RAM, which is important since the L2 VMs does not work on Intel hosts in my tests unless you deactivate and reactivate Hyper-V in the L1 guest. Do you have a special request regarding what OS the 1st or 2nd level guests have to be? Else I'll use "Server 2025 with updates" for L1 guest, and "Server 2012 R2 German and English" for L2 since the latter are already there. regards, Joachim OtahalNested-V Dedup corruption 26100.1742 and Insider 26296.5001 (and Server 2022) is still there.
Nested-V, host can be Server 2019, Server 2022, Windows 10, Windows 11. Can be Intel or AMD. SSD recommended, 32 GB minimum. 1st level guest is Server 2022, 2025 or the newest vNext, dedup on drive D : active where the nested VMs reside. And they experience data corruption, and in some rare cases even on the D : drive of the 1st level guest. Does not happen if 1st level guest is Server 2019, rock solid then. (I have to use D : with a space, because that over-eagerness of office products to force auto-guess and auto-correct things crept into this forum too, converting it to a smiley!) The video, in that case Server 2022 on Win11 as host: https://www.joumxyzptlk.de/tmp/microsoft/S2022-Nested_Deduplication_VDI-Hyperv_profile_kills_filesystem_v2.mp4 See here my previous posts: https://techcommunity.microsoft.com/t5/windows-server-insiders/server-vnext-26040-and-server-2022-deduplication-data-corruption/m-p/4047321 Here: https://techcommunity.microsoft.com/t5/windows-server-insiders/server-2025-hyper-v-deduplication-corruption-is-still-there/m-p/4136286 Here: https://techcommunity.microsoft.com/t5/windows-server-insiders/26063-deduplication-data-corruption-is-still-there/m-p/4067827 Still perfectly reproducible, in this case the host was just upgraded to Win11 24h2. What is new and different? I know you (i.e. Microsoft) not only does have a bug entry, it got reproduced, and in one case I even know the CPU type to be an E5-1650 where this bug got reproduced (though with ~50% chance there since that CPU is old and slow). I even know it got sorted into "existing features fixed" category this bug belongs since deduplication is not a new feature. Here the two screens from todays re-test: Happy fixing! I am lucky I never used deduplication on a CSV in a Hyper-V Cluster - I was just about to do in 2023, but then this happened. kind regards, Joachim OtahalRe: Active Directory Advanced Threat Hunting - Tracing the cause of account lockouts and password errors
In my experience with large security logs adding "ProviderName" speeds up the query significantly: -FilterHashtable @{ProviderName="Microsoft-Windows-Security-Auditing";LogName="Security";ID=4625} If only failed logins are needed I add "Keywords=4503599627370496" aka "Audit Fail": -FilterHashtable @{ProviderName="Microsoft-Windows-Security-Auditing";LogName="Security";ID=4625;Keywords=4503599627370496} Only -FilterXPath syntax is still faster, and the "[Provider[@Name='Microsoft-Windows-Security-Auditing']]" helps speeding that up too. But I had not yet the need to use -FilterXPath for those 4740/4625 security events, only for NTLM logging where the amount of log entries is huge.Other things to improve (to add to the survey)
If you open an mmc console, like lusrmgr.msc gpedit.msc gpmc.msc dsa.msc etc, every administrator, since the year 2000, moves the divider between the tree pane and the list pane to the right. If you look at every linkedin video you can find, every administrator does that move. Every time. Between three and six seconds wasted. For the last ~24 years. And the divider is too thin to grab on slow remote sessions, costing more time. Updating that default in mmc.exe, to make the tree pane at least double the width and that divider two pixels bigger, would be great. My suggestion would be 250% the current width. For Edge on Windows Server, it should skip the enforced "do you want to use your data?" at the start. We administrators log on to many MANY servers, often after weeks or month of not logging on, and about every time that wizard comes up and costs us about 20+ seconds. The default start page should be an empty page for data protection reasons and not Bing, let alone that many servers are not allowed to go on the internet anyway. (The same applies to clients as well, every time wasting time with that Edge questions, and no way around, even if we only log in once and never again for the whole life time of that client). All cloud features (OneDrive, including the notification to backup to OneDrive, Azure/Entra tool etc) should be not installed, but available as feature/role/capability without needing internet to install those.Re: 26063 deduplication data corruption is still there.
The full dump and the logs are requested for a different machine, an i7-4960x running Server 2022, which did no crash again yet to have such a dump - I activated the "full dump please" option. That single crash may be completely independent from that deduplication issue, which I reproduce in machines I have more confidence in (both have ECC RAM, for example). The i7-4960x machine may show its age since it is eleven years old now, so I have to rule it out as reliable test machine for the deduplication issue.Server 2025 Hyper-V deduplication corruption is still there
I can still reproduce this with 100% confidence on two VERY different machines and hosts with my "easy to reproduce" package: - AMD Ryzen 5950 with Windows 11 23H2 (self built) - Dual Xeon 6226R with Server 2019 (HP Proliant DL 380). As I posted quite often now: If you use deduplication for Hyper-V, i.e. where the virtual machines reside, you will get data corruptions, your virtual machines will crash, and you may even get a filesystem corruption on the drive hosting the machines. For more details see here: 26063 deduplication data corruption is still there. - Microsoft Community Hub As for now: If you used deduplication in Hyper-V scenario, like you used it with Server 2012 up to server 2019, don't bother with Server 2022 and Server 2025 until that bug is fixed. My offer still stands for a live demonstration how I reproduce and talk about the differences how Microsoft tried to reproduce this and how I reproduce this. This is the result for "right about now" albeit without "funk soul brother"...Re: 26063 deduplication data corruption is still there.
I can still perfectly reproduce this with build 26212, with 100% reliability, I suppose I have to recommend not using deduplication in Hyper-V on the local volume hosting the virtual machines with both Server 2022 and Server 2025. https://techcommunity.microsoft.com/t5/windows-server-insiders/server-2025-hyper-v-deduplication-corruption-is-still-there/m-p/4136286
