Windows Server 2019 Backup Stopped working
Hi I have a Windows Server 2019 Standard that Windows Server 2019 Backup Stopped working. Worked great for over a year. Here is error : The backup operation that started at '2021-06-13T03:30:22.766521300Z' has failed with following error code '0x8078014D' (There was a failure in updating the backup for deleted items.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved. I also got an error with invalid option . I am backing up one Folder to a NAS. ioSafe running Synology OS. As a side note : I have a VM server running on this server that is backing up 2 folders to same NAS. Same issue. What I have tried. 1. Freed up space on NAS. 2. On NAS resetup folders backed up to. Made sure folders for Physical Backup and VM have correct permission and NAS user with permission to do the job. 3. Uninstalled Windows server Backup and reinstalled. 4. Re Setup Backup Jobs. 5. Server updates are done monthly and I think NAS OS is up to date. Any Ideas? Thanks you very much in advance! Dexter Southerland
Windows Backup taking waaaaay to long
While I'm not a heavy user of these MS forums I have had to resort to them from time to time over the last 15-20 years. Yet I still can't figure out the organizational structure and it seems I can never find the right forum for my query. Almost every time my post gets moved to the correct forum or message board, or someone gives me a link directly to it. I expect it to be no different this time, and I'm perfectly fine with that. So here we go. I have Windows Server 2025 installed as a VM using MS's built-in Hyper-V on a Server 2025 computer. the VM is set up as a DC and all that stuff functions exactly as it should. However, doing the backup has suddenly gone from taking anywhere from 2 hours to a max that comes close to but has never exceeded four hours. Obviously, it depends on how much there is to actually back up. I've already gone through the troubleshooting tips to do things like checking the VSS settings and a bit of other stuff I can't exactly recall at the moment. I have an external physical 1TB usb hard drive attached to the physical computer and then it's attached as a drive to the Server 2025 VM and shows up in computer management/disk manager ad Disk 1, as it should. I have the VM set up to use this Disk 1 as the backup disk with the Windows Server Backup program. Some things I note and add here in case it matters. - The size of the VM disk for this Server 2025 VM is 500GB and the partition size of Drive C shows as 498.91GB with the remaining shown as 100MB for the EFI system partion and 1001MB for the recovery partition. - When backup starts, a new disk labeled Disk 2 appears in the disk management window on the VM and I note it's the same size as Drive C on the VM at 498.91GB. I'm wondering if this has anything to do with why my backups suddenly went from taking a max of 4 hours to as long as 20 hours to complete. Where is this virtual disk created? I looked on the VM host machine in the C:\programdata\microsoft\windows\Virtual Hard Disks directory, and it's not there. It's not on the VM machine because the virtual hard disk directory doesn't exist in that same location on the VM. THe host machine itself has a 2TB hard drive in it with 993GB of free space. Any advice or suggestions here? I have no idea why backups went from 2-4 hours to taking 20 hours or more to complete. Thanks for any help, advice or suggestions anyone can offer here. -CarlHyper V Duplicates recovery points
Hi, We have a HyperV host in a Windows Server 2016 STD with 3 VM Servers Domain Controller (Windows Server 2019) APP Server (Windows Server 2019) DB Server (Sequel) (Windows Server 2016) The host have 4 TB GB HHD 128 GB RAM 2 Processors 2.20GHz We used Veeam backup tool for backups. Suddenly with every backup run, HyperV creates a Veeam recovery Checkpoint. The free disk space of the disk is shrinking considerably. We also noticed that when every Veeam recovery point is created it creates a duplicate of the AVHDX, MRT and RCT files in the Virtual Hard Disks folder. I disabled the Veeam backup jobs until it can be fixed, because if we get out of disk space the VM Servers will not startup. NOTE: This are the production Servers and no other environment for test exists. Hope you can help me thorough this issue.Announcing Native NVMe in Windows Server 2025: Ushering in a New Era of Storage Performance
We’re thrilled to announce the arrival of Native NVMe support in Windows Server 2025—a leap forward in storage innovation that will redefine what’s possible for your most demanding workloads. Modern NVMe (Non-Volatile Memory Express) SSDs now operate more efficiently with Windows Server. This improvement comes from a redesigned Windows storage stack that no longer treats all storage devices as SCSI (Small Computer System Interface) devices—a method traditionally used for older, slower drives. By eliminating the need to convert NVMe commands into SCSI commands, Windows Server reduces processing overhead and latency. Additionally, the whole I/O processing workflow is redesigned for extreme performance. This release is the result of close collaboration between our engineering teams and hardware partners, and it serves as a cornerstone in modernizing our storage stack. Native NVMe is now generally available (GA) with an opt-in model (disabled by default as of October’s latest cumulative update for WS2025). Switch onto Native NVMe as soon as possible or you are leaving performance gains on the table! Stay tuned for more updates from our team as we transition to a dramatically faster, more efficient storage future. Why Native NVMe and why now? Modern NVMe devices—like PCIe Gen5 enterprise SSDs capable of 3.3 million IOPS, or HBAs delivering over 10 million IOPS on a single disk—are pushing the boundaries of what storage can do. SCSI-based I/O processing can’t keep up because it uses a single-queue model, originally designed for rotational disks, where protocols like SATA support just one queue with up to 32 commands. In contrast, NVMe was designed from the ground up for flash storage and supports up to 64,000 queues, with each queue capable of handling up to 64,000 commands simultaneously. With Native NVMe in Windows Server 2025, the storage stack is purpose-built for modern hardware—eliminating translation layers and legacy constraints. Here’s what that means for you: Massive IOPS Gains: Direct, multi-queue access to NVMe devices means you can finally reach the true limits of your hardware. Lower Latency: Traditional SCSI-based stacks rely on shared locks and synchronization mechanisms in the kernel I/O path to manage resources. Native NVMe enables streamlined, lock-free I/O paths that slash round-trip times for every operation. CPU Efficiency: A leaner, optimized stack frees up compute for your workloads instead of storage overhead. Future-Ready Features: Native support for advanced NVMe capabilities like multi-queue and direct submission ensures you’re ready for next-gen storage innovation. Performance Data Using DiskSpd.exe, basic performance testing shows that with Native NVMe enabled, WS2025 systems can deliver up to ~80% more IOPS and a ~45% savings in CPU cycles per I/O on 4K random read workloads on NTFS volumes when compared to WS2022. This test ran on a host with Intel Dual Socket CPU (208 logical processors, 128GB RAM) and a Solidigm SB5PH27X038T 3.5TB NVMe device. The test can be recreated by running "diskspd.exe -b4k -r -Su -t8 -L -o32 -W10 -d30 testfile1.dat > output.dat" and modifying the parameters as desired. Results may vary. Top Use Cases: Where You’ll See the Difference Try Native NVMe on servers running your enterprise applications. These gains are not just for synthetic benchmarks—they translate directly to faster database transactions, quicker VM operations, and more responsive file and analytics workloads. SQL Server and OLTP: Shorter transaction times, higher IOPS, and lower tail latency under mixed read/write workloads. Hyper‑V and virtualization: Faster VM boot, checkpoint operations, and live migration with reduced storage contention. High‑performance file servers: Faster large‑file reads/writes and quicker metadata operations (copy, backup, restore). AI/ML and analytics: Low‑latency access to large datasets and faster ETL, shuffle, and cache/scratch I/O. How to Get Started Check your hardware: Ensure you have NVMe-capable devices that are currently using the Windows NVMe driver (StorNVMe.sys). Note that some NVMe device vendors provide their own drivers, so unless using the in-box Windows NVMe driver, you will not notice any differences. Enable Native NVMe: After applying the 2510-B Latest Cumulative Update (or most recent), add the registry key with the following PowerShell command: reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Policies\Microsoft\FeatureManagement\Overrides /v 1176759950 /t REG_DWORD /d 1 /f Alternatively, use this Group Policy MSI to add the policy that controls the feature then run the local Group Policy Editor to enable the policy (found under Local Computer Policy > Computer Configuration > Administrative Templates > KB5066835 251014_21251 Feature Preview > Windows 11, version 24H2, 25H2). Once Native NVMe is enabled, open Device Manager and ensure that all attached NVMe devices are displayed under the “Storage disks” section. Monitor and Validate: Use Performance Monitor and Windows Admin Center to see the gains for yourself. Or try DiskSpd.exe yourself to measure microbenchmarks in your own environment! A quick way to measure IOPS in Performance Monitor is to set up a histogram chart and add a counter for Physical Disk>Disk Transfers/sec (where the selected instance is a drive that corresponds to one of your attached NVMe devices) then run a synthetic workload with DiskSpd. Compare the numbers before and after enabling Native NVMe to see the realized difference in your real environment! Join the Storage Revolution This is more than just a feature—it’s a new foundation for Windows Server storage, built for the future. We can’t wait for you to experience the difference. Share your feedback, ask questions, and join the conversation. Let’s build the future of high-performance Windows Server storage together. Send us your feedback or questions at nativenvme@microsoft.com! — Yash Shekar (and the Windows Server team)
Unable to Build Switchless Storage using Network ATC
3-Node HV/S2D Cluster using Switchless Storage and Network ATC getting Network HUD Error I have a Dell 3-node server cluster being used to host Hyper-V with S2D. Each node is identical and certified to pass S2D requirements. The networking consists of 2 onboard 1 Gbps and 2 quad-port PCI(e) NICs at 10/25 Gbps. The 2 onboard ports are being used for the management intent, 2 ports from each of those NICs are being connected to 2 top-of-rack switches for the compute intent at a total of 12 10 Gbps connections, and the other 2 ports from each of those NICs are being connected directly to the other nodes in a dual mesh method for the storage intent at a total of 6 25 Gbps connections. Very similar to the Microsoft diagram for a three-node storage switchless, dual TOR, dual link deployment network reference pattern for Azure Local, but I am using Windows Server 2025 Datacenter, not Azure Local. The problem I am encountering is when I attempt to create the switchless storage intent, I get an error (shown below) that reads "Failed to fetch physical NIC mapping from the Network HUD service. Please diagnose..." If I leave off the switchless option, it builds without issue. The error only occurs when I attempt to use the switchless option. I tried without configuring the NIC IP addresses first, as well as trying configuring the NIC IP addresses beforehand. I tried without having a cluster built first, as well as building a failover cluster without storage before running the network intent command. The error I posted below shows my last attempt after having the NICs configured with IP addresses and having a failover cluster established before running the command. The error has remained the same throughout the process. Has anyone else run into an issue like this with switchless storage? It seems like everything that I am trying to do is within scope and should be a supported solution. My fallback plan is to use my existing switches, but that drops my connection speeds down from 25 Gbps to 10 Gbps due to hardware limitations.
