Poor VM Storage Performance with Storage Spaces Direct

Hey Everyone,

I am building a 4-node cluster with Windows Server 2022, Hyper-V, and Storage Spaces Direct. Everything is going well except for storage performance for guest VMs, it is performing much slower than expected. In general, the Hyper-V hosts see ~10x the performance of the Hyper-V guest.

This environment is used for VDI and may host other production VMs depending on how our VMWare quote comes back later this spring.

Hyper-V Hosts:

• Model: Dell 740XD2
• RAM: 768 GB
• CPU: 64 cores (2 sockets x 32 cores)
• Storage: 2x 256GB SAS SSD (OS), 4x 1.6TB NVME SSD, 16x 3.2TB NVME SSD
• Network: 2x Broadcom 25GbE, Cisco ACI

Performance Numbers:

Computer   ReadMiBSec   WriteMiBSec   ReadIOPS   WriteIOPS   Hardware Storage
HVGuest21          60            18      7,654       2,297   HV VM    S2D-volume21

HVHost21          636           190     81,353      24,338   Dell     S2D-volume21
HVHost22          616           184     78,899      23,603   Dell     S2D-volume22
HVHost23          509           153     65,215      19,524   Dell     S2D-volume23
HVHost24          606           181     77,575      23,210   Dell     S2D-volume24

• Performance data was generated using diskSpd.exe
• Command: diskspd.exe -b8k -d30 -o4 -t8 -h -r -w23 -L -Z1G -c20G C:\ClusterStorage\[hvhostxx]\DiskSpd.dat
• Host storage is aligned
  • There is one virtual disk per HV host
  • Each virtual disk is assigned to the corresponding host
  • Each host uses the corresponding disk for testing
• Guest VM storage is also aligned
• Guest VM: W2022 Core, 8 CPU core, 4GB RAM
• If the host and storage are un-aligned, performance metrics drop to ~50% for the HV hosts and the HV guest
• There is minimal load on the HV hosts, with 2 idle test VMs per host.

In our VMWare environment, our guest VM throughput and IOPS numbers are ~3x what we see with Hyper-V / S2D using VSAN or NetApp/NFS, identical servers and networking, and the VMWare servers are hosting ~60 VMs each.  This is not apples-to-apples, just for reference.

Questions

• Is it typical to see this much difference in storage performance between the host and the guest?
• Can I do something to improve the guest storage performance?

I hope I'm just missing something.  Any suggestions would be appreciated.

Thanks,

Rob