Forum Discussion

Jeff_Riechers's avatar
Jeff_Riechers
Copper Contributor
Mar 04, 2024

VMXnet3 Server 2025 performance issues

Had an odd one crop up today.  I updated my Server 2025 template to the 26063.1 release and then sysprepped it.  Converted it back to a new VM and network performance was .25 Mbps down.  Both on my local lan and on the internet.

 

I removed the VMXnet3 and put in an E1000 and performance was exactly as expected.

 

I shut the VM down, removed the E1000 put in a new network with vmxnet3 and again got horrible performance.

 

I have not seen this with other deployments of 2025 earlier builds.

18 Replies

  • ElizabethGreene's avatar
    ElizabethGreene
    Copper Contributor

    As a data point, my customer was experiencing this after upgrading Win10 to Win11.  We were able to resolve it by changing "TCP Checksum Offload (IPv4)" to disabled on the vmxnet3 NIC.  We're continuing troubleshooting and I've bookmarked this thread to come back and add more data when we get to the bottom of it.  

    Their VMs have the NIC named "Ethernet0", so this PowerShell command does the needful. 

    Set-NetAdapterAdvancedProperty -Name "Ethernet0" -DisplayName "TCP Checksum Offload (IPv4)" -DisplayValue "Disabled"

     

  • philippeannet's avatar
    philippeannet
    Copper Contributor

    Got a similar issue with a W11 VM on an Intel MBP (MacOS 15.3) with Fusion Pro... Speedtest.net gets me a solid 450-500Mbps upload, but barely half of that in download (???????????), although the host gets 6-700Mbps in download... I played with all possible offloads in the adapter settings (VMXnet3 and e1000), same story, pretty incomprehensible !!

    • jeffriechers's avatar
      jeffriechers
      Copper Contributor

      This ended up being my fix on all machines.  Run this and it improves the NIC performance.  I think it is something to do with the new settings in the network stack itself.  https://www.jeffriechers.com/wiki/24h2-windows-11-and-server-2025-vmxnet3-issues/

    • MysticFoxDE's avatar
      MysticFoxDE
      Brass Contributor

      Hi philippeannet,

      on an Intel MBP (MacOS 15.3) with Fusion Pro

      😯 ... 👍👍👍 ... You are definitely very brave, I like that!

      Now to the important things.

      With an uplink speed of 1 Gbit/s, you should only use VMXnet or E1000 vNIC and never VMXnet3, as this is intended for uplinks of 10 Gbit/s or more.

      In addition, you should deactivate the Defender and also the phishing protection of W11 as a test, as they also like to steal a lot of performance and slow you down considerably. 😔     

      Then you could deactivate the RSC on the W11 in particular, but also the RSS as a test.

      netsh int tcp set global RSS=Disabled
      netsh int tcp set global RSC=Disabled

       

      Otherwise you can also run the following script in the VM.

      https://github.com/MysticFoxDE/WINDOWS-OPTIMIZATIONS/blob/main/W10ANDW11-NETWORK-TCP-DESUBOPTIMIZATION.ps1

      😉

       

      Best Regards from Germany

      Alex

  • MysticFoxDE's avatar
    MysticFoxDE
    Brass Contributor

    Hi Jeff,
    which NIC exactly (manufacturer type) does your ESXi use as an uplink interface for the corresponding vSwitch?

    My crystal ball tells me that maybe VMQ is activated on the VMXnet3 vNIC, but that this is not configured correctly in terms of hardware.
    You can try VMXnet3 without VMQ, which should run much better and would also confirm my suspicions.
    But, without VMQ or SR-IOV, you can't really utilize a 10G or even faster vNIC.

    As mentioned, you should also deactivate both RSC in the VM and its counterpart, i.e. TSO in the ESXi.

    Best Regards from Germany
    Alex

  • Bhanous's avatar
    Bhanous
    Copper Contributor

    E1000 are not supported in some scenarios. Jeff could it be that the RSS / RSC config got compromised or receive buffers?

  • alexeybes's avatar
    alexeybes
    Copper Contributor

    I had same issues. After I disabled all kinds of "offloads" and "interruptions" options VMXnet3 works fine.

    • Jeff_Riechers's avatar
      Jeff_Riechers
      Copper Contributor

      I have moved back to E1000 at this point so I can use NVMe memory tiering.  Some people have started seeing issues with Windows 11 24H2 as well.  Do you have a breakdown of the mods you did so I can test them in my lab?

      • alexeybes's avatar
        alexeybes
        Copper Contributor

        Control Panel - Network and Sharing Center - Change adapter settings
        Right Click on vmxnet3 Ethernet Adapter - Properties - Configure... - Advanced

        Interrupt Coalescing Sheme Mode - Disabled
        Interrupt Moderation - Disabled
        IPv4 Checksum Offload - Disabled
        IPv4 TSO Offload - Disabled
        Large Send Offload V2 (IPv4) - Disabled
        Large Send Offload V2 (IPv6) - Disabled
        Offload IP Option - Disabled
        Offload tagged traffic  - Disabled
        Offload TCP Option - Disabled
        Priority / VLAN tag - Priority & VLAN Disabled
        Receive Side Scaling - Disabled
        Recv Segment Coalescing (IPv4) - Disabled
        Recv Segment Coalescing (IPv6) - Disabled
        Speed & Duplex - 10 Gbps Full Duplex
        TCP Checksum Offload (IPv4) - Disabled
        TCP Checksum Offload (IPv6) - Disabled
        UDP Checksum Offload (IPv4) - Disabled
        UDP Checksum Offload (IPv6) - Disabled

        I'm not sure which one helped.
        Didn't have time to check one-by-one.

  • aru_vmw's avatar
    aru_vmw
    Copper Contributor
    Hi Jeff
    Do you still see the problem reproduced with 26080 build?
    • jeffriechers's avatar
      jeffriechers
      Copper Contributor
      2025 has gotten better. But as soon as I enable RDS on it the performance tanks. Just was testing out the latest pre-releases and still seeing this.
      • Karl-WE's avatar
        Karl-WE
        MVP

        Hi Jeff what about the b26100 and December LCU? 

  • J_Hawkins1460's avatar
    J_Hawkins1460
    Copper Contributor

    Jeff_Riechers Delete your ghost NIC and your vmxnet driver. When you add the new NIC it'll poll for the actual VMware tools driver vs what it thinks works. Sysprep seems to do this in 2019+ sometimes if you don't have the drivers manually added to drivers and just VMware tools. E1000 will always work.

    • Karl-WE's avatar
      Karl-WE
      MVP
      E1000 are not supported in some scenarios. Jeff could it be that the RSS / RSC config got compromised or receive buffers?
      • jeffriechers's avatar
        jeffriechers
        Copper Contributor

        Karl-WE I don't see how. I have built machines from templates, and from ISO.  All seem to fail.  Same hardware 2022, 2019, 10 and 11, Debian and Ubuntu all run fine.  Just 2025 seems to not like vmxnet3.

         

        It is VMWare 7 still, so it could be something with that build of ESX.

  • aru_vmw's avatar
    aru_vmw
    Copper Contributor

    Jeff_Riechers

    Could you provide further information about your setup? For example, which version of ESXi are you using? Also, what is the version of the vmxnet3 driver?how many vCPU you allocated to your VM?

     

Resources