Slower SMB read performance for large files in 22H2
Published Oct 03 2022 10:35 AM 25.2K Views
Microsoft

Heya folks, Ned here again. There is a performance reduction in 22H2 when copying larger files from a remote computer down to a Windows 11 computer or when copying from any Windows computer to a remote Windows 11 computer. 

 

Fix update 12/14/2022: An updated version of this preview fix is now available in Windows 11 Insider Preview Build 25267. This version "Fixed an issue where restartable copies of a file (for example using robocopy) with alternate data streams were hanging frequently in the latest Dev Channel builds." The final fix for Windows 11 22H2 production computer will come in a normal monthly Cumulative Update through Windows Update once validated in Insider builds.  

 

Fix update 11/28/2022: The preview fix for this issue is now available in Windows 11 Insider Preview Build 25252. The final fix for Windows 11 22H2 production computer will come in a normal monthly Cumulative Update through Windows Update once validated in Insider builds.  

 

For the latest information about this issue, please see the known issues section of KB5017389

If you are using Microsoft 365 admin center and have Windows Release Health available there, you can view this issue in the Windows 11, version 22H2 section. The ID on Microsoft 365 is WI442499.

46 Comments

Thanks for letting me know and bringing the temporary solution. :happyface:

Super Contributor

Hi @Ned Pyle thanks for letting us know.

 

May I share some thoughts and questions? 

 

1. Could it be the "lazy-writeback", introed and enabled by default since 1809 builds could be a cause?

 

Even when not utilizing SMB over network but usual local copy jobs e. g. from / two nvme one might notice a massive jigsaw effect when copying solid bigger files, especially with File Explorer, not so much with Robocopy as of its nature of multithreading.

Unfortunately there is no flag such as ADMX / CSP to disable lazy write-back for Explorer, to revert to the default behaviour of Windows Client and Server versions before build 1809.

The fact unbuffered IO does help, could be indicating this. 

 

2. Could you exlude the now wider default use of SMB compression, to be a potential cause for the issue? 

 

Microsoft

I can't share too many details yet, but we know exactly where it is getting slower in Kernel, nothing to do with SMB or compression. 

Senior Member

I would just want to correct 40% slower statement from Ned, im seeing way over 100% difference on setups like 10gb NICs (5gb/s vs 2gb/s) difference, or on SMB multi channel setups with multiple gigabit NICs.

Microsoft

@ivicask Thanks, I'll make sure that's clear above. The variance can be quite large

Senior Member

Thanks for letting us know, @Ned Pyle!

Pedro

 

Occasional Visitor

@Ned Pyle

Can this issue cause the copied data to arrive corrupted on the destination drive?

Microsoft

Not to our knowledge. In fact, it should make corruption less likely

Occasional Visitor

Was this fixed in KB5019509?

Microsoft

@jfm666 No changes yet

Senior Member

Is Windows 10 22H2 equally affected or just Windows 11 22H2? What's the ETA of the fix? I am holding off upgrading Windows 11 systems to 22H2.

Microsoft

@guhoo Windows 11. I don't have an ETA from the team that owns this code, it's not my own. 

Senior Member

@guhoo  Tested on Windows 10 22H2 is not affected and copies full speeds as usual. 

Senior Member

What is the ETA for the fix? It's been 3 weeks already and this is a very high profile bug in Windows 11 22H2.

Microsoft

@guhoo I cannot share timelines, sorry.  

Frequent Visitor

Even though I have excluded source and destination in windows security real-time protection, disabling it completely gave back most of the nic throughput.

 

Having this regression around for an entire month, is uh, disappointing...

 

 
Senior Member

I have Windows Security disabled. Transfer speeds are still cut by 50%.

Frequent Visitor

My NIC performance is reduced at least by 50%.  WiFi unaffected.  Have tested on multiple Win 11 22H2 PC's, multiple routers, multiple branded NICs - same results. 

Frequent Visitor

Correction.  WiFi performance is degraded as well.

New Contributor

slowdown SMB copy problem continues (as in not fixed), even with newly released KB5019980 update for Win11 22H2 as the problem is still noted there.

Senior Member

It shows a lack of concern for this flagship desktop OS.

Regular Visitor

Dear Microsoft team when will this severe bug be fixed ? Frankly this has taken too long to fix at this point. Please update us on when we can expect the fix ? Thank You

Senior Member

I think it's time for Elon Musk to acquire Microsoft to light a fire under them.

Someone said once: "Feedback is a gift.". 

@Lance Longreen quality feedback vs @guhoo any helpful feedback.

Microsoft

See updated note in the blog post. The preview fix for this issue is in Windows 11 Insider Preview Build 25252

 

The permanent production fix will come in a normal monthly cumulative update after this is validated. As I've mentioned before, the fix is not in SMB, and since it's not a component I own, I don't have further details on timing. 

Super Contributor

Good to hear. Thanks for the follow up Ned, much appreciated. 

New Contributor

thanks Ned.

 

though the fix didn't seem to make it onto the recently released 22621.1020 & 22623.1020 insider preview beta builds, just only with insider build 25252

New Contributor

Could this be affecting OneDrive sync? We recently migrated a client to OneDrive from their on premises file server and we had a report yesterday that it was taking hours to upload 60x 5MB photos. 

Regular Visitor

@erpmanila3w Where do you see that its fixed in insider build 25252 ?

Senior Member

@Lance Longreen , see "Fix update 11/28/2022" in Ned's post :)

Senior Member

SMB SLOW RATE TRANSFER FILE NOT SOLVE 

Regular Visitor

Does the last comment mean there is still a bug and it was not fixed in insider build 25252 ?!

 

Also when can we expect a mainstream non insider release of this severe bug ?

 

It's been far too much time at this point for something this severe.

Super Contributor

 

@MOGAbyte  please share your usecase and reproduction steps. And your winver (Windows Build).

 

For me it is solved in release channel or other channels of Windows 11. Especially when writing to USB 3.x via UASP.

Senior Member

What does "release or other channel" mean? Latest public available build is "22621.963" and its not solved yet, this bug is know since early versions of 22H2 somewhere early in the the summer, so its unsolved for 6+ months now....

 

I never had any speed issues writing to USB, this was exclusively bug with network, and impact can be drastic in business environments, but everyone is affected just dont notice it necessary.

 

For example im copying 200GB VHDX, i have 2x 1gbit NIC, im getting around 70MB/s,  while im getting around  220MB/s on previous windows build.

 

That means i need to wait 3x times more in order to copy files..

@ivicask it is a multi-layered issue. It does not only affect SMB this means also local file operation.

When you bring up teamed network it is getting more complicated. We have either LFBO Teaming or SET Teaming which work absolutely differently.

Can you actually uninstall a Windows CU and get the former performance?

Senior Member

I dont notice any speed impact on local files, im copying right now 2 files between 2 samsung nwme drives and its copying full speed that this drives are capable.

 

I dont remember if its even CU update which you can uninstall or you need to use option to go to previous version of windows, but it works fine on any 22H1 version, and not on any 22H2 no matter the updates/CU.

 

Regular Visitor

Multi layered issues also including local file copies now ?!

 

It has been over 6 months now.

 

Microsoft team and @Ned Pyle , when will this severe bug finally receive the attention it needs from your dev team and get fixed ?!

New Contributor

@Lance Longreen 

maybe in upcoming Jan 10, 2023 patch Tuesday update for 22H2 or newer for non-insiders

 

Does the last comment mean there is still a bug and it was not fixed in insider build 25252 ?!

no, it was already fixed in build 25252 & greater; just not yet fixed with 2262x (22H2)

 

I'm not sure if the recent insider preview 22621.1095/22623.1095 Jan. 5 builds include the fix

 

edit - just wait until Tuesday January 10, Lance (it's only ONE MORE day).  maybe MS will publicly release a fix by then

Regular Visitor

@erpmanila3w if its 1 day until patch then surely the patch notes are available by now ???? and we can see if its in this patch or are You just guessing that it might perhaps maybe be in a patch Tuesday ?

Senior Member

Installed 22623.1095 inside VM, its NOT fixed!

https://imgur.com/a/8EV9ia3

Senior Member

The update that came out today does not fix it. It is still listed as a known issue: https://support.microsoft.com/en-us/topic/january-10-2023-kb5022303-os-build-22621-1105-c45956c6-4cc...

Regular Visitor

Don't know what to say. The fact that something so trivial and at the same time very core, integral and important such as this has gone unfixed for plus 6 month in Windows by Microsoft says a lot, unfortunately not very good things. 

Super Contributor

Yuck. Thanks for the confirmation @MikeG1765 + @ivicask. Anyone of you having unified (premier) support? I wonder why what the underlying change was to raise this regression.

New Contributor

@MikeG1765 

thanks Mike - the KB5022303 update from Jan. 10 (build 22621.1105) definitely does NOT fix this problem

 


 

Don't know what to say. The fact that something so trivial and at the same time very core, integral and important such as this has gone unfixed for plus 6 month in Windows by Microsoft says a lot, unfortunately not very good things.


@Lance Longreen 

yup.  how highly disappointing that the problem still remains unfixed.

 

gonna have to wait for either a late January or early February update

Occasional Visitor

This is getting extremely frustrating. More so the fact that this has not been prioritized as an issue requiring an immediate hotfix and it's been going on for months now. And yes, it does affect local transfers as well as we can see on our workstations with Megaraid RAID6 SSD arrays. Not only it has not been prioritized but it is also being downplayed in the KB notes as something affecting few users when it is affecting everyone as it looks like a kernel issue.
And no we can not use robocopy. Not everyone just "copies files". A lot of us use applications over the network.
We are a small software engineering company and we heavily use very large VMWare Workstation images served over the network. The performance is crawling and we are resorting to copying images locally in order to work. Our entire workflow is messed up and we're spending 4/5 of work time, swapping images around. And we can not revert workstations to previous builds either because we need the latest WDK.

Just because the majority of Windows users do not notice the underlying problem, it doesn't mean this should not have been fixed yesterday. Do we really have to contact every major tech news outlet site informing them of the situation so this gains publicity and traction before Microsoft prioritizes a hotfix?

New Contributor

@Nodens  wrote:

Just because the majority of Windows users do not notice the underlying problem, it doesn't mean this should not have been fixed yesterday. Do we really have to contact every major tech news outlet site informing them of the situation so this gains publicity and traction before Microsoft prioritizes a hotfix?

 


NO, Nodens!  don't even think of going to that extreme,  sheesh!

heck, the problem won't get fixed until maybe either mid-February or in March so folks just come back next season when winter changes into spring (either late March or April) and wait patiently for the public official fix.  the recent 1/26 KB5022360 update for 22H2 (for non-insiders) still has the copy performance slowdown problem and MS only released new beta insider builds (22621.1245 & 22623.1245) {KB5022358} that possibly contains the fix but are still only available for windows insiders.

Co-Authors
Version history
Last update:
‎Dec 14 2022 01:21 PM
Updated by: