SMB compression behavior & settings changes
Published Aug 26 2022 01:55 PM 6,614 Views
Microsoft

Heya folks, Ned here again. As you know from a previous post, we released an end-to-end SMB compression capability with Windows 11 and Windows Server 2022. You can learn all about it at SMB Compression | Microsoft Docs

 

There was an idiosyncrasy in the initial design that we changed with the release of Windows Server 2022 update KB5016693 (OS Build 20348.946) and Windows 11 update KB5016691. From now on, if you request compression over SMB, we will attempt to compress the file regardless of its size or seeming suitability for compression. It doesn't matter how you request compression - robocopy, share setting, registry values - we will try to compress the file.  

 

For those playing SMB Trivial Pursuit edition: previously, SMB compression used a default algorithm where it attempted to compress the first 524,288,000 bytes (500 MiB) of a file during transfer and track that at least 104,857,600 bytes (100 MiB) compressed within that 500-MiB range. If fewer than 100 MiB was compressible, SMB compression stopped trying to compress the rest of the file. If at least 100 MiB compressed, SMB compression attempted to compress the rest of the file. This led to a need for understanding a complex formula and registry settings for various file sizes and formats - much too esoteric.  

 

This does mean that transferring less-compressible or uncompressible files will now get CPU time spent trying to compress them, but the solution is to just know your files. JPG, ZIP, DOCX, etc. formats that are already compressed see no advantage from SMB compression no matter their size. VHDX, ISO, DMP, and other large files with lots of whitespace will compress quite well. Don't request compression for those uncompressible files or don't set compression on servers if you only copy those types of files to them. We're thinking about ways to improve this in a future release so we can detect file types that we should skip compressing - stay tuned.

 

This demo shows the ways you can use compression as an IT pro or have your users get compression without having to train them. It's only about 4 minutes and worth a watch. 

 

 

Furthermore, we now have a set of Group Policy, PowerShell and registry options to control the legacy behaviors and to let you avoid editing the registry by hand for configuring client compression. 

 

Group policy

 

\Computer Configuration\Administrative Templates\Network\Lanman Workstation

Disable SMB Compression

Use SMB Compression by Default

 

\Computer Configuration\Administrative Templates\Network\Lanman Server

Disable SMB Compression

Request traffic compression for all shares

 

These all write to the following location and will override the 'native' registry settings as usual with group policy:

 

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\LanmanWorkstation
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\LanmanServer

 

PowerShell for SMB client

 

Set-SMBClientConfiguration [-EnableCompressibilitySampling <bool>] [-CompressibilitySamplingSize <uint64>] [-CompressibleThreshold <uint64>] [-DisableCompression <bool>] [-RequestCompression <bool>] 

-DisableCompression $true or $False - never compress even if server or application requested 
-RequestCompression $true or $false - always request compression even if server or application didn't specify it 

-EnableCompressibilitySampling $true or $false - control legacy sampling behavior 

-CompressibilitySamplingSize 1- 9,007,199,254,740,992 - size in bytes of range to sample in a file looking for compressibility

-CompressibleThreshold - 1- 9,007,199,254,740,992 - size in bytes of compressible data that must be found within that range

 

You shouldn't need to touch the sampling settings without a good reason. Maybe hit me up in the comments first if you're feeling the urge to touch those. :) A keen-eyed reader might notice that a range of 9,007,199,254,740,992 is not the Uint64 in the PowerShell. That's because the real limit in the code is 9PiB, not the 184,467,440,737,095,551,615 bytes of a UInt64 value we had to set in the Posh. If you've got individual files bigger than 10 petabytes, I would love to hear your story, regardless! 

 

These values are stored in the registry under: 

 

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanWorkstation\parameters

Name

Type

Default

Bounds

PS interface (Set-SmbClientConfiguration)

EnableCompressedTraffic

REG_DWORD

0

0 or 1

RequestCompression

DisableCompression

REG_DWORD

0

0 or 1

DisableCompression 

EnableCompressibilitySampling

REG_DWORD

0

0 or 1

EnableCompressibilitySampling 

CompressibilitySamplingSize

REG_QWORD

524288000

0 to 8 PiB

CompressibilitySamplingSize

CompressibleThreshold

REG_QWORD

104857600

0 to 8 PiB

CompressibleThreshold

 

PowerShell for SMB Server

 

 Set-SmbServerConfiguration [-DisableCompression <bool>] [-RequestCompression <bool>] 

-DisableCompression $true or $False - never compress even if client requested 
-RequestCompression $true or $false - always request compression even if client didn't specify it

 

These values are stored in the registry under: 

 

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanServer\parameters

 

Name

Type

Default

Bounds

PS interface (Set-SmbServerConfiguration)

EnableCompressedTraffic

REG_DWORD

0

0 or 1

RequestCompression

DisableCompression

REG_DWORD

0

0 or 1

DisableCompression 

 

I'm updating SMB Compression | Microsoft Docs & PowerShell Help to cover all this too, don't worry. 

 

You should get huge benefits from SMB Compression in many day-to-day operations, it's a time saving, bandwidth saving, and - now- very easy to use feature. More improvements are coming, this isn't the end.

 

Until next time,

 

- Ned "Squishing" Pyle

14 Comments

Awesome demo @Ned Pyle Thank you for Sharing with the Community :stareyes:

Microsoft

@James van den Berg :) thanks James

Nice! Great feature! 

Occasional Visitor

Heyho,

 

do I read it correctly in that

  • compression-enabled share
  • 1.000MB JPG

Windows will compress during the whole copy process?

Microsoft

@Kugelfischer it would try if you have told the share to ask for compression, but with a jpg it won't get you any savings, as you probably guessed. Our next phase of this feature is to add intelligence around skipping compression attempts on uncompressible file types. 

 

If you're copying a lot of already-compressed small files (zip, jpg, docx, etc), you can either not use SMB compression or you can tinker with the sampling settings to just not compress smaller files in general.

Occasional Contributor

@Ned Pyle 

an option to mark a share for mandatory SMB compression would be highly desired/very useful.

Microsoft

@Rafał Fitt That is possible now, you can set a share or entire smb server to request compression of SMB clients

Occasional Visitor

@Ned PyleOk, that's why I was asking.

 

You will know of more uses cases, but a generic

- try to compress a few chunks at once during transfer start

- try to compress X chunks every now and again

with a quick-but-low-comp algo (gzip or whatever) to determine if you apply

- comp level x-y [1-6 or however many levels]

might already result into a pretty sophisticated self-adjusting automatism.

 

One might add other factors like slow link detection into that mix, but that might already become pretty irrelevant once that logic is up.

Again, you'll be way more aware of use cases.

 

But compression is definitely something that should be on-by-default 8) so really looking forward to it.

Microsoft

@Kugelfischer That's basically how original sampling works today. The problem we found was that many overall inefficient file types (like VHDX, for instance) filled in their data sequentially at the start of the file, so we'd sample a bunch, find no compressible data, then give up on a lot more data that would've compressed. You can get around that by sampling a lot more often but it's expensive to do this operation and we hope to find something more cheaply deterministic. We also want to understand the file type so we don't have to waste cycles ever bothering. My ultimate goal would be to have compression be on by default, but I can't until I do this work.

 

I have a very sharp algorithm dev I'm noodling with on this, it's just a question of priorities for now.  We are very busy and have an 'ok' solution for the meantime.

Senior Member

Hi @Ned Pyle ,

 

If I'm not mistaken the registry keys mapping to the PS cmdlets are wrong: I guess the key for GP has been taken and the substring "SOFTWARE\Policies" was removed, but actually the right locations are:

* Client: 

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanWorkstation\Parameters

* Server: 

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanServer\Parameters

 

Good to read about how the default values are changed, benefiting most people and use cases. I've also noticed tuning has become better: higher ranges for the named values CompressibilitySamplingSize and CompressibleThreshold, but also the introduction of RequestCompression for the SMB server. Of course GP policies are welcome as well!

 

I'm definitely awaiting the new intelligent feature you're talking about in the comments, so please give us a heads-up, when you have more to share ;)

 

Thanks again!

Pedro

 

Microsoft

@PadrePedro thanks those are dumb typos from me copy pasting all over the place. Fixing now

It would be really nice to have the actual compression ratio while copying the files using smb compression.

Microsoft

@Gabriel Luiz Agreed, on my to do list. I don't love the current perf counters and it would be nice if robocopy gave instant gratification here too.

Tip

 

If using administrative templates based on central repository. 

 

Copy the new ADML LanmanServer.adml and LanmanWorkstation.adml files.

 

Copy the new ADMX LanmanServer.admx and LanmanWorkstation.admx files.

 

GabrielLuiz_0-1661997606862.png

 

GabrielLuiz_1-1661997616194.png

 

I got a scare here, when I didn't see the new GPOs on the server.:suprised:

Co-Authors
Version history
Last update:
‎Nov 07 2022 03:33 PM
Updated by: