Hi folks, Ned Pyle guest-posting today about SMB Compression, a long-awaited option coming to Windows, Windows Server, and Azure. As you know, the growth of cheap storage and plentiful network bandwidth over the past decade had an immediate effect: files got bigger and less efficient ;). SMB Compression tackles this by adding optional inline whitespace compression to file transfers, removing congestion and copy time from offenders like virtual machine disks, raw graphics, scientific data, and other large file formats. Depending on the (in)efficiency of file formats and the IO pattern, the performance increase in copies can be impressive:
Everyone who’s ever seen a ZIP file knows what compression is, so let’s go straight to the demo and see it. Here I compare using robocopy with and without the new /compress parameter on a very compressible file, then try again between a very compressible and very un-compressible file.
As you saw, we also provide for the case of “what if I don’t know how well a file will compress? I don’t want to make my copy slower in order to find out.” The solution is we measure compression efficient for an admin-specified file size and if we aren’t seeing savings, we just turn compression off automatically.
SMB Compression uses a negotiation mechanism, so multiple compression algorithms are possible, and vendors can add their own. Currently the list is XPRESS (also known as LZ77), XPRESS Huffman (LZ77+Huffman) and LZNT1. This gives us excellent CPU performance bang for our compression buck. It’s not tied to dialects so much as OS versions – there will not be an SMB 3.1.2 to make this work. SMB Compression is already part of Windows 10 and the SMB2 protocol spec, although it currently has no usage options or server OS to make use of it.
The first usage options of SMB Compression will be:
- Hyper-V Live Migration with SMB
- Robocopy
- Xcopy
We don’t have a user experience for Chad from Accounting, currently; our first phase will be for applications and admins like you. One thought is that we could enable compression on a per mapped-drive case and greatly lower the threshold of measuring efficiency, but we’ll be experimenting and asking you questions on this as the feature evolves.
The question I typically get at this point is: when is this coming? I don’t have a good answer yet, but as we get firmer, I will get more details out there. This is a key technology for Azure Files and Windows Server computing, so all I can say is that it’s coming. As you can see from the demo, we’re far along. Check back at the IT Ops Talk and FileCab blogs for more details and info on Insider Previews this year. We’re working with third parties to offer this option – ask your SMB vendors about their plans.
I hope you’ve enjoyed learning about this new feature, I think it’s going to help you save a lot of wasted time and bandwidth. If you have questions, hit me up on twitter or DM me on TechCommunity.
- Ned Pyle