SMB over QUIC: Files Without the VPN
Published Mar 02 2020 12:01 AM 105K Views
Microsoft

Update 8/17/2021: this is all available now, come and get it! https://aka.ms/smboverquic 

 

Hi folks, Ned Pyle guest-posting today about SMB over QUIC, a game-changer coming to Windows, Windows Server, and Azure Files. In today’s world, SMB file share access for mobile users requires expensive & complex VPNs. Departments trying to use Azure Files often find their ISP has blocked port 445. Even though users are just as likely to be deskless and organizations are doing more hybrid computing than ever, SMB hasn’t kept up.

 

That’s all changing with SMB over QUIC.

 

QUIC is an IETF-standardized protocol that replaces TCP with a web-oriented UDP mechanism that theoretically improves performance and congestion, but still tries to maintain TCP’s reliability & broad applicability. Unlike TCP, QUIC is always encrypted and requires TLS 1.3 with certificate authentication of the tunnel.

 

1.png

 

QUIC’s already in use in Windows 10 through the Edge browser and other apps. With SMB over QUIC – I don’t have a clever marketing name for this yet :) – QUIC becomes the transport, optionally replacing TCP/IP and RDMA, as well as a tunnel securing all SMB payloads with encryption, even if SMB encryption is not enabled, all while multiplexing over port 443 to an enlightened share. An admin will be able to opt-in to this new capability by deploying a Windows Server at the edge of the network, installing a certificate trusted by clients, then enabling the QUIC option. Or enable it on their Azure Files instance.

 

We have two design imperatives for SMB over QUIC:  

 

  1. Secure: Prevent man-in-the-middle and spoofing by malicious parties as well as guarantee no sniffing of that sweet file payload or allowing any user credentials onto the Internet. The entire SMB conversation – negotiate capabilities, authentication, authorization, message bodies – all occur inside the QUIC layer, just like if the user was in an IPSEC or VPN tunnel. Yes, it even blankets NTLM challenges.

  2. Simple: The user experience for SMB over QUIC can’t change from their corpnet/LAN/branch office experience, it’s too expensive to retrain users. So, we don’t add extra UI or command-line arguments to the client experience – their updated Windows 10 machines will simply try TCP and RDMA like always, but then wait briefly and try QUIC too. This means if they can get faster perf on a local network with RDMA or unencrypted TCP, they will. And if they are travelling or an admin mandates QUIC, they can get that instead. All seamless to the end user and their apps.

Here’s a quick (heh) demo of the user experience. Spoiler alert: a user probably can’t tell anything changed except that SMB now works when I’m at a hotel for Microsoft Ignite.

 

 

The question I always get at this point is: when is this coming? I don’t have a good answer yet, but as we get firmer, I'll get more details out there. This is a key technology for Azure Files and Windows Server edge computing, as well as our mobile strategy, so all I can say is that it’s coming. As you can see from the demo, we’re far along. Check back at the ITOpsTalk.com and FileCab blogs for more details and info on Insider Previews this year. We are working with third parties to offer up this choice in other mobile platforms as well – you should be asking your vendors what their plans are.

 

I hope you’ve enjoyed learning about this new feature, I think it’s a real game changer. If you have questions, hit me up on twitter or DM me on TechCommunity.

 

- Ned Pyle

51 Comments
Steel Contributor

Awesome! I hope we will see this for Azure Files too. Any news on that @Ned Pyle ?

Microsoft

@Jonas Back yes the plan is to bring it to Azure Files too. Makes perfect sense there 

Copper Contributor

@Ned Pyle good move forward. Has the architecture also being designed to allow for SSL-offloading in DMZ (which obviously breaks the client-SSL there), and reestablishing to the backend winserver using a new/different and NOT user-specific certificate (so the SMB authentication itself doesn't rely on the certificate identity)?

Microsoft

@thorsten_rood This is more a QUIC question, so I'm leery of speaking out of turn. I'd recommend talking to their experts, and I'll try to find out here with the Windows QUIC team.

 

All SMB authentication still happens normally within the TLS tunnel (as if it was a VPN) so SMB is not relying on cert-based identity or auth - it will still use NTLM or Kerberos (with KDC proxy). This model is just swapping out the transport, SMB is unchanged.

Copper Contributor

thank you Ned. so maybe (as you said it's decoupling transport authN from file authN) we might trial around breaking and reestablishing the transport as described using existing offloading ADCs. ;)

Microsoft

@thorsten_rood See, you already know more than I do about this :D 

Copper Contributor

Hi

 

Very good! When will this be available?  Will this be available on server 2019? I have a windows server 2019 with AZF sync agent installed I want to offer my remote users mappings to my on-premise domain joined server 2019 that syncs with AzF.

 

Regards,


T

Iron Contributor

@Ned Pyle , I can't wait for this. We at GE are in the middle of a large legacy DC to Azure migration and this would be very useful. If you need a guinea pig to test this out please let me know.  

Microsoft

@Steskalj That's great to hear! If your TAM wants to arrange a call with me about details or feedback, I'm sure we could spare 30 min for GE ;)

Microsoft

@TT-XX-TT Hi. I don't have official timelines and platforms yet, but the goal is the next version of Windows Server & Azure Files. There is a possibility of backport to some flavor of WS2019 but nothing officially in plan.

Copper Contributor

@Ned Pyle: You refer to this once as "QUIC over SMB" — I assume that's a typo? 

Microsoft

@lfaraone-dbx Doh! I will fix, thank you

Copper Contributor

How about a name like most wrapped protocols, SoQUIC (So Quick)? O.o

 

Microsoft

@Nellson :o

Hi Ned. It's been about 4 weeks since your last update and some customers want to use this functionality now. They are in WFH status and need access to their Azure File Share. They have a S2S VPN. Most of the major ISPs in their area block access to port 445. Do you have a new availability date or a suggestion of alternative access methods that are available today? I would think this would have a high priority like WVD. Thanks, Bruce

Microsoft

Hi. I hear you and I wish I could make this go faster - the SMB over QUIC feature is basically done. But it relies on several components of the Windows OS being completed, and it is tied to the Windows ship cycle. I don't have control over any of these things. Note that is just around Win10 - Azure Files supporting QUIC is another beast on its own schedule and it's not near ready. 

 

Right now VPN to AF is their only solution, either site to site or (probably better for home-based users) point to site https://docs.microsoft.com/en-us/azure/storage/files/storage-files-configure-p2s-vpn-windows

 

 

 

 

Steel Contributor

@Ned Pyle Thanks for the honest reply - this makes it easier for me as a partner to give the customers what to expect. We'll go for VPN at the moment.

 

Do you think we'll see Azure AD authentication/integration for Azure Files some day? And I'm not talking Azure AD Domain Services integration, I mean direct Azure AD integration? Even though I think OneDrive/SharePoint is the way to go, we see customers asking for this. But maybe technology wise, this is not where Microsoft is heading but rather pointing at other solutions?

Microsoft

No sweat, I try to be real :). 

 

Re: AAD integration and Azure Files future integration, I don't have any real insights on the plan there. I suggest emailing AzureFiles@microsoft.com, you'll get to that PM team and they might have some questions for you or feedback they'd like to gather.

Copper Contributor

This will be nice. Really makes me want to consider using Windows as a file server over others. I have been debating for a while, so, thanks for the hand!

Copper Contributor

Very nice! Hopefully Samba will adopt it as well :stareyes:

....and good graphics, it is clearly seen that in this chain vpn app and is not needed

Copper Contributor

@Ned Pyle any updates on this topic?

Microsoft

@mafrank Getting closer! A keen eye will note in the Windows Insiders and Windows Server Insiders that the SMB over QUIC client has started to appear in wire captures and SMB powershell has started to update. More to come. :)

Steel Contributor

@Ned Pyle Awesome - thanks for keeping us updated with the nitty gritty details on where in the release cycle you are. I'll enable this straight away once it hits public on all my Azure Files :)

Copper Contributor

@Ned Pyle Thanks for the great news. Is it possible/planned to open SMB over QUIC for 3rd party vendors like Netapp or proxy it via something like a QUIC gateway?

Microsoft

@mafrank we have been working with any partners interested in using SMB over QUIC and will have our annual plugfest soon to help them. I can't name names yet but I have watched a few partner demos of their SMB over QUIC recently :)

Copper Contributor

@Ned Pyle Any update on when this will be previewing with Azure Files?

Microsoft

@EricNiemiec as soon as I can make it happen but I don't have a date yet, sorry. 

Brass Contributor

The next step would then to manage multipathing over QUIC gateways. QUIC gateways would have to be protocol aware (SMB, NFS, MS-SQL ?..) , .... Looks like reinventing the wheel at app level when Multipath TCP deals with these questions at network level in an app/protocol agnostic fashion for LAN and WAN scenarios. It’s time to wake-up!

Copper Contributor

@Ned Pyle Nine months have passed, "baby" should be due right about now? :happyface:

Copper Contributor

Hi @Ned Pyle , do you have any date for availability yet?

Another question: What are the plans to secure access to those fileshares in combination with conditional access? I assume coporations will not simply allow quic to fileshares without an extra layer of identity checks. I could see sth. like AzureAD Application proxy in front of those fileshares but is this the direction you are heading to?

 

Thanks

Christian

Brass Contributor

Hi @christianlehrer , very good question. This is where "QUIC gateways" come in and blow-up the naive model of SMB over QUIC over the Internet. Otherwise, for internal datacenter, Multipath TCP provides a far more comprehensive, and versatile solution to resilience, scalability and network awareness. .

Copper Contributor

@Ned Pyle 
Is there a private preview for this, yet?  Not that you need help, but I can get customers for trials....

Microsoft

@RussellDespain Russell old pal! :D 

 

I'm not allowed to talk about dates yet but... news is coming soon.

Brass Contributor

2021 should be an amazing year, because it will be possible to compare several approaches to Multipathing for SMB and NFS.

 

I have in mind according to different use cases, comparing:

- SMB over QUIC on Windows with multipathing managed by SMB Mutli-channel

- SMB (w/o Multi-channel) over Multipath TCP (on Linux, iOS, Android, MacOS)

- NFS 4.1 with Session Trunking

- NFS 4 w/o Session Trunking but over Multipath TCP

 

in follow-up to some early linear scalability experiments performed by Martin Houry during his internship in 2016.

 

@Ned Pyle, I would be useful that you share too your point of view according to the current IETF WG discussions about Multipathing requirements support for QUIC (aka MP-QUIC, MPQUIC, ...) and especially about SMB Multi-channel requirements both in LAN and WAN scenarios. 

 

The same exact question applies also for NFS Session Trunking to either use Multipath QUIC (when defined and implemented) and/or Multipath TCP (once implemented on Windows). 

 

According to me, one a the key learning is the upper layer knows the application needs and the lower layer may "knows" the network capabilities.

 

That's where new APIs to manage these traffic engineering questions would be required and protocol implementation should become extensible and no longer monolithic (e.g MS tcpip stack, lwIP, ...).

 

I can share on this topics the excellent research currently in progress at https://pluginized-protocols.org/tcp/ 

 

 

Microsoft

@Olivier Hault  Howdy. Thanks for sharing all this. From the SMB side, we reply on the MS QUIC team here in Windows for much of the transport story specific capabilities (the same way we do for TCP), so I'd definitely recommend engaging with them directly. They are very eager for feedback, we work with their PMs and lead Devs constantly.

 

That said so, I still want your feedback on the SMB aspects and if the SMB over QUIC implementation needs further options for you to use it (for example, a proxy gateway/forwarder you mentioned before is part of our roadmap). We do support SMB multichannel in QUIC scenarios, we also support the new SMB compression options, but there is plenty of SMB over QUIC v2 work in my roadmap and I've love your thoughts on populating it. 

 

I'm going to read these links you shared, they look very interesting. Thanks!

Copper Contributor

Hi Ned,

 

Does conditional access figure into this in any way? For example, I would prefer that a work-from-home employee only be able to connect to the file server on his work laptop, and one that is also "compliant".  In that scenario there would need to be more than just a username and password required to connect (otherwise the user could potentially connect from his personal laptop, etc.). I would guess this sort of scenario could work by requiring Microsoft Intune enrollment of the client device for device compliance checks (as is already possible) and leverage Azure conditional access. There would need to be some communication between Intune / Azure and the on-premises Windows server, letting the server know the device the user is connecting from is compliant before allowing the user to connect to SMB. With Azure conditional access, over course, many other things could be "checked" in addition to device compliance, such as location client is trying to connect from. For example, we may want to limit connections to only those inside the United States and Canada.

Microsoft

Hi @ffmike. This is a great point we've started thinking about (but don't have a schedule for yet; still just noodling). I'd love to pick your brain on this in a month or so, hear what your org would like to see.

Hello @Ned Pyle I am not sure if I understood the design 2 correctly. 

If I have a Windows 2022 LTSC it seems the SMB over QUIC firewall settings are disabled (incoming) by default for Domain Network / Private Network.
Is this correct, and if so, why?

I have checked that Windows 10 21H1 has no SMB over QUIC predefined firewall rules, so I can only assume it will be a thing in Windows 10 21H2 just as with Server 2022 is 21H2.

Let's assume the client and server both support QUIC
- do I need to actively configure the firewall for same domain or same subnet / private (see above)

- will QUIC be used for SMB preferably over TCP or SMBDirect (I am not sure if this is used in Client/Server, the only scenario I connect it to is something like ScaleOut-FS)

- If a client / server SMB transfer uses QUIC, does this make SMB encryption (e.g. configured via Server Manager or WAC) obsolete?

- If a client / server SMB transfer uses QUIC how does this affect SMB Multichannel as I've learnt some basics here: SMB Multichannel performance - Azure Files | Microsoft Docs

Thanks in advance for your time on elaborating my questions, a docs or a new article for more details!

 

Microsoft

This will all be made clear in a few weeks with some announcements and docs. But for your four last questions:

 

- We will take care of it automatically

- No

- We won't double encrypt by default but you can override this if you want

- MC still works

Thanks Ned for the timely and brief response. 

Iron Contributor

Ned, your August says this is all available now, but I'm still not seeing this option on Azure Files - what am I missing?

Microsoft

HI @Mike Crowley. It's available in Windows Server 2022 Datacenter: Azure Edition preview but SMB over QUIC is not available in Azure Files yet. It will be some time before it is there, long enough that I don't have an ETA to share from that team.

Brass Contributor

Hello all,

I find the technology highly exciting. What I can't read exactly from the current documentations so far:

Is there a dependency that the Windows client must be joined or hybrid joined in onPrem AD?

Or is it sufficient if the client is only joined in Azure AD and the user has a hybrid identity with a valid Kerberos token?

Microsoft

@GerritEllmer Sorry for the delay, holidays were on. There's no need to join any domain, the user account is using Kerberos or NTLM for the SMB auth and the client machine is using a certificate.    

Brass Contributor

@Ned Pyle  Thank you for your answer. That fits for me.  :smile:

But how do I get a Windows Server 2022 Datacenter: Azure Edition running in my onPremise data center?

 

My scenario is:

  • We have a classic data center with various file services based on SMB.
  • We have AAD only Joined Devices and Hybrid Users with valid Kerberos.
  • User and computer certificates are also available.

How can I now use SMB over QUIC to allow my users to access my onPremise file shares from their AAD Joined Clients?

Microsoft

@GerritEllmer Azure Edition will be supported on Azure Stack HCI for on-prem datacenters - it's currently in private preview. I don't have a public date yet, though, I have to wait for HCI folks to be ready. Azure Edition is already... ready. :)

Hi @Ned Pyle thanks for the update, could you - please - try to convince the experts working on servicing to rename the Product Category in WU Catalog / WU from

Server to a more common name using the correct product name?
This is what we get. Now it is still in preview I hope this can be changed.

Status Quo:

K_WesterEbbinghaus_0-1644532065135.png


If I had a magic wand:
Why is Server 2022 Hotpatch Category is not named "Windows Server 2022 Azure Edition" or "Windows Server Azure Edition hotpatch" or even "Windows Server 2022 Azure Edition hotpatch" it ruins alphabetical sorting in the WSUS product list.


I know it is not your area but since you mentioned it is in preview I hope you can raise this little point. (Artem Prochinkin explained me why the other product category on the top is named in a weird way, too but this won't be changed anymore, so my hope is that at least this category will follow your product naming convention :)

Brass Contributor

@Ned Pyle 9 month ago you wrote that "SMB over QUIC is not available in Azure Files yet." Is there anything new?

 

Documentation mentions the option of using a VM with Server 2022 as a proxy, but I don't like a VM in a cloud-native solution ...

Copper Contributor

Hi @Ned Pyle thanks for the article, now very old but still excellent!

 

And it was fantastic to see Microsoft make this available on Windows 11 and Server 2022, but unfortunately this is still a very niche use case, and not what most businesses can take advantage of (yet). The two big game changing features that we're still hoping for are:

1 - Windows 10 client

2 - Azure Files native support (I do appreciate there is an option to use a File Sync server https://docs.microsoft.com/en-us/azure/storage/files/storage-files-networking-overview#smb-over-quic...

 

Is there any hint that either of these things might be on the roadmap soon? It would be a shame if we have to wait for widespread adoption of Windows 11 before we can take advantage of this technology.

 

 

Microsoft

@Simon_M_999 Thanks! It's absolutely planned for Azure Files, it's key to the future of that service. I don't have an ETA from them, hoping to get that firmed up soon. For Win10 we have discussed this heavily but are waiting on some decisions from other dependency & business teams - so for that one I can't say for sure if it's going to happen or not. It's definitely not "no" currently, but I don't have a firm "yes"...

Co-Authors
Version history
Last update:
‎Aug 17 2021 03:18 PM
Updated by: