Mastering Configuration Manager Bandwidth limitations for VPN connected Clients

StefanRöll · ‎Apr 03 2020

Hi, Jonas, Roland and Stefan here!

We are Microsoft Premier Field Engineers (PFEs) based in Germany focused on Microsoft Endpoint Manager related topics. Due to the current situation we get a lot of questions from our customers around Configuration Manger traffic for VPN connected Clients.

TL;DR:

Providing Content to VPN connected Endpoints can be challenging. While the ideal solution would be VPN split tunneling and content downloads directly from the Internet, the implementation may currently not be possible. Reducing the bandwidth of Distribution Points (DPs) and follow some guidelines can help you to successful distribute patches without overloading your VPN infrastructure until you are able to configure VPN split tunnel.

If you have VPN split tunnel in place just scroll to scenario 1 and follow one of the mentioned links.

The challenges:

With the current pandemic outbreak of COVID-19 many of us are working from home forced or voluntarily. In addition to the unknown duration of this pandemic outbreak, this also has the potential to change the way we work in the future. Many organizations might offer their teams additional work from home (WFH) opportunities after the crisis.

Managing your Devices (especially security updates and software installations) is necessary and will become challenging as the majority of your work force will be connected to the corporate network via VPN. Depending on the number of clients even a couple of 100MB security updates will quickly add up to several Gigabyte or Terabyte that needs to be pushed out over your VPN network. Without further consideration you can quickly overload your VPN connection causing other applications to degrade in performance or to completely fail. Trust us, you don't want ConfigMgr to be the reason for network overload.

What is a VPN split tunnel configuration?

A VPN split tunnel configuration allows for some traffic to go through the VPN tunnel and for all the other traffic to go directly to the internet. That is extremely useful in a work from home scenario were maybe hundreds of users connect to your VPN gateway and you want to avoid every traffic to go through the VPN, into your on-premises infrastructure and out to the internet again.

A great example is Office 365 with Teams, Outlook and SharePoint traffic. Read about the setup for Office 365 here: How to quickly optimize Office 365 traffic for remote staff & reduce the load on your infrastructure...

A simplified VPN split tunnel setup could look like this:

In the example picture above the on-premises infrastructure is using a subnet of 10.0.0.0/24 and a VPN subnet of 10.200.2.0/24 for your VPN-Clients.The client in the example is connected your on-premises infrastructure via VPN. Locally on the client you would end up with the LAN or Wi-Fi Adapter connected to your home router and having the IP 192.168.2.52 for example and a second adapter connected to the corporate on-premises VPN gateway having the IP 10.200.2.234 as shown in the picture below:

The clients local route table could then look like this:

Everything send to 10.0.0.0/24 will be send to your VPN adapter 10.200.2.234 and therefore to your on-premises infrastructure (second line in the picture) and everything else 0.0.0.0 will be send to your local home router 192.168.2.1 and therefore directly to the internet (first line in the picture). So, the traffic is split and either send to the on-premises infrastructure or the internet directly, hence the name VPN split tunnel configuration.

If you are using local outbound firewall rules, you might need to adjust them, to allow traffic to internet resources according to your split tunnel configuration.

If you are using a web proxy for your clients, change your proxy configuration to bypass the download URLs which should go directly to the internet. Otherwise the traffic might be routed through your VPN to the proxy and from the proxy to the internet and is not using the split tunnel bypassing VPN as shown in the picture below:

Considerations / Preparations

Talk to your network team how much bandwidth ConfigMgr is allowed to utilize over VPN
We recommend to always use dedicated boundary groups for VPN Addresses to:
- redirect VPN traffic to specific DPs if you have dedicated DPs for VPN clients
- be able to report download volume on a per VPN boundary group basis
- disable Peer Cache for VPN Clients (via Boundary Group Options)
- use IP range boundaries, because the VPN connection might have a 255.255.255.255 subnet mask - with IP subnet boundaries you would end up having one boundary per client
Deadline and start time should not be the same time, to avoid many parallel downloads, because in that case the download will start almost immediately after policy arrival. But if you set the deadline to be in the near future, the download start time will be slightly randomized
Plan your deployment strategy based on overall download time and maybe stretch the deployment times
- Download example:
  - You need to distribute 500MB to 5000 clients over VPN
  - You would need to transfer 500MB * 5000 Clients = ~2,5TB of data through the VPN gateway
  - Assuming you have 200Mbit/s available on the VPN gateway, all clients downloading at the same time would take around 24h
    - Keep in mind that there is also some protocol overhead and the actual download time could be higher
  - So, the VPN would not be available for any other traffic for around 24h, assuming no quality of service (QoS) is in place
- Consider enabling BranchCache on your DPs and Clients. This will help to reduce the amount of data that needs to be downloaded over VPN, by leveraging the Data Deduplication feature of BranchCache. You can find more technical details in the Speed-up Content Distribution with PullDPs and BranchCache Blog, as the technique behind the feature applies to this VPN scenario as well.
- Don’t forget the proxy configuration as mentioned in the split tunnel section above

Scenario #1: With VPN split tunnel or no VPN at all

If your clients are directly connected to the internet or if you have a VPN split tunnel configuration in place, you can make use of the modern content delivery methods described in the below articles:

Managing ConfigMgr clients on the Internet:

Managing remote machines with cloud management gateway in Microsoft Endpoint Configuration Manager

Patching ConfigMgr clients on the Internet:

Managing Patch Tuesday with Configuration Manager in a remote work world

For Office 365 traffic:

How to quickly optimize Office 365 traffic for remote staff & reduce the load on your infrastructure...

Office 365 VPN testing:

Office 365 Network Onboarding tool POC updated with VPN testing

Scenario #2: Without VPN split tunnel and with dedicated DPs for VPN subnets

You are not able to configure VPN split tunnel yet and you have dedicated DPs for all your VPN clients available.

As mentioned in the considerations section, always use dedicated boundary groups for VPN clients and set your dedicated DPs as possible sources.

In that scenario we really need to make sure that the VPN connection will not be stressed with all your clients downloading content and clogging up the VPN connection (as shown in the left picture below), so that business critical traffic can still flow as needed.

There are different ways to achieve this, but from a ConfigMgr admin perspective, the easiest way to achieve this is by setting a maximum bandwidth limit on the IIS of the DPs or set a QoS policy which we describe in scenario 3.

Let's assume we have 200Mbit/s on the VPN gateway overall and we need 100Mbit/s for business-critical data.

If we limit the bandwidth on our DPs to a combined maximum of 100Mbit/s, we will still have the business critical 100Mbit/s left on the VPN gateway. (as shown on the right picture below)

To configure the IIS bandwidth limitation, just open the IIS Manager on your DP,

select the Default Web Site => Advanced Settings => Limits => Maximum Bandwidth

The technical reference for IIS Limits can be found here.

Please keep in mind that the limit is configured in Bytes per Second. To help you with the conversion from Megabit to Bytes, you can use the Windows 10 Calculator by choosing Data as shown in the picture below:

Scenario #3: Without split tunnel, but NO dedicated DPs for VPN subnets:

The challenges are the same as in scenario 2 and we need to keep business critical data flowing, but you don't have dedicated DPs just for VPN Clients. Instead the DPs are also used by clients on your on-premises infrastructure and a bandwidth limit as in scenario 2 might not be the right choice, because that would also limit the on-premises bandwidth.

Instead we could use local QoS policies directly on the DPs and just limit the bandwidth for every subnet for VPN clients.

To configure a local QoS Policy, go to Start => Run => gpedit.msc

In the Local Group Policy Editor, expand Computer Configuration => Windows Settings => Policy-based QoS

Do a right click and create a new policy.

Give it a name and specify the Outbound Throttle Rate:

Please keep in mind that the limit is configured in Kilobytes or Megabytes per second and not Kilobit or Megabit per second.

Click Next, select All applications and Next

Configure the destination IP Range of your VPN Clients. Only this Range will be affected by the throttling:

Click Next and Finish to complete the configuration. The throttling limit will be in place right away.

If you have multiple DPs or VPN ranges, you need to configure this accordingly on each DP and for each subnet.

The technical reference for QoS Policies can be found here.

Other configuration options:

BITS Throttling:

There is the possibility of using BITS Throttling via Client Settings or GPO, but we would advise against it. First, even if you limit the download speed of each client to 1MBit/s, you can still overload your VPN gateway. Second, all downloads via BITS would be throttled, not only ConfigMgr Traffic.

LEDBAT:

LEDBAT can be an easy solution to get your Traffic concerns out of the way with a single checkbox. While this is a strength of this solution, it is also the only configuration option you have - you can only turn it on or off. When it works it is great, but if your VPN gateway is under high load, VPN connected clients might not be able to download important security updates at all. Therefore, if you want to use LEDBAT, monitor that your clients are still able to download content. More technical Details about LEDBAT can be found here.

Express Updates:

Before Windows 10 1803 you had the possibility to use Express Updates to reduce the amount of data that needs to be downloaded from a client. If most or all your clients are above 1803, don´t consider enabling this feature. It will add a lot of overhead to your infrastructure and clients.

Conclusion:

The best case would be to have a VPN split tunnel configuration or a direct internet connection and consume as much as possible directly from internet resources to minimize pressure on the VPN gateway and to have the most flexibility when using modern delivery technologies.

But since sometimes configuring VPN split tunnel can be challenging for some customers, one of the above solutions can help you on the way to a modern managed client.

Stay safe and split the tunnel

Roland Spindeler

Premier Field Engineer - Microsoft Germany

Jonas Ohmsen

Premier Field Engineer - Microsoft Germany

Stefan Röll

Premier Field Engineer - Microsoft Germany

Disclaimer:

This posting is provided "AS IS" with no warranties, and confers no rights

Changelog:

V1.0 - 04/03/2020 - initial release

V1.1 - 04/06/2020 - added local firewall rules, IP Range for BG, Peer Cache BG Options and Express Updates section