First published on TECHNET on Feb 09, 2017
Windows 10 Insiders can now take advantage of overlay networking and Docker swarm mode to manage containerized applications in both single-host and clustering scenarios.
Containers are a rapidly growing technology, and as they evolve so must the technologies that support them as members of a broader collection of compute, storage and networking infrastructure components. For networking, in particular, this means continually striving to achieve better connectivity, higher reliability and easier management for container networking. Less than six months ago, Microsoft released Windows 10 Anniversary Edition and Windows Server 2016, and even as our first versions of Windows with container support were being celebrated we were already hard at work on new container features, including several container networking features.
Our last Windows release showcased Docker Compose and service discovery —two key features for single-host container deployment and networking scenarios. Now, we’re expanding the reach of Windows container networking to multi-host (clustering) scenarios with the addition of a native overlay network driver and support for Docker swarm mode , available today to Windows Insiders as part of the upcoming Windows 10, Creators Update.
Docker swarm mode is Docker’s native orchestration tool, designed to simplify the experiencing of declaring, managing and scaling container services. The Windows overlay network driver (which uses VXLAN and virtual overlay networking technology) makes it possible to connect container endpoints running on separate hosts to the same, isolated network. Together, swarm mode and overlay enable easy management and complete scalability of your containerized applications, allowing you to leverage the full power of your infrastructure hosts.
Swarm mode is a Docker feature that provides built in container orchestration capabilities, including native clustering of Docker hosts and scheduling of container workloads. A group of Docker hosts form a “swarm” cluster when their Docker engines are running together in “swarm mode.”
A swarm is composed of two types of container hosts: manager nodes, and worker nodes . Every swarm is initialized via a manager node, and all Docker CLI commands for controlling and monitoring a swarm must be executed from one of its manager nodes. Manager nodes can be thought of as “keepers” of the Swarm state—together, they form a consensus group that maintains awareness of the state of services running on the swarm, and it’s their job to ensure that the swarm’s actual state always matches its intended state , as defined by the developer or admin.
Worker nodes are orchestrated by Docker swarm via manager nodes. To join a swarm, a worker node must use a “join token” that was generated by the manager node when the swarm was initialized. Worker nodes simply receive and execute tasks from manager nodes, and so they require (and possess) no awareness of the swarm state.
[caption id="attachment_9625" align="alignnone" width="600"]
Figure 1: A four-node swarm cluster running two container services on isolated overlay networks.[/caption]
Figure 1 offers a simple visualization of a four-node cluster running in swarm mode, leveraging the overlay network driver. In this swarm, Host A is the manager node and Hosts B-D are worker nodes. Together, these manager and worker nodes are running two Docker services which are backed by a total of ten container instances, or “replicas.” The yellow in this figure distinguishes the first service, Service 1 ; the containers for Service 1 are connected by an overlay network. Similarly, the blue in this figure represents the second service, Service 2 ; the containers for Service 2 are also attached by an overlay network.
Under the covers, Swarm and overlay are enabled by enhancements to the Host Network Service (HNS) and Windows libnetwork plugin for the Docker engine, which leverage the Azure Virtual Filtering Platform (VFP) forwarding extension in the Hyper-V Virtual Switch. Figure 2 shows how these components work together on a given Windows container host, to enable overlay and swarm mode functionality.
[caption id="attachment_9655" align="alignnone" width="550"]
Figure 2: Key components involved in enabling swarm mode and overlay networking on Windows container hosts.[/caption]
Overlay networking was enabled with the addition of an overlay network driver plugin to the HNS service, which creates encapsulation rules using the VFP forwarding extension in the Hyper-V Virtual Switch; the HNS overlay plugin communicates with the VFP forwarding extension to perform the VXLAN encapsulation required to enable overlay networking functionality.
Overlay networking support was the main hurdle that needed to be overcome to achieve Docker swarm mode support on Windows. Aside from that, additions also needed to be made to the Windows libnetwork Plugin—the plugin to the Docker engine that enables container networking functionality on Windows by facilitating communication between the Docker engine and the HNS service.
Currently, Windows supports DNS Round-Robin load balancing between services. The routing mesh for Windows Docker hosts is not yet supported, but will be coming soon. Users seeking an alternative load balancing strategy today can setup an external load balancer (e.g. NGINX) and use Swarm’s publish-port mode to expose container host ports over which to load balance.
Together, Docker Swarm and support for overlay container networks enable multi-host scenarios and rapid scalability of your Windows containerized applications and services. This new support, combined with service discovery and the rest of the capabilities that you are used to leveraging in single-host configurations, makes for a clean and straight-forward experience developing containerized apps on Windows for multi-host environments.
To get started with Docker Swarm and overlay networking on Windows, start here .
The Datacenter and Cloud Networking team worked alongside our partners internally and at Docker to bring overlay networking mode and Docker swarm mode support to Windows. Again, this is an exciting milestone in our ongoing work to achieve better container networking support to Windows users. We’re constantly seeking more ways to improve your experience working with containers on Windows, and it’s only with your feedback that we can best decide what to do next to enable you and your DevOps teams.
We encourage you to share your experiences, questions and feedback with us, to help us learn more about what you’re doing with container networking on Windows today, and to understand what you’d like to achieve in the future. Visit our Contact Page to learn more about the forums that you can use to be in touch with us.
Windows 10 Insiders can now take advantage of overlay networking and Docker swarm mode to manage containerized applications in both single-host and clustering scenarios.
Containers are a rapidly growing technology, and as they evolve so must the technologies that support them as members of a broader collection of compute, storage and networking infrastructure components. For networking, in particular, this means continually striving to achieve better connectivity, higher reliability and easier management for container networking. Less than six months ago, Microsoft released Windows 10 Anniversary Edition and Windows Server 2016, and even as our first versions of Windows with container support were being celebrated we were already hard at work on new container features, including several container networking features.
Our last Windows release showcased Docker Compose and service discovery —two key features for single-host container deployment and networking scenarios. Now, we’re expanding the reach of Windows container networking to multi-host (clustering) scenarios with the addition of a native overlay network driver and support for Docker swarm mode , available today to Windows Insiders as part of the upcoming Windows 10, Creators Update.
Docker swarm mode is Docker’s native orchestration tool, designed to simplify the experiencing of declaring, managing and scaling container services. The Windows overlay network driver (which uses VXLAN and virtual overlay networking technology) makes it possible to connect container endpoints running on separate hosts to the same, isolated network. Together, swarm mode and overlay enable easy management and complete scalability of your containerized applications, allowing you to leverage the full power of your infrastructure hosts.
What is “swarm mode”?
Swarm mode is a Docker feature that provides built in container orchestration capabilities, including native clustering of Docker hosts and scheduling of container workloads. A group of Docker hosts form a “swarm” cluster when their Docker engines are running together in “swarm mode.”
A swarm is composed of two types of container hosts: manager nodes, and worker nodes . Every swarm is initialized via a manager node, and all Docker CLI commands for controlling and monitoring a swarm must be executed from one of its manager nodes. Manager nodes can be thought of as “keepers” of the Swarm state—together, they form a consensus group that maintains awareness of the state of services running on the swarm, and it’s their job to ensure that the swarm’s actual state always matches its intended state , as defined by the developer or admin.
Note: Any given swarm can have multiple manager nodes, but it must always have at least one .
Worker nodes are orchestrated by Docker swarm via manager nodes. To join a swarm, a worker node must use a “join token” that was generated by the manager node when the swarm was initialized. Worker nodes simply receive and execute tasks from manager nodes, and so they require (and possess) no awareness of the swarm state.
[caption id="attachment_9625" align="alignnone" width="600"]
Figure 1: A four-node swarm cluster running two container services on isolated overlay networks.[/caption]
Figure 1 offers a simple visualization of a four-node cluster running in swarm mode, leveraging the overlay network driver. In this swarm, Host A is the manager node and Hosts B-D are worker nodes. Together, these manager and worker nodes are running two Docker services which are backed by a total of ten container instances, or “replicas.” The yellow in this figure distinguishes the first service, Service 1 ; the containers for Service 1 are connected by an overlay network. Similarly, the blue in this figure represents the second service, Service 2 ; the containers for Service 2 are also attached by an overlay network.
Note: In this case, the two Docker services happen to be connected by separate/isolated overlay networks. It is also possible, however, for multiple container services to be attached to the same overlay network.
Windows Network Stack Implementation
Under the covers, Swarm and overlay are enabled by enhancements to the Host Network Service (HNS) and Windows libnetwork plugin for the Docker engine, which leverage the Azure Virtual Filtering Platform (VFP) forwarding extension in the Hyper-V Virtual Switch. Figure 2 shows how these components work together on a given Windows container host, to enable overlay and swarm mode functionality.
[caption id="attachment_9655" align="alignnone" width="550"]
Figure 2: Key components involved in enabling swarm mode and overlay networking on Windows container hosts.[/caption]
The HNS overlay network driver plugin and VFP forwarding extension
Overlay networking was enabled with the addition of an overlay network driver plugin to the HNS service, which creates encapsulation rules using the VFP forwarding extension in the Hyper-V Virtual Switch; the HNS overlay plugin communicates with the VFP forwarding extension to perform the VXLAN encapsulation required to enable overlay networking functionality.
On Windows, the Azure Virtual Filtering Platform (VFP) is a software defined networking (SDN) element, installed as a programmable Hyper-V Virtual Switch forwarding extension. It is a shared component with the Azure platform, and was added to Windows 10 with Windows 10 Anniversary Edition. It is designed as a high performance, rule-flow based engine, to specify per-endpoint rules for forwarding, transforming, or blocking network traffic. The VFP extension has been used for implementing the l2bridge and l2tunnel Windows container networking modes and is now also used to implement the overlay networking mode. As we continue to expand container networking capabilities on Windows, we plan to further leverage the VFP extension to enable more fine-grained policy.
Enhancements to the Windows libnetwork plugin
Overlay networking support was the main hurdle that needed to be overcome to achieve Docker swarm mode support on Windows. Aside from that, additions also needed to be made to the Windows libnetwork Plugin—the plugin to the Docker engine that enables container networking functionality on Windows by facilitating communication between the Docker engine and the HNS service.
Load balancing: Windows routing mesh coming soon
Currently, Windows supports DNS Round-Robin load balancing between services. The routing mesh for Windows Docker hosts is not yet supported, but will be coming soon. Users seeking an alternative load balancing strategy today can setup an external load balancer (e.g. NGINX) and use Swarm’s publish-port mode to expose container host ports over which to load balance.
Boost your DevOps cycle and manage containers across Windows hosts by leveraging Docker swarm mode today
Together, Docker Swarm and support for overlay container networks enable multi-host scenarios and rapid scalability of your Windows containerized applications and services. This new support, combined with service discovery and the rest of the capabilities that you are used to leveraging in single-host configurations, makes for a clean and straight-forward experience developing containerized apps on Windows for multi-host environments.
To get started with Docker Swarm and overlay networking on Windows, start here .
The Datacenter and Cloud Networking team worked alongside our partners internally and at Docker to bring overlay networking mode and Docker swarm mode support to Windows. Again, this is an exciting milestone in our ongoing work to achieve better container networking support to Windows users. We’re constantly seeking more ways to improve your experience working with containers on Windows, and it’s only with your feedback that we can best decide what to do next to enable you and your DevOps teams.
We encourage you to share your experiences, questions and feedback with us, to help us learn more about what you’re doing with container networking on Windows today, and to understand what you’d like to achieve in the future. Visit our Contact Page to learn more about the forums that you can use to be in touch with us.
Updated Mar 22, 2019
Version 2.0
![](\"https://techcommunity.microsoft.com/t5/image/serverpage/image-id/97079iC73A0095DBFCAA1D\")
\n Figure 1: A four-node swarm cluster running two container services on isolated overlay networks.[/caption]\n ![](\"https://techcommunity.microsoft.com/t5/image/serverpage/image-id/97080i93852552B45239BB\")
\n Figure 2: Key components involved in enabling swarm mode and overlay networking on Windows container hosts.[/caption]\n