Windows Calico on Azure Kubernetes Services Public Preview

[Keiko Harada]

Last year, Microsoft collaborated with Tigera and released Project Calico to the Windows platform. This contribution to the open-source community was very well received by the community and we got many requests from customers to also support Calico for Windows server on Azure Kubernetes Service (AKS). I’m pleased to announce that Calico for Windows is now available in public preview on AKS.

We have started seeing customers with Linux and Windows operating system (OS) node pools on a single cluster. With more customers adopting containers and microservices, customers are developing applications that runs across Linux and Windows containers. With our support for Calico on Windows in AKS, customer can now have a single network policy experience on node pool cluster running both Linux and Windows containers to secure their network traffic.

In this blog post, I’ll walk you through the steps to install AKS with Calico and setup network policy.

Prerequisites:

• Kubernetes version 1.20.2 or above.
• Bash environment in Azure Cloud Shell (bash)
• Ability to create AKS clusters
• Knowledge on Kubernetes and kubectl

Overview:

• Create AKS cluster with Calico enabled with Direct Server Return (DSR)
• Setup polices for Windows and Linux
• Test the network policy

Step 1: Set up cluster with Calico enabled

We have simplified the process of enabling Calico on a cluster by including it as part of an addon to AKS cluster creation.

Step 1.1: Set the preview feature flag for Calico feature on windows

To use Calico with Windows node pools, you also need to register the “Microsoft.ContainerService/EnableAKSWindowsCalico” flag since it is a public preview feature.

Register the EnableAKSWindowsCalico feature flag by following the command as shown in the following example:

$ az feature register --namespace "Microsoft.ContainerService" --name "EnableAKSWindowsCalico"

You can check on the registration status using running this command below:

$ az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnableAKSWindowsCalico')].{Name:name,State:properties.state}"

When ready, refresh the registration of the Microsoft.ContainerService resource provider using the following command:

$ az provider register --namespace Microsoft.ContainerService

Step 1.2: Create AKS cluster with Calico Addons

Here, we will create an AKS cluster with Calico enabled. To enable Calico network policy on Windows, the network plugin must be "azure" since Windows on AKS supports Azure CNI network plug-in only. 

Note: When setting up your Windows node pools to your cluster, it is required to add the windows-admin-username and windows-admin-password parameters as you see in the example below.

$ rg=MyResourceGroup
$ location=MyRegion
$ PASSWORD_WIN="MyPassword"

$ az group create -n $rg -l $location

$ az aks create \
    --resource-group $rg \
    --name k8s \
    --node-count 1 \
    --enable-addons monitoring \
    --windows-admin-username azureuser \
    --windows-admin-password $PASSWORD_WIN \
    --kubernetes-version 1.20.2 \
    --vm-set-type VirtualMachineScaleSets \
    --load-balancer-sku standard \
    --network-plugin azure \
    --node-vm-size Standard_D2s_v3 \
    --network-policy calico

$ az aks nodepool add \
    --resource-group $rg \
    --cluster-name k8s \
    --os-type Windows \
    --name npwin \
    --node-count 1 \
    --kubernetes-version 1.20.2 \
    --node-vm-size Standard_D2s_v3

When Calico is enabled, you should be able to see the calico system pods running inside the calico-system namespace. To connect to AKS, please refer to connect to cluster doc and run the following command.

$ kubectl get pods -n calico-system
NAME                                   		READY   STATUS    RESTARTS   AGE
calico-kube-controllers-6b4fc665cc-6pmgq   	1/1     Running   0          6d
calico-node-474qb                         	1/1     Running   0          6d
calico-typha-5889b7f49d-9bblq              	1/1     Running   0          6d

Step 2: Setup policies on Windows and Linux

Once the cluster is created, we will create client and server pods on Linux and Windows nodes and verify connectivity between the pods.

Step 2.1: Create Linux Pod

Run the following command which includes the yaml file.

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: calico-demo
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: busybox
  name: busybox
  namespace: calico-demo
spec:
  containers:
  - args:
    - /bin/sh
    - -c
    - sleep 360000
    image: busybox:1.28
    imagePullPolicy: Always
    name: busybox
  nodeSelector:
    beta.kubernetes.io/os: linux
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: calico-demo
spec:
  containers:
  - name: nginx
    image: nginx:1.8
    ports:
    - containerPort: 80
  nodeSelector:
    beta.kubernetes.io/os: linux
EOF

Step 2.2: Create Windows Pod

We’ll create a client (powershell) and server (porter) pod on the Windows nodes.

Note: AKS uses Windows Server 2019 as the host OS version only. Container images build using other Windows Server versions are not support. (mcr.microsoft.com/windows/servercore:1809)

• Create powershell pod 

Run the following command which includes the yaml file.

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: pwsh
  namespace: calico-demo
  labels:
    app: pwsh
spec:
  containers:
  - name: pwsh
    image: mcr.microsoft.com/windows/servercore:1809
    args:
    - powershell.exe
    - -Command
    - "Start-Sleep 360000"
    imagePullPolicy: IfNotPresent
  nodeSelector:
    kubernetes.io/os: windows
EOF

• Create the porter server pod

Run the following command which includes the yaml file.

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: porter
  namespace: calico-demo
  labels:
    app: porter
spec:
  containers:
  - name: porter
    image: calico/porter:1809
    ports:
    - containerPort: 80
    env:
    - name: SERVE_PORT_80
      value: This is a Calico for Windows on AKS.
    imagePullPolicy: IfNotPresent
  nodeSelector:
    kubernetes.io/os: windows
EOF

You should be able to see all the pods running in calico-demo namespace. Make sure pods are all up and running.

$ kubectl get pod -n calico-demo --watch
NAME      	READY   STATUS    RESTARTS   AGE
busybox   	1/1     Running   1          5d19h
nginx     	1/1     Running   0          5d19h
porter    	1/1     Running   0          5d19h
pwsh      	1/1     Running   1          5d19h

Step 2.3: Verify the connectivity between pods

• Verify that the busybox pod can reach the porter pod on port 80.

$ kubectl exec -n calico-demo busybox -- nc -vz $(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') 80

• If the connection from the busybox pod to the porter pod succeeds, we will get output like the following:

192.168.40.166 (192.168.40.166:80) open

• Verify that the powershell pod can reach the nginx pod.

$ kubectl exec -n calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po nginx -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5

• If the connection succeeds, we will get output like:

RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Accept-Ranges: bytes
                    Content-Length: 612
                    Content-Type: text/html
                    Date: Mon, 01 Mar 2021 19:00:39 GMT
                    ETag: "56a78fbf-264"
                    Last-Modified: Tue, 26 Jan 2016 ...
Forms             :
Headers           : {[Connection, keep-alive], [Accept-Ranges, bytes],
                    [Content-Length, 612], [Content-Type, text/html]...}
Images            : {}
InputFields     : {}
Links             : {@{outerHTML=<a href="http://nginx.org/">nginx.org</a>;
                    tagName=A; href=http://nginx.org/}, @{outerHTML=<a
                    href="http://nginx.com/">nginx.com</a>; tagName=A;
                    href=http://nginx.com/}}
ParsedHtml        :
RawContentLength  : 612	 

Step 3: Test the Network Policy

Now, we’ll apply a basic network policy to isolate pod traffic.

Step 3.1: Run the Network Policy

In this example, we will only allow busybox to communicate.

$ kubectl apply -f  - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-busybox
  namespace: calico-demo
spec:
  podSelector:
    matchLabels:
      app: 'porter'
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: busybox
EOF

Note: If you want to use calico api, calicoctl is required. It is not available in Azure Cloud Shell by default. You would need to download calicoctl on Azure Cloud Shell. For more information on using calicoctl, please refer to calico docs.

Step 3.2: Validate whether the policy works

• Verify that the busybox pod is still able to reach the porter pod

$ kubectl exec -n calico-demo busybox -- nc -vz $(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') 80

• Verify that the powershell pod is not able to reach the porter pod

$ kubectl exec -n calico-demo pwsh -- powershell Invoke-WebRequest -Uri http://$(kubectl get po porter -n calico-demo -o 'jsonpath={.status.podIP}') -UseBasicParsing -TimeoutSec 5 

The request times out with a message like:

Invoke-WebRequest : The operation has timed out.
At line:1 char:1
+ Invoke-WebRequest -Uri http://10.240.0.122 -UseBasicParsing -TimeoutS ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:Htt
   pWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe
   ll.Commands.InvokeWebRequestCommand

command terminated with exit code 1 

Conclusion:

Hope you found the following steps helpful in getting started with Calico for Windows on AKS. Give this feature a try and let us know what you think and how we can improve it.

For more details, please see below:

• Network Policy on Azure Kubernetes Services Documentation
• If you have any questions or concerns about AKS, please go to AKS github
• If you like to check details about project calico with Windows

[James van den Berg]

Thank you @Keiko Harada for Sharing with the Community 
Great Blogpost!