Securing Microservices with Cilium and Istio

The adoption of Kubernetes and containerized applications is booming, leading to new challenges in visibility and security. As the landscape of cloud-native applications is rapidly evolving so are the number of sophisticated attacks targeting containerized workloads. Traditional tools often fall short in tracking the usage and traffic flows within these applications. The immutable nature of container images and the short lifespan of containers further necessitate addressing vulnerabilities early in the delivery pipeline.

Comprehensive Security Controls in Kubernetes

Microsoft Azure offers a range of security controls to ensure comprehensive protection across various layers of the Kubernetes environment. These controls include but are not limited to:

• Cluster Security: Features such as private clusters, managed cluster identity, and API server authorized ranges enhance security at the cluster level.
• Node and Pod Security: Hardened bootstrapping, confidential nodes, and pod sandboxing are implemented to secure the nodes and pods within a cluster.
• Network Security: Advanced Container Networking Services and Cilium Network policies offer granular control over network traffic.
• Authentication and Authorization: Azure Policy in-cluster enforcement, Entra authentication, and Istio mTLS and authorization policies provide robust identity and access management.
• Image Scanning: Microsoft Defender for Cloud provides both image and runtime scanning to identify vulnerabilities and threats.

Let’s highlight how you can secure micro services while scaling your applications running on Azure Kubernetes Service (AKS) using service mesh for robust traffic management, and network policies for security.

Micro segmentation with Network Policies

Micro segmentation is crucial for enhancing security within Kubernetes clusters, allowing for the isolation of workloads and controlled traffic between microservices. Azure CNI by Cilium leverages eBPF to provide high-performance networking, security, and observability features. It dynamically inserts eBPF bytecode into the Linux kernel, offering efficient and flexible control over network traffic. Cilium Network Policies enable network isolation within and across Kubernetes clusters. 

Namespace Scoped Network Policies using Azure CNI by Cilium

Cilium also provides an identity-based security model, offering Layer 7 (L7) traffic control, and integrates deep observability for L4 to L7 metrics in Kubernetes clusters. A significant advantage of using Azure CNI based on Cilium is its seamless integration with existing AKS environments, requiring minimal modifications to your infrastructure. Note that Cilium Clusterwide Network Policy (CCNP) is not supported at the time of writing this blog post.

FQDN Filtering with Advanced Container Networking Services (ACNS)

Traditional IP-based policies can be cumbersome to maintain. ACNS allows for DNS-based policies, providing a more granular and user-friendly approach to managing network traffic. 

ACNS/Cilium FQDN Policy Flow

This is supported only with Azure CNI powered by Cilium and includes a security agent DNS proxy for FQDN resolution even during upgrades. 

It’s worth noting that with Cilium’s L7 enforcement, you can control traffic based on HTTP methods, paths, and headers, making it ideal for APIs, microservices, and services that use protocols like HTTP, gRPC, or Kafka. At the time of writing this blog, this capability is not supported via ACNS. More on this in a future blog!

AKS Istio Add-On: Mutual TLS (mTLS) and Authorization Policy

Istio enhances the security of microservices through its built-in features, including mutual TLS (mTLS) and authorization policies. The Istiod control plane, acting as a certificate authority, issues X.509 certificates to the Envoy sidecar proxies via the Secret Discovery Service (SDS). Integration with Azure Key Vault allows for secure management of root and intermediate certificates.

The PeerAuthentication Custom Resource in Istio controls the traffic accepted by workloads. By default, it is set to PERMISSIVE to facilitate migration but can be set to STRICT to enforce mTLS across the mesh. Istio also supports granular authorization policies, allowing for control over IP blocks, namespaces, service accounts, request paths, methods, and headers.

The Istio add-on also supports integration with Azure Key Vault (AKV) and the AKV Secrets Store CSI Driver Add-On for plug-in CA certificates, where the root CA lives on a secure machine offline, and the intermediate certs for the Istiod control plane are synced to the cluster by the CSI Driver Add-On. Additionally, certificates for the Istio ingress gateway for TLS termination or SNI passthrough can also be stored in AKV.

Defense-In-Depth with Cilium, ACNS and Istio

Combining the capabilities of Cilium's eBPF technologies through ACNS and AKS managed Istio addon, AKS provides a defense-in-depth strategy for securing Kubernetes clusters. Azure CNI's Cilium Network Policies and ACNS FQDN filtering enforce Pod-to-Pod and Pod-to-egress policies at Layer 3 and 4, while Istio enforces STRICT mTLS and Layer 7 authorization policies. This multi-layered approach ensures comprehensive security coverage across all layers of the stack.

Istio and Cilium better together

Now, let’s highlight the key steps in achieving this:

Step 1: Create an AKS Cluster with Azure CNI (by Cilium), ACNS and Istio Addon enabled.

az aks create \
  --resource-group $RESOURCE_GROUP \
  --name $CLUSTER_NAME \
  --location $LOCATION \
  --kubernetes-version 1.30.0 \
  --node-count 3 \
  --node-vm-size standard_d16_v3 \
  --enable-managed-identity \
  --network-plugin azure \
  --network-dataplane cilium \
  --network-plugin-mode overlay \
  --pod-cidr 192.168.0.0/16 \
  --enable-asm \
  --enable-acns \
  --generate-ssh-keys

Step 2: Create Cilium FQDN policy that allows egress traffic to google.com while blocking traffic to httpbin.org.
Sample Policy (fqdn-filtering-policy.yaml):

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: sleep-network-policy
  namespace: foo
spec:
  endpointSelector:
    matchLabels:
      app: sleep
  egress:
    - toFQDNs:
        - matchPattern: "*.google.com"
    - toEndpoints:
      - matchLabels:
          "k8s:io.kubernetes.pod.namespace": foo
          "k8s:app": helloworld
    - toEndpoints:
      - matchLabels:
          "k8s:io.kubernetes.pod.namespace": kube-system
          "k8s:k8s-app": kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY

Apply policy: kubectl apply -f fqdn-filtering-policy.yaml

Step 3: Create an Istio deny-by-default AuthorizationPolicy. This denies all requests across the mesh unless specifically authorized with an “ALLOW” policy.

Sample Policy (istio-deny-all-authz.yaml):

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-nothing
  namespace: aks-istio-system
spec:
  {}

Apply policy: kubectl apply -f istio-deny-all-authz.yaml

Step 4: Deploy an Istio L7 AuthorizationPolicy to explicitly allow traffic to the “sample” pod in namespace foo for http “GET” requests.

Sample Policy (istio-L7-allow-policy.yaml):

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-get-requests
  namespace: foo
spec:
  selector:
    matchLabels:
      app: sample
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: [“GET”]

Apply policy: kubectl apply -f istio-L7-allow-policy.yaml

Step 5: Deploy an Istio strict mTLS PeerAuthentication Resource to enforce that all workloads in the mesh only accept Istio mTLS traffic.

Sample PeerAuthentication (istio-peerauth.yaml):

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: strict-mtls
  namespace: aks-istio-system
spec:
  mtls:
    mode: STRICT

Apply policy: kubectl apply -f istio-peerauth.yaml

These examples demonstrate how you can manage traffic to specific FQDNs and enforce L7 authorization rules in your AKS cluster.

Conclusion

Traditional IP and perimeter security models are insufficient for the dynamic nature of cloud-native environments. More sophisticated security mechanisms, such as identity-based policies and DNS names, are required. Azure CNI, powered by Cilium and ACNS, provides robust FQDN filtering and Layer 3/4 Network Policy enforcement. The Istio add-on offers mTLS for identity-based encryption and Layer7 authorization policies. A defense-in-depth model, incorporating both Azure CNI and service mesh mechanisms, is recommended for maximizing security posture.

So, give these a try and let us know (Azure Kubernetes Service Roadmap (Public)) how we can evolve our roadmap to help you build the best with Azure.

Credit(s): Niranjan Shankar, Sr. Software Engineer, Microsoft