Choosing the right networking model for Azure Kubernetes Service (AKS): Azure CNI vs. Kubenet

When deploying Azure Kubernetes Service (AKS), one of the most common decisions is choosing between Azure CNI and Kubenet networking models. Each model offers unique benefits and trade-offs, and the right choice depends on your workload requirements, scalability needs, and integration with Azure services.

In this updated post, we’ll not only explore Azure CNI and Kubenet but also dive into the new options available with Azure CNI, such as Azure CNI Overlay, Dynamic IP Allocation, Static Block Allocation, and Cilium-powered Azure CNI.

Understanding AKS networking models

AKS supports two primary networking models:

1. Azure CNI (Container Networking Interface): Provides full integration with Azure Virtual Networks (VNets), making it ideal for enterprise-grade workloads.
2. Kubenet: A simpler, NAT-based networking solution designed for lightweight and cost-sensitive clusters.

Each model manages how pods and nodes communicate within and outside the cluster but differs significantly in IP assignment, routing, and scalability.

Azure CNI variants

Azure CNI now includes several options, each catering to specific needs:

1. Azure CNI (Traditional):
   • Pods receive IP addresses directly from the VNet subnet.
   • Ideal for clusters requiring direct pod-to-resource connectivity.
   • Drawback: Can lead to IP exhaustion for large clusters.
2. Azure CNI with dynamic IP allocation:
   • Pods are assigned IPs from a dedicated subnet for pods.
   • Benefit: Dynamically allocates IPs, conserving VNet address space.
   • Use Case: Large clusters requiring efficient IP management.
3. Azure CNI overlay:
   • Pods receive IPs from a private CIDR range outside the VNet.
   • Benefit: Conserves VNet IP addresses and supports large-scale clusters.
   • Drawback: Adds slight latency due to NAT for external communication.
4. Azure CNI powered by Cilium:
   • Offers advanced features like enhanced network security and observability.
   • Flexible pod IP assignment (VNet subnet or overlay).
   • Use Case: Advanced workloads needing features like encryption, policy enforcement, and dynamic scaling.

Detailed comparison table: Azure CNI vs. Kubenet

1. Azure CNI: Enterprise-grade networking (recommended)

How It Works

• Pod IPs: Pods are assigned IPs directly from the Azure VNet subnet.
• Routing: Pods communicate directly with other pods, VNet resources, peered VNets, and on-premises networks without NAT.
• Advanced features: Azure CNI supports private link, service endpoints, and Azure network policies for enhanced security and integration.

Key Benefits

1. Full VNet integration:
   • Pods can directly access Azure services such as databases, storage accounts, and other VNet resources.
2. Performance:
   • Direct pod-to-VNet communication eliminates the need for NAT, resulting in lower latency and better performance.
3. Scalability:
   • Ideal for production-grade workloads requiring integration with hybrid or multi-cloud environments.

Considerations

• IP address planning:
  • Since each pod consumes a routable IP from the VNet, careful planning is required to ensure sufficient IP space for scaling.
• Larger VNet requirements:
  • Works best in environments with ample IP address space in the VNet.

CNI networking model

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. Kubenet: Lightweight networking

How It Works

• Pod IPs: Pods are assigned private, non-routable IPs from a CIDR block that is not part of the VNet.
• Routing: Pod communication relies on NAT (Network Address Translation) through the node’s NIC.

Key Benefits

1. Simpler IP management:
   • Pods don’t consume IP addresses from the VNet, reducing the need for large subnets.
2. Cost-effective:
   • Easier and cheaper to operate, making it a good fit for dev/test environments or lightweight workloads.
3. Lightweight:
   • Requires less overhead compared to Azure CNI.

Considerations

• Limited VNet access:
  • Pods cannot directly access VNet resources without NAT, which adds latency.
• Performance trade-offs:
  • Network-intensive workloads may suffer from additional latency due to NAT.

Kubenet networking model

 

 

 

 

 

 

 

 

 

 

 

 

Which networking model should you choose?

Azure CNI (traditional):

• Best for small to medium clusters with ample VNet IP space.
• Provides direct connectivity with low latency.

Azure CNI with dynamic IP allocation:

• Ideal for large clusters where efficient IP management is critical.

Azure CNI overlay:

• Suitable for large-scale clusters with limited VNet IP space.
• Best for clusters with internal pod communication.

Azure CNI powered by Cilium:

• For advanced workloads needing enhanced security, encryption, or observability.

Kubenet:

• Lightweight and cost-effective, perfect for dev/test environments or small workloads.

References

• Azure CNI overview
• Kubenet configuration

Conclusion

With the various networking options available in AKS, it’s crucial to align your choice with your cluster's size, scalability requirements, and workload needs. Azure CNI variants offer flexibility, scalability, and advanced networking features, while Kubenet provides a simple, lightweight option for basic clusters.

Resources for AKS enthusiasts

To deepen your understanding of AKS and stay up-to-date with the latest features, check out these valuable resources:

• AKS Newsletter: Stay informed with updates, best practices, and tips from AKS experts. The newsletter is a must-follow for anyone managing or deploying AKS clusters.
• AKS Learning Path: A comprehensive guide to mastering AKS, covering topics from foundational concepts to advanced features. Ideal for both beginners and experienced professionals.