Container Networking with Azure Application Gateway for Containers (AGC): Overlay vs. Flat AKS

Architecture at a glance

Diagram: “AGC with AKS Networking Models (Overlay & Flat)” – attached below.

• Left: internet ↦ AGC Frontend (Public/Private)
• Middle: AGC (Gateway) ↦ AGC Subnet (/24) ↦ AKS Node Subnet(s)
• Right (two options):
  • Overlay: Pods use Overlay CIDR (separate routing domain). AGC can proxy directly to pods via the overlay routing extension.
  • Flat: Pods get VNet‑routable IPs (Azure CNI Pod/Node Subnet). AGC forwards to pod/service IPs inside the VNet.
• Security & Ops: NSGs, Azure Firewall, Kubernetes Network Policies (Azure NP, Calico, Cilium), TLS, Observability.

Key facts you can bank on

• AGC auto‑detects whether your cluster runs CNI Overlay or CNI—you don’t change your Gateway/Ingress specs just to switch models .
• For Overlay, AGC extends the routing domain to the AGC subnet so it can reach overlay pod CIDRs directly .
• Network policies (Azure NP, Calico, Cilium) are supported with AGC .
• Subnet for AGC must be /24 and only one AGC deployment per subnet.
• ALB Controller (the AGC Kubernetes controller) must be v1.7.9+ for Overlay support .
• AGC overview and resource model: what it is, where it fits.

Why AGC for AKS?

• Layer‑7 intelligence: Host/path/header/Query‑string routing, redirects, rewrites, mTLS to backends.
• Near real‑time convergence when pods/services change (scale‑out, rollouts) so ingress keeps pace with GitOps/CI [community coverage aligns with Learn guidance].
• First‑class Azure: NSGs, Azure Firewall, Private Link, zones, managed identity—plus policy/monitoring integration.
• Gateway API OR Ingress: adopt Gateway API incrementally while still supporting existing Ingress resources.

The two AKS networking models (and what AGC does)

1) Azure CNI Overlay (Overlay networking)

• When to choose: Conserve VNet IPs, run very large clusters, keep IP management simple.
• How it works: Pods get IPs from overlay CIDR (not from VNet). Azure builds a separate routing domain for pod CIDRs; AGC extends that domain to its subnet to proxy directly to pods .
• Notes: Requires ALB Controller v1.7.9+; policies (Azure NP, Calico, Cilium) are supported .

2) Flat networking (Azure CNI Pod/Node Subnet)

• When to choose: You need direct pod reachability from on‑prem or peered VNets; VNet IP space is ample.
• How it works: Pods get VNet‑routable IPs. AGC forwards traffic into the VNet directly.
• Trade‑off: Plan your subnets carefully to avoid IP exhaustion.

Good news: In both models, you don’t change your Gateway/Ingress specs just because the underlay is different. AGC handles it.

Quick start with Gateway API

Ensure you’ve installed and configured the ALB Controller for AGC in your AKS cluster (v1.7.9+ recommended for Overlay) and created the AGC subnet (/24)

1. Create a simple app (Deployment + Service):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  labels: { app: hello }
spec:
  replicas: 3
  selector: { matchLabels: { app: hello } }
  template:
    metadata: { labels: { app: hello } }
    spec:
      containers:
        - name: web
          image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
          ports: [{ containerPort: 80 }]
---
apiVersion: v1
kind: Service
metadata:
  name: hello
spec:
  selector: { app: hello }
  ports:
    - port: 80
      targetPort: 80

2) Define the GatewayClass and Gateway for AGC:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: azure-alb
spec:
  controllerName: alb.networking.azure.io/gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: web-gw
  namespace: default
spec:
  gatewayClassName: azure-alb
  listeners:
    - name: http
      protocol: HTTP
      port: 80

Tip: The controllerName can differ by release. If your cluster already ships an azure-albGatewayClass, reuse it and skip creating your own. Validate with:

kubectl get gatewayclasses -o wide

3) Route traffic with HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: hello-route
spec:
  parentRefs:
    - name: web-gw
  hostnames:
    - "hello.example.com"   # replace with your DNS name
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: hello
          port: 80

4) (Optional) Add basic header rewrite & redirect policies
(illustrative—check the AGC policy CRDs supported by your controller version):

# Example: redirect /old to / (HTTP 301)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: hello-redirect
spec:
  parentRefs:
    - name: web-gw
  hostnames: ["hello.example.com"]
  rules:
    - matches:
        - path: { type: PathPrefix, value: "/old" }
      filters:
        - type: RequestRedirect
          requestRedirect:
            statusCode: 301
            path: { type: ReplacePrefixMatch, replacePrefixMatch: "/" }
      backendRefs:
        - name: hello
          port: 80

Deploy:

kubectl apply -f hello.yaml
kubectl apply -f gateway.yaml
kubectl apply -f route.yaml

When the Gateway becomes Accepted=True and the listeners are programmed, you’ll see a public/private IP on the AGC frontend. Point your DNS (hello.example.com) to that IP and you’re live.

Subnetting, peering & version

• AGC Subnet must be /24, and only one AGC per subnet .
• Don’t place AGC in VNet region A while AKS nodes are in region B, or mix VNets via regional/global peering—those placements aren’t supported for the AGC‑to‑AKS path .
• For Overlay, confirm ALB Controller ≥ 1.7.9.
• Keep network policies consistent across namespaces; AGC respects your chosen NP engine (Azure NP, Calico, Cilium).