Application gateway for Containers vs Application Gateway Ingress Controller - What's changed?
Application Gateway for Containers is a new application (layer 7) load balancing and dynamic traffic management product for workloads running in a Kubernetes cluster. At the time of writing this service is currently in public preview. In this article we will look at the differences between AGIC and Application Gateway for containers and some of the great new features available through this new offering.
Application Gateway for Containers is the evolution of the Application Gateway Ingress Controller (AGIC). AGIC was created to allow customers to leverage Azures native Application Gateway to expose Kubernetes Services externally. AGIC continuously updates the Application Gateway so that the resources specified through Ingress resources on the cluster are exposed. Application gateway for Containers also syncs your application gateway resource to your Kubernetes services however with a much richer feature set. Let's take a look at what is new.
Kubernetes Gateway API vs Ingress
One of the key differences between AGIC and Application Gateway for Containers is native support for the Kubernetes Gateway API. The Gateway API was formally announced in 2021 as the evolution of traditional Kubernetes ingress. If you would like to read about why this Gateway was created please refer to this Kubernetes blog: https://kubernetes.io/blog/2021/04/22/evolving-kubernetes-networking-with-the-gateway-api/.
Application Gateway for Containers still supports the ingress resource but also allows customers to make the most of the feature rich Gateway API meaning customers can use a combination of both Azure native and Kubernetes native routing to perform actions like multi site hosting or end to end TLS that otherwise would have required complex annotations with ingress controllers such as NGINX or Traefik.
Architecture
As we can see from the architecture below the Application Gateway for containers is very similar to AGIC.
When deploying Azure Application Gateway for Containers you will need to install the ALB controller on your cluster. Currently unlike AGIC which also has a Kubernetes add on ALB Controller can only be installed using HELM. Similarly to AGIC the only supported networking plugin in Azure CNI.
Although the architecture is similar there are some additional components to provision and manage when using Application Gateway for Containers with the Gateway API.
Core Components
- Control Plane - Application Gateway for containers is an Azure parent resource that deploys the control plane. The control plane orchestrates the proxy configuration based on customer requirements.
- Frontend - An Azure child resource. The frontend defines the entry point client traffic should be received by a given application gateway for containers. A frontend can only be associated to one Application Gateway for Containers. Each frontend provides a unique FQDN which can be referenced by a CNAME record.
- Associations - An association defines a connection point into a virtual network. An association is a 1:1 mapping of an association resource to an Azure Subnet that has been delegated for use by the application gateway. During the creation of an association the underlying data plane is provisioned and connected to a subnet within the defined VNET's subnet.
- ALB Controller - The ALB is the orchestrator for the configuration of the Application Gateway for Containers Azure resource and watches for CRD's and resource configurations such as Gateways or Ingress'. The ALB is installed via HELM and has two pods that run once called the "ALB Controller Pod" which is responsible for orchestrating the configuration changes back to the Gateway and the "ALB Controller Boostrap Pod" which is responsible for management of the CRD's.
Deployments
There are also two different deployment options which differ from the options available with AGIC:
- Bring your own Deployment - The bring your own deployment option is great for customers looking to have independent resource lifecycles and granular control over provisioning. In this deployment strategy, deployment and lifecycle of the Application Gateway for Containers resource, Association and Frontend resource is assumed via Azure portal or your chosen IaC solution. If using this method and Gateway API resources then the Front end resource should be provisioned in Azure prior to the Gateway Resource creation on the cluster.
- Managed by ALB Controller - The managed solution is a brilliant option for customers looking to centralise management of their Application Gateway for Containers resources as the ALB Controller itself is responsible for the lifecycle of the resources relating to it including the Application Gateway for containers resource in Azure which the ALB will create itself when an ApplicationLoadBalancer CRD is defined on the cluster. Contrasting this to the BYO deployment method for the Gateway API every time a Gateway resource is created referencing the Application Load Balancer resource the ALB controller provisions a new front end resource in Azure and manages its lifecycle based on the lifecycle of the corresponding gateway resource. For an example please view the code snippets below:
Application Load Balancer Resource
apiVersion: alb.networking.azure.io/v1
kind: ApplicationLoadBalancer
metadata:
name: alb-test
namespace: alb-test-infra
spec:
associations:
- $ALB_SUBNET_ID
Gateway Resource with ALB Reference
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: gateway-01
namespace: test-infra
annotations:
alb.networking.azure.io/alb-namespace: alb-test-infra
alb.networking.azure.io/alb-name: alb-test
spec:
gatewayClassName: azure-alb-external
listeners:
- name: https-listener
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- kind : Secret
group: ""
name: frontend.com
Comparison Table - Application Gateway for Containers vs Application Gateway Ingress Controller
| Application Gateway Ingress Controller |
Application Gateway for Containers |
|
| Support for Ingress | ✓ | ✓ |
| TLS Termination | ✓ | ✓ |
| End to End TLS | ✓ | ✓ |
| Path based routing | ✓ | ✓ |
| Query String Match | X | ✓ |
| Header based routing | X | ✓ |
| Method based routing | X | ✓ |
| Support for Gateway API | X | ✓ |
| MTLS | X | ✓ |
| Traffic Splitting | X | ✓ |
| Automatic Retries | X | ✓ |
There are some additional benefits to using Application Gateway for containers when compared to AGIC such as the ability once GA to scale past the existing limits with AGIC (1400 Pods 100 Listeners) and also improved performance relating to the real time updating of the application gateway as new routes are created within your cluster. I plan on releasing a follow up blog once GA to review metrics relating to these performance increases.
Availability
Currently this service is available in the following regions:
- Australia East
- Central US
- East Asia
- East US
- East US2
- North Central US
- North Europe
- South Central US
- Southeast Asia
- UK South
- West US
- West Europe
Final Thoughts
Application Gateway for Containers opens the door to many exciting Azure native use cases for managing inbound traffic for public services hosted in AKS. Internal or private services are not suitable for either AGIC or Application gateway for Containers as fully private V2 Application Gateways are not yet GA however are in preview so this may be something that changes in the future. For private service load balancing using ingress with the private lb annotations would still be the recommended solution. As with all Azure Preview features supported is limited so running in production is not advised. With all that being said once Application Gateway for Containers becomes generally available it will certainly reduce the operational complexity of performing complex routing on AKS as third party services will not need to be installed and managed on the cluster. The potential for performance and scaling increases also open the door to use cases that were not previously possible with AGIC.
The ability to manage both Ingress and Gateway resources with Application Gateway for Containers will provide an easy transition in the future (Once GA) for customers currently using AGIC.
If you want to learn more about how Application Gateway for Containers from how to deploy to how requests are accepted and routed please take a look at some of the links below:
- What is Application Gateway for containers?
- QuickStart to deploy Application Gateway for Containers
Backend MTLS with Application Gateway for Containers - Gateway API (preview) | Microsoft Learn