As documented in Use Azure tags in Azure Kubernetes Service (AKS), you can use Azure tags on an AKS cluster to associate its related resources to a given workload or tenant. For some resources, such as a managed data disk created via a persistent volume claim or an Azure Public IP created by a public Kubernetes service, you can also use Kubernetes manifests to set Azure tags. Azure tags are a helpful mechanism to track resource usage and charge back their costs to separate tenants or business units within an organization. This article explains how to set Azure tags for AKS clusters and related resources.
Tags are metadata elements that you apply to your Azure resources. They're key-value pairs that help you identify resources based on settings relevant to your organization. If you want to track the deployment environment for your resources, add a key named Environment. To identify the resources deployed to production, give them a value of Production. Fully formed, the key-value pair becomes Environment = Production. You can apply tags to your Azure resources, resource groups, and subscriptions. For more information on Azure Tags, see Use tags to organize your Azure resources and management hierarchy.
You can use Azure Policy to create tagging rules and conventions for Azure resources, including AKS clusters or its resources. By creating a policy, you avoid the scenario of resources being deployed to your subscription that don't have the expected tags for your organization. Instead of manually applying tags or searching for resources that aren't compliant, you create a policy that automatically applies the needed tags during deployment. Tags can also now be applied to existing resources with the new Modify effect and a remediation task. For more information, see Assign policy definitions for tag compliance.
Before diving into how using Azure tags in AKS, let's see what happens when you set and update Azure tags with AKS clusters and their related resources:
az --version
. If you need to install it or update your version, see Install Azure CLI.{"Key1": "val1", "kEy1": "val2", "key1": "val3"}
results in Key1
and val1
being set.
When you create or update an AKS cluster with the --tags
parameter, the following are assigned the Azure tags that you've specified:
NOTE: Azure Private DNS only supports 15 tags. tag resources.
To create a cluster and assign Azure tags, you can run az aks create with the --tags
parameter, as shown in the following command. Running the command creates a myAKSCluster
in the myResourceGroup
with the tags dept=IT
and costcenter=9999
.
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--tags dept=IT costcenter=9999 \
--generate-ssh-keys
NOTE: To set tags on the initial node pool, the node resource group, the virtual machine scale set, and each virtual machine scale set instance that's associated with the initial node pool, also set the
--nodepool-tags
parameter.
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--tags dept=IT costcenter=9999 \
--nodepool-tags dept=IT costcenter=9999 \
--generate-ssh-keys
IMPORTANT: If you're using existing resources when you're creating a new cluster, such as an IP address or route table,
az aks create
overwrites the set of tags. If you delete that cluster later, any tags set by the cluster will be removed.
You can use the az aks show command to verify that the tags have been properly applied to the cluster and related resources. The cluster tags for myAKSCluster
are shown in the following example:
$ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'
{
"clusterTags": {
"costcenter": "9999",
"dept": "IT"
}
}
To update the tags on an existing cluster, you can run the az aks update command with the --tags
parameter. Running the command updates the myAKSCluster
with the tags team=alpha
and costcenter=1234
.
az aks update \
--resource-group myResourceGroup \
--name myAKSCluster \
--tags team=alpha costcenter=1234
You can use the az aks show command to verify that the tags have been applied to the cluster. For example:
$ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'
{
"clusterTags": {
"costcenter": "1234",
"team": "alpha"
}
}
IMPORTANT: Setting tags on a cluster by using az aks update overwrites the set of tags. For example, if your cluster has the tags
dept=IT
andcostcenter=9999
and you use az aks update with the tagsteam=alpha
andcostcenter=1234
, the new list of tags would beteam=alpha
andcostcenter=1234
.
In the following example, the businessUnit=FastTrack for Azure
and supportedBy=Paolo Salvatori
tags of the AKS cluster were created via Azure Policy, while the IaC=Bicep
and workload=tags
tags were created at creation time via Bicep.
You can apply one or more Azure tags to a new or existing node pool in your AKS cluster. Tags applied to a node pool are applied to each node within the node pool and are persisted through upgrades. Tags are also applied to new nodes that are added to a node pool during scale-out operations. Adding a tag can help with policy tracking or cost estimation tasks.
When you create or update a node pool with the --tags
parameter, the tags that you specify are assigned to the following resources:
To create a node pool with an Azure tag, run az aks nodepool add with the --tags
parameter. Running the following command creates a contosoNodePool
node pool with the tags tenant=contoso
and costcenter=5555
in the myAKSCluster
.
az aks nodepool add \
--resource-group myResourceGroup \
--cluster-name myAKSCluster \
--name contosoNodePool \
--node-count 1 \
--tags tenant=contoso costcenter=5555 \
--no-wait
You can use the az aks show command to verify that the tags have been applied to the contosoNodePool
node pool.
$ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'
[
{
"nodepoolName": "nodepool1",
"tags": null
},
{
"nodepoolName": "contosoNodePool",
"tags": {
"tenant": "contoso",
"costcenter": "5555"
}
}
]
To update a node pool with an Azure tag, you can run the az aks nodepool update command with the --tags
parameter. Running the following command updates the contosoNodePool
node pool with the tags appversion=1.0.0
and costcenter=4444
in the myAKSCluster
, which already has the tags tenant=contoso
and costcenter=5555
.
az aks nodepool update \
--resource-group myResourceGroup \
--cluster-name myAKSCluster \
--name contosoNodePool \
--tags appversion=1.0.0 costcenter=4444 \
--no-wait
IMPORTANT: Setting tags on a node pool by using the az aks nodepool update command overwrites the set of tags. For example, if your node pool has the tags
tenant=contoso
andcostcenter=5555
, and you use the az aks nodepool update command with the tagsappversion=1.0.0
andcostcenter=4444
, the new list of tags would beappversion=1.0.0
andcostcenter=4444
.
You can use the az aks show command to verify that the tags have been appropriately updated on the node pool.
$ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'
[
{
"nodepoolName": "nodepool1",
"tags": null
},
{
"nodepoolName": "contosoNodePool",
"tags": {
"appversion": "1.0.0",
"costcenter": "4444"
}
}
]
In the following example, the osDiskType=ephemeral
and osType=Windows2022
tags of the node pool VMSS were created using the az aks nodepool create command with the --tags
parameter at provisioning time.
You can apply Azure tags to Azure public IPs, managed disks, and Azure files by using a Kubernetes YAML manifest.
The service.beta.kubernetes.io/azure-pip-tags
annotation can be to determines what Azure tags should be applied to the Azure public IP used by a Kubernetes service. The supported format is a=b,c=d,...
. After updated, the old user-assigned tags would not be replaced by the new ones.
For example, the following YAML file creates a Kubernetes deployment that uses NGINX container image, and exposes it via a public LoadBalancer Kubernetes service. The service definition uses the costcenter=1234,tenant=Contoso
annotation to set Azure tags on the Azure public IP resource created by the cloud controller manager.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: nginx
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /
port: 80
failureThreshold: 1
initialDelaySeconds: 1
periodSeconds: 30
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /
port: 80
failureThreshold: 1
initialDelaySeconds: 1
periodSeconds: 30
timeoutSeconds: 5
startupProbe:
httpGet:
path: /
port: 80
failureThreshold: 1
initialDelaySeconds: 1
periodSeconds: 30
timeoutSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
annotations:
service.beta.kubernetes.io/azure-pip-tags: costcenter=1234
spec:
externalTrafficPolicy: Local
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
In the following example, the costcenter=1234
tag of the Azure Public IP in the node resource group of the AKS cluster was created by the service.beta.kubernetes.io/azure-pip-tag
annotation in the YAML manifest used to deploy the Kubernetes service.
IMPORTANT: Setting tags on files, disks, and public IPs by using Kubernetes updates the set of tags. For example, if your disk has the tags dept=IT and costcenter=5555, and you use Kubernetes to set the tags team=beta and costcenter=3333, the new list of tags would be dept=IT, team=beta, and costcenter=3333. Any updates that you make to tags through Kubernetes will retain the value that's set through Kubernetes. For example, if your disk has tags dept=IT and costcenter=5555 set by Kubernetes, and you use the portal to set the tags team=beta and costcenter=3333, the new list of tags would be dept=IT, team=beta, and costcenter=5555. If you then remove the disk through Kubernetes, the disk would have the tag team=beta.
You can apply Azure tags to Azure managed disks and Azure files by using a Kubernetes YAML manifest created via a persistent volume claim PVC. Today you cannot specify metadata and, in particular, tags in a PVC that are passed down to a CSI driver. Today, the only way to tag managed resources dynamically created by a PVC on a cloud platform is using the tags parameter of the Azure Disks CSI driver:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: contoso-managed-csi-premium
provisioner: disk.csi.azure.com
parameters:
skuname: Premium_LRS
tags: costcenter=1234,tenant=Contoso
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: contoso-managed-csi-premium
spec:
accessModes:
- ReadWriteOnce
storageClassName: contoso-managed-csi-premium
resources:
requests:
storage: 50Gi
In a multi-tenant environment, this approach can lead to two kinds of problems:
You can use the following workaround to avoid the proliferation of storage classes:
In the following example, the costcenter=1234
and tenant=Contoso
tags of the Azure managed disk in the node resource group of the AKS cluster were created by the tags
parameter in the YAML manifest used to create the contoso-managed-csi-premium
storage class used by the persistent volume claim (PVC).
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.