Any activity to your Kubernetes cluster is handled as API request. So, when you create a new pod, changes container image of your deployment or read information from a configmap or secret, you basically generate requests to the Kubernetes API. We can ask Kubernetes to record all these requests and related data and metadata in log repository called Audit Logs. Each event at its different stages of execution generates event record and can be stored in audit logs based on some pre-defined policies. Please read this article on auditing carefully to understand the defined stages and defined audit levels. Remember, turning on audit logs means increase in memory consumption as well as storage consumption.
As you can see in the above-mentioned article, for a regular Kubernetes cluster, you can create your custom policy file by defining various stages and audit levels and then edit the kube-apiserver manifest file to include this policy file and the log path. But in case of AKS, as it is a managed Kubernetes service, you cannot access your master nodes and hence you are not allowed to create/implement your custom policy file. But you can configure your AKS cluster to send logs to a Log Analytics workspace or other destinations like a storage account or event hub. Enabling Audit Logs for your AKS cluster is also a recommendation from CIS Benchmark document for AKS. You can get detailed steps in this document. I am repeating those steps here for reader’s benefit. Assuming you are using Azure Portal:
- Select the resource group for your AKS cluster, such as myResourceGroup. Don't select the resource group that contains your individual AKS cluster resources, such as MC_myResourceGroup_myAKSCluster_eastus.
- On the left-hand side, choose Diagnostic settings.
- Select your AKS cluster, such as myAKSCluster, then choose to Add diagnostic setting.
- Enter a name, such as myAKSClusterLogs, then select the option to Send to Log Analytics.
- Select an existing workspace or create a new one. If you create a workspace, provide a workspace name, a resource group, and a location.
- In the list of available logs, select the logs you wish to enable. For this example, enable the kube-audit and kube-audit-admin logs. Common logs include the kube-apiserver, kube-controller-manager, and kube-scheduler. You can return and change the collected logs once Log Analytics workspaces are enabled.
- When ready, select Save to enable collection of the selected logs.
Assuming you did all steps given above, you are now ready to view some logs from control plane from your AKS cluster. Here is one such example of how to do it:
The output will look like this:
Or, you can run Kusto queries as given in this article.
That’s pretty much it. You now know how to configure Audit Logs to your AKS Cluster. We will talk about Network Policy in the next part of this series.
