Introduction
A key part of Kubernetes security includes making sure the cluster is configured to industry and company best practices. This entails controlling what users can do on the cluster and blocking actions that don’t comply with pre-defined best practices.
Out of the box, Kubernetes does not provide a mechanism to write and deploy fine grained policies required per your security and compliance mandates. As a result, you will probably leverage something like Gatekeeper along with Open Policy Agent (OPA).
Defender for Containers protects your Kubernetes clusters by continuously assessing them to get visibility into misconfigurations and help mitigate identified threats. To get insight into the workload configuration on the cluster, the Azure Policy for Kubernetes is deployed as part of the Defender for Containers plan. The Azure Policy for Kubernetes extends the Gatekeeper v3 admission controller webhook for OPA. Gatekeeper is needed to check if the policy is correct before enforcing it. On Azure Kubernetes Service (AKS), it is deployed as an add-on. For Arc Enabled Kubernetes, which includes on-premises clusters and clusters hosted in Google Cloud or Amazon Web Services, it is deployed as an extension.
In this blog, we will go more into detail about how Azure Policy for Kubernetes, uses Gatekeeper with OPA in the Defender for Containers plan.
How does the policy deployment work?
The Azure Policy for Kubernetes checks with the Azure Policy service for policy assignments to the cluster, deploys policy definitions into the cluster as constraints and constraint templates and reports details back to the Azure Policy service.
Figure 1 shows at a high level how the Gatekeeper works with policy and responds to admission requests. When a user makes a request to the API server, the API server will send an admission request to the Gatekeeper validating Webhook and asks if this request should be allowed to proceed. To make a decision, the Gatekeeper will consult the Policy CRs(Core Rules). The constraint shows the gatekeeper what requirement needs to be met to make the request from the API server while the template shows exactly how to make those requests.
When enabling Defender for Containers on your subscriptions, it recommends deploying the Azure Policy agent on your Kubernetes clusters. Once the agent is deployed, Defender for Containers provides best practices via recommendations across all Kubernetes clusters. Some recommendations have parameters that must be configured before they can be used effectively. The only action required by the customer is to adjust policy parameters of those recommendations to the organizational policy. For example, in the recommendation "Container images should be deployed only from trusted registries", customers must define known and trusted registries.
Additionally, with “Container CPU and memory limits should be enforced”, customers must define the max allowed CPU units and memory bytes in the cluster in order for Defender for Containers to report non-compliant clusters.
Organizations can also leverage the “Additional Information” section of the recommendation to view parameters that need to be configured. Another parameter organizations may consider are exclusions. Organizations may have components on the cluster such as an image or container that they want to be excluded from being monitored in relevant recommendations. Where applicable, the “Additional Information” section of the recommendation would highlight the option to configure exclusions. The exclusion would apply to all relevant data plane recommendations and are not customizable per recommendation. In the recommendation “Privileged containers should be avoided”, organizations can configure exclusions based on container images. Once configured, the exclusion would apply across all data plane recommendations where an image is relevant.
To be more proactive towards Kubernetes security, consider changing the recommendation from “audit” mode to “deny”. The deny effect would prevent the creation of resources that don’t satisfy the recommendation.
Exporting Affected Components
Once the parameters have been configured for the recommendations, you can view the affected components by navigating to the unhealthy resources tab and clicking on the unhealthy cluster.
The affected components will pop up on the left blade. This information is retrieved via API calls and can be exported using a PowerShell script.
Limitations
Before deploying the Azure Policy Add-on please be aware of existing limitations and recommendations.
Summary
In this article, we demonstrated an easy way to deploy and manage the policies for your Kubernetes clusters. Through the Azure Policy for Kubernetes, which is deployed as part of the Defender for Containers plan, organizations can leverage several built-in recommendations to monitor compliance on the Kubernetes data plane. We also showed you how to view the affected components for each recommendation, export them and apply exclusions.
More Information:
- Container security with Microsoft Defender for Cloud | Microsoft Learn
- Secure your Containers from Build to Runtime - YouTube
- Learn Azure Policy for Kubernetes - Azure Policy | Microsoft Learn
Reviewers
Tomer Spivak, Senior PM(Defender for Cloud)
Maya Herskovic, Senior PM(Defender for Cloud
Mekonnen Kassa, Principal GPM (CxE, Defender for Cloud)
