Introduction
Container binary drift refers to the phenomenon where a running container deviates from its original image over time. This can happen due to various reasons, such as manual updates, automated processes, or security vulnerabilities. Essentially, the container starts to differ from the static snapshot it was created from, leading to potential inconsistencies and security risks.
When thinking of container image drifts, it is important to understand the following:
- Security Risks: Image drift can introduce security risks, as the container may run software or processes that were not part of the original image. This can create a security blind spot, as traditional image scanning may not detect these changes
- Detection: Detecting image drift involves monitoring the container for changes that deviate from the original image. This can be done using tools that compare the running container's state with its original image.
- Prevention: To prevent image drift, it is recommended to implement image immutability, regularly update base images, and use image scanning tools. Monitoring and alerting for image drift can also help in identifying and addressing any deviations.
In this 3-part series we will look at the:
Part 1: Newest detection “binary drift” and how you can expand the capability using Microsoft XDR Portal https://learn.microsoft.com/en-us/defender-xdr/microsoft-365-defender-portal. We will also look what you get as result of native integration between Defender for Cloud and Microsoft XDR. We will also showcase why this integration is advantageous for your SOC teams
Part 2: Further expanding on the integration capabilities, we will demonstrate how you can automate your hunts using Custom Detection Rules https://learn.microsoft.com/en-us/defender-xdr/custom-detection-rules. Reducing operational burden and allowing you to proactively detect Kubernetes security issues. Wherever applicable, we will also suggest an alternative way to perform the detection
Part 3: Bringing AI to your advantage, we will show how you can leverage Security Copilot both in Defender for Cloud and XDR portal for Kubernetes security use cases.
Note: To keep the discussion contained, we will assume that your container workloads are running on Azure Kubernetes Services (AKS) and that your AKS cluster leverages Azure’s RBAC (https://learn.microsoft.com/en-us/azure/aks/azure-ad-rbac). We also assume that you are using Azure Container Registry (ACR) for storing images (https://learn.microsoft.com/en-us/azure/container-registry/container-registry-concepts)
Capabilities needed to detect the drift
Let’s discuss what you will need to set up in your environment to detect and triage the drift. (Remember not all drifts might be malicious, it might very well be a user or pipeline error)
Setting up Defender for Containers
We assume that you have already enabled Defender for Containers if not please follow the directions listed here https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-containers-enable?tabs=aks-d...
We will set up the Defender for Containers to detect binary drifts. The feature is available for the Azure (AKS), Amazon (EKS), and Google (GKE) clouds.
To detect the drift you will need to set up Drift Policies (https://learn.microsoft.com/en-us/azure/defender-for-cloud/binary-drift-detection#configure-drift-po...). These policies define what you want or do not to alert on. You can create exclusions by setting higher priority rules for specific scopes or clusters, images, pods, Kubernetes labels, or namespaces.
In the sample rule below
Fig. Binary drift detection rule
- Scope description: Human understandable description of where you are trying to detect the binary drift
- Cloud Scope: Refers to Azure, AWS, or GCP where the rule applies. If you expand a cloud provider, you can select specific subscription. If you don't select the entire cloud provider, new subscriptions added to the cloud provider won't be included in the rule.
- Resource Scope: Here you can narrow the scope to a specific object - Container name, Image name, Namespace, Pod labels, Pod name, or Cluster name.
- Allow list of processes: List of processes that will not trigger an alert on the given Resource Scope
Also, note that there each rule has a priority and they are evaluated in ascending order. There is a default rule to ignore the binary drift detection
Fig. Default rule
Pre-requisites for generating Alerts
Once you set up the rule it will be deployed on the Kubernetes nodes using Defender for Container’s enhanced sensor https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-containers-introduction#sens.... You can check the Defender for Container Settings https://portal.azure.com/#view/Microsoft_Azure_Security/DataCollectionBladeV2/subscriptionId/<SUB_ID_WITH _BINARY_DRIFT_DETECTION>/defendersPlan/Containers
Fig. Defender Sensor needs to be enabled
With Binary Drift detection rules set and the Defender Sensor enabled, you are all set to detect the binaries that are executing but did not originate from the original image.
Reviewing the alerts (a case study)
You can see the alerts in the “Security Alerts” pane, like so
Fig. Binary Drift alert
For example, the image that the container is running does not come with wget
An attacker probably got hold of this container and downloaded this utility to import some tools.
The alert gives you information about where the activity happened like the object namespace, image, cluster, etc.
This might or might not be enough information for you to act. Say, if you want to identify “how” this drift came to be for example, did a user logged on to container and downloaded the said binary. To supplement the information provided by the alert we can then use Defender XDR portal (https://learn.microsoft.com/en-us/defender-xdr/microsoft-365-defender-portal)
Summary
This article showed you how to leverage binary drift detection and in the next article we will focus on how you can use XDR Portal to build more context around this alert and conduct hunts.
We will also share some queries that can serve as starter [Part 2 ].