Agentless scanning for virtual machines in the cloud – technical deep dive

Over the past three years, a notable shift has unfolded in the realm of cloud security. Increasingly, security vendors are introducing agentless scanning solutions to enhance the protection of their customers. These solutions empower users with visibility into their security posture and the ability to detect threats — all achieved without the need to install any additional software, commonly referred to as an agent, onto their workloads.

This transformative phase in cloud security, embracing the agentless approach, owes its development to the robust suite of management APIs offered by cloud service providers. In this blog post, our focus will center on the technical aspects of agentless scanning applicable to virtual machines operating in the cloud. Whether it be an Azure Virtual Machine, an AWS EC2 instance, or a Google Cloud Compute instance, for simplicity's sake, we will term them as cloud-native virtual machines (VMs).

We will cover the following topics:

• What is agentless scanning for cloud native virtual machines?
• How does agentless scanning for cloud native VMs work?
• What are the new challenges that security vendors are facing?
• As a customer, what you should validate when purchasing an agentless scanning security solution?
• Microsoft Defender for Cloud’s agentless scanning

What is agentless security for cloud native virtual machines?

Agentless scanning for cloud native VMs is a technique for extracting security insights from the operating system of a virtual machine while avoiding the installation of additional software (a security agent) on the VM. These insights encompass various aspects, including:

• Identifying vulnerabilities on the VM resulting from outdated software or missing OS patches (vulnerability management).
• Detecting malware within the VM file system.
• Investigating operating system logs to identify incidents such as brute force attacks.

While traditional Endpoint Detection & Response security agent (EDR) offers unparalleled depth in threat prevention, detection and response, agentless scanning for cloud VMs stands out as a flexible, lightweight option, particularly effective for rapid deployment in new environments, temporary workloads, or for providing initial security coverage before EDR deployment. This approach is also ideal for managing legacy systems and diverse cloud assets, where installing agents might be impractical. EDR excels in real-time threat monitoring, whereas agentless scanning ensures seamless deployment with zero impact on the guest OS. We recommend a balanced strategy that combines the strengths of both approaches, tailored to the unique needs of your infrastructure, ensuring comprehensive and adaptable security based on these complementary approaches.

How does agentless scanning for cloud native VMs work?

Each virtual machine (VM) deployed in the cloud is equipped with one or more attached disks, with one serving as the operating system disk and others designated as data disks.

In every prominent cloud service provider, including Azure, AWS, and GCP, these disks function as independent cloud-native resources, each with its own set of management APIs. These APIs offer a mechanism for duplicating a disk. Following duplication, the copy can be seamlessly attached to another VM running in the cloud.

This procedure offers a method to access the file system of one VM from another, facilitating the analysis of the 'source' VM and extraction of security findings, all without the need to install an agent.

The seemingly straightforward solution depicted in Diagram 1 introduces several challenges:

1. The new VM functions as an exact replica of the source VM, inheriting the identical operating system, configuration, and software upon OS startup. This raises a concern, especially when the source VM is part of a distributed cluster. Do we desire the new VM to automatically join this cluster? Probably not.
2. Running security analysis on the new VM poses a significant question: How will the security scanning software be installed on the new VM? Complications may arise if the source VM operates on an operating system that is not supported by the vendor's security analysis software.
3. There's a potential risk of threat actors exploiting this solution by injecting malicious software set to run on startup, posing a threat to the new VM created by the security vendor. This becomes particularly perilous when the agentless scanning is offered as a full Software as a Service (SaaS) solution, and the new VM is generated within the security vendor's cloud environment.

To address these challenges, a solution involves generating the new VM with its dedicated OS disk, which is under the control and maintenance of the security vendor providing the agentless scanning solution. Subsequently, the disks of the source VM, encompassing its OS disk, can be linked to the new VM as data disks.

The approach illustrated in Diagram 2 empowers the security vendor to introduce its own operating system disk, thereby gaining complete control over the software in operation. Executables set to auto-run from the source VM will not execute, given that the OS disk of the source VM is now connected merely as another data disk.

As evident, the technique outlined above essentially involves scanning the disks of the VMs by accessing their file systems. By delving into the operating system files and logs, whether the VM operates on a Linux or Windows OS, the security vendor can extract security insights from the scanned VM, as if it was investigated by querying the operating system API directly.

What are the new challenges that security vendors are facing?

Implementing an agentless scanning solution introduces several challenges for security vendors. To comprehensively understand these challenges, we'll dissect them into distinct aspects:

1. Security Analysis Challenges: Agentless scanning solutions, which conduct analysis by scanning the file system of the target VMs, face limitations in utilizing off-the-shelf security engines for the analysis. This is because most of the off-the-shelf security engines are designed to operate on the specific operating system being scanned, executing commands against the operating system API. For instance, when analyzing Windows VM, an off-the-shelf security engine would typically query the Windows registry for data. In the context of an agentless security solution, the approach differs, requiring the solution to parse the registry database file and scrutinize its binary data.
2. File System Challenges: Simply attaching a disk to a VM does not grant immediate access to the attached disk's file system. To access the file system of the attached disk, security vendors must mount the partitions existing on the disk. Each partition has its own file system format, and the multitude of formats includes NTFS (Windows), ext3, ext4, LVM2, among others. Some of these formats are intricate to mount, and not every off-the-shelf disk mounting program may be capable of handling them. Achieving substantial scanning coverage for their customers necessitates security vendors to support this extensive list of file systems.
   For more information about the procedure of attaching and mounting a data disk to a VM, refer to this tutorial provided by Microsoft Azure.
3. Scale Challenges: The deployment of a VM or any other compute resource solely for the purpose of agentlessly scanning another VM can incur substantial costs. Consequently, agentless scanning solutions must incorporate considerations for scalability into their architecture, aiming for a cost-effective approach. The effectiveness of the solution can be evaluated based on metrics such as scan duration, scanning intervals, and other relevant factors.
4. Encryption Challenges: Agentless scanning solutions that rely exclusively on scanning disks may face limitations when dealing with virtual machines with strict disk encryption policies.. For instance, in the case of Microsoft Azure confidential VMs, the OS disk is encrypted in a manner that restricts access solely to the confidential VM itself. It's crucial to highlight that confidential VMs are often regarded as crown jewels in most scenarios.
5. Uncovered temporary storage: Every cloud provider allocates temporary storage to virtual machines, be it the Azure VM temporary disk or the AWS EC2 instance store. This temporary storage serves as a short-term storage solution for applications and processes. However, when duplicating VM disks through cloud-native APIs, the copied disks do not include data from the temporary disk. Consequently, the content on the temporary disk is excluded from scanning by the agentless scanning solution.
6. SaaS Challenges: Some security vendors provide their agentless scanning solution as a full Software as a Service (SaaS), implying that scans occur in a cloud environment owned and maintained by the security vendor. This introduces a distinct set of challenges that the vendor must address:
   1. Security Concerns: In a SaaS solution, VMs used for scans are created within the security vendor's production environment. Malicious users may attempt to exploit this by placing malicious executables on their own VMs and waiting for them to be executed. Consequently, the security vendor should treat scanned VMs as 'hostile' and implement a zero-trust policy. The risk escalates in multi-tenant SaaS solutions lacking physical isolation between different customers.
   2. Additional Scale Challenges: Centralized agentless scanning in an environment owned and maintained by the security vendor introduces potential scale issues. Each cloud service provider has resource creation limits and API throttling limits. Consequently, the security vendor must factor these limitations into its strategy, ensuring the distribution of scanning workloads. Failure to do so may result in the security vendor reaching a point where it cannot manage the scanning workload, leading to virtual machines not being scanned as per the promised scanning interval.

As a customer, what should you validate when purchasing an agentless scanning security solution?

When acquiring an agentless scanning security solution, ensure that it delivers the necessary coverage and visibility for your cloud workload while maintaining the privacy and security of your data. This can be achieved by addressing the following questions:

Coverage questions

1. Scanning Interval: What is the guaranteed scanning interval?
2. File System Format Coverage: Which file system formats are supported by the agentless security solution?
3. Data Disks Coverage: Does the agentless security solution scan all data disks, or is the focus solely on operating system disks? It's crucial for comprehensive security to include scanning data disks, especially for malware, high-severity software vulnerabilities (such as Log4j), and the detection of secrets stored on data disks.
4. Supported Operating System Distributions: Does the agentless security solution support your specific operating system distribution? This includes considerations for down-level Windows servers or various Linux distributions.
5. Security Coverage: What security analysis does the agentless scanning solution conduct? Does it perform a full malware scan or rely on heuristic-based methods? Does it scan for vulnerabilities?
6. Supported Disk Sizes and Configurations: Cloud providers offer various disk encryption methods using customer-managed keys (CMK). For instance, Azure customers can encrypt disks using Azure Disk Encryption or Disk Encryption Sets with CMK. Ensure that the encryption methods you use align with those supported by the agentless security solution. Additionally, consider special disk SKUs, such as Azure ultra disk, which may require specific support from the scanning solution.
   The full list of methods for Azure can be found here .
7. Scan Status Visibility: As a customer, assert that your cloud workload is fully covered by the agentless security solution. Which can only be done if the scanning solutions provide that visibility, including information on when a VM was scanned and, if not, the reasons for the omission.

Data privacy & security questions

These questions become particularly pertinent when the agentless security solution is presented as a Software as a Service (SaaS), wherein the disk content is replicated to a scanning environment controlled and maintained by the security vendor.

1. Environment Isolation and Security Hardening Level: Given that the entire disk data is duplicated in the scanning environment owned by the security vendor, it is imperative, as a customer, to ensure that the scanning environment is thoroughly secured, hardened, and compliant with stringent security regulations. Additionally, it is crucial to minimize the retention period of the copied disks to mitigate potential risks.
2. Scanning Environment Isolation: In a multi-tenant agentless scanning platform, especially within a zero-trust environment, it is advisable to ensure complete isolation of scanning environments for each customer. This additional security measure enhances the overall robustness of the system.

Microsoft Defender for Cloud agentless scanning for machines

Microsoft Defender for Cloud presents a multi-cloud agentless scanning solution that extends support to AWS and GCP, while seamlessly providing a native solution tailored for Azure customers. The agentless scanning solution adheres to Microsoft's rigorous standards for enterprise-grade security, ensuring strict compliance with privacy and security regulations even at massive scale.

Microsoft Defender for Cloud is one of the few CNAPP (Cloud-Native Application Protection Platform) solutions that offer both Cloud Security Posture Management and Cloud Workload Protection by integrating both agentless and agent-based approaches to secure and safeguard your virtual machines in the cloud.

Leveraging Microsoft Defender for Endpoint enhances real-time threat detection and prevention seamlessly. This integrated approach allows customers to enjoy the best of both worlds—comprehensive security posture for their cloud workload and dynamic threat detection and protection, especially for crown jewels that in some cases cannot be scanned through agentless scanning platforms.

Our engineering team is committed to ongoing investment in the agentless scanning platform. This involves expanding coverage and incorporating new agentless security scanners to ensure a continuously robust and evolving security solution.

Summary

• Agentless scanning for virtual machines harnesses the native capabilities provided by major cloud providers.
• Currently, agentless scanning primarily involves analyzing the file system of virtual machines by accessing their disks through cloud-native management APIs. However, these solutions may not be capable of examining the memory footprint, making them less effective against threats like fileless attacks (in contrast to an EDR).
• When evaluating an agentless security solution, ensure it aligns with your coverage requirements, including file system format coverage, scanning of data disks, and evaluation of specific agentless security features (e.g., agentless antimalware). For SaaS-based solutions, verify that data remains secured and private, given that the entire content of your virtual machines' file system is copied to the security vendor's cloud environment.
• While agentless scanning offers quick security posture on most cloud virtual machines without user intervention, there are inherent blind spots, such as confidential computing (considered crown jewels) and VM temporary disk scanning, which may require additional protection.
• Due to the limitations of agentless scanning, the installation of an EDR solution, like Microsoft Defender for Endpoint, becomes essential for real-time threat detection and prevention.
• Microsoft Defender for Cloud adopts a comprehensive approach by providing an enterprise-grade agentless scanning solution for multiple cloud providers (Azure, AWS, and GCP) and seamlessly integrating with Microsoft Defender for Endpoint. This integration ensures holistic protection for your entire cloud environment, particularly safeguarding crown jewels.