Understanding Azure VM Agents recommendation

[Chandrasekhar_Arya]

The core functionality from a workload perspective is generally built into the operating system (OS) image that is run in a VM. However, there are additional requirements such as configuration, additional software installation, monitoring, security and resiliency that are in general addressed by agents that are installed in the Guest OS. We had a tough challenge in understanding them and selecting the right agents and also understand the functionality of each agents during troubleshooting. Along with  @Ashok_Ambati  we put this document to discusses the functions and installation of the agents that are pertinent to CSP managed services. They are classified into the following groups -

• Core Agents for Windows VMs

• Core Agents for Linux VMs

• VM Extensions

• VM Extensions for Azure Monitor

Core Agents for Windows VMs

Functions Supported

The Windows VM (Guest) Agent is a secure lightweight process running in a Windows VM that performs the following functions

• Interact with the Azure Fabric Controller to

  • Obtain an IP address via DHCP

  • Perform name resolution using Azure DNS

  • Respond to health check requests from Load Balancers

• Windows VM Agent supports automatic collection and transfer of Event Logs, OS Logs, Azure Logs and some registry keys to the VM's host for use in the investigation of issues.

• Install Extensions

NOTE: The Azure Fabric Controller is part of the Azure management plane. It is responsible to assign infrastructure resources that Azure resources being created by customers map to. Agents on VMs access this Controller using the special IP address of 168.63.129.16.

Windows VM Agent Installation

The Azure VM Agent is installed by default on any Windows VM deployed from an Azure Marketplace image using any of the Azure Portal, PowerShell, Command Line Interface, or an Azure Resource Manager template. In this case updates of the Windows VM Agent are automatically performed.

When using a custom VM image, the VM Agent installer can be downloaded and manually installed. In this case updates of the Windows VM Agent need to be performed manually.

Core Agents for Linux VMs

Functions Supported

The Linux VM (Guest) Agent runs in a Linux VM and performs the following functions

• Interact with the Azure Fabric Controller to

  • Obtain an IP address via DHCP

  • Perform name resolution using Azure DNS

  • Respond to health check requests from Load Balancers

• Image Provisioning, Networking and Kernel Configuration (Details can be viewed at https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/agent-linux)

• Install Extensions

Linux VM Agent Installation

The Linux VM Agent is preinstalled on images obtained from the Azure Marketplace. Azure endorses certain Linux distributions that integrate the Linux VM Agent package into their images and repositories. When creating custom images from such endorsed Linux distributions, Installation can use the relevant RPM or DEB package.

The Linux VM Agent uses a configuration file (/etc/waagent.conf) that specifies if configuration is to be performed at provisioning time and if so, what the configuration should be. It is our understanding a default configuration is provided with all images obtained from the Azure Marketplace. For other images it is expected the configuration is provided by the image builder.

With regard to updates, it is our understanding VM instances based on images obtained from the Azure Marketplace will have their Linux VM Agents updated automatically. Other instances will need manual updation.

Virtual Machine (VM) Extensions

Azure VM extensions are small applications that provide post-deployment configuration and automation tasks on Azure VMs. For example, if a virtual machine requires software installation, anti-virus protection, or to run a script inside of it, a VM extension can be used. Azure VM extensions can be run with the Azure CLI, PowerShell, Azure Resource Manager templates, and the Azure portal. Extensions can be bundled with a new VM deployment, or run against any existing system.

Some commonly used extensions are -

• Azure Disk Encryption for Linux - leverages the dm-crypt subsystem in Linux to provide full disk encryption on select Azure Linux distributions and is integrated with Azure Key Vault to manage disk encryption keys and secrets.

• Azure Disk Encryption for Windows - leverages BitLocker to provide full disk encryption on Azure virtual machines running Windows and is integrated with Azure Key Vault to manage disk encryption keys and secrets

• Key Vault for Linux and Windows - monitors a list of observed certificates stored in key vaults, and, upon detecting a change, retrieves, and installs the corresponding certificates on the VM.

• Azure Policy guest configuration for Linux and Windows - performs audit and configuration operations inside virtual machines for policies such as security baseline definitions.

• VM Snapshot for Azure Backup for Linux and Windows - when Azure Backup is enabled for a VM, it triggers installation of this extension the first time a backup is initiated. This extension supports an application consistent backup of the Azure virtual machine without the need to shutdown the VM.

More detailed information on extensions, including available extensions, their installation and use can be viewed at https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/overview. A number of third-party extensions for tools such as Chef also exist.

VM Extensions for Azure Monitor

Azure Monitor leverages a number of options in terms of agents that warrant a more detailed discussion.

Linux VM Extensions

The recently launched Azure Monitor agent consolidates the main functions of Metrics and Log gathering and provides additional capabilities such as sending data to multiple workspaces and improved management of extensions. However it has certain drawbacks in comparison to the existing agents such as lack of support for gathering file based and IIS logs. More details on this agent are provided at https://docs.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-overview?tabs=PowerS....

In contrast there are multiple existing agents each of which is described in terms of its functionality -

• Log Analytics Agent - sends Syslog and Performance data to Log Analytics workspace.

• Dependency Agent - sends Process Dependencies and Network connection metrics via the Log Analytics agent to Log Analytics workspace.

• Telegraf agent - sends Performance data to Azure Monitor Metrics.

Windows VM Extensions

The recently launched Azure Monitor agent functions in the same manner for Windows as described above for Linux.

In contrast the multiple existing agents are described below -

• Log Analytics Agent - sends Event Logs, Performance data, File based logs, IIS logs and other data in support of various Insights services to Log Analytics workspace.

• Dependency Agent - sends Process Dependencies and Network connection metrics via the Log Analytics agent to Log Analytics workspace.

• Diagnostics Extension - sends Performance data to Azure Monitor Metrics and Logs from various sources to services such as Azure Storage and Event Hub.

Recommendations

Where the additional data or flexibility in terms of feeding different services is not required, the recommendation is to leverage the newly launched Azure Monitor agent. Assumption at this point is that the Dependency Agent will work with the new Azure Monitor agent. The expectation is that in time this new agent will evolve to provide all the functionality currently provided by the existing agents.

A detailed comparison of the newly launched Azure Monitor agent with the existing agents can be viewed at https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agents-overview.