Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Automating the onboarding on-premises, AWS and GCP VMs on Sentinel with Azure Arc
Published Jun 26 2020 03:33 AM 6,624 Views
Microsoft

To collect events in Azure Sentinel from VMs and servers, we use the Microsoft Monitoring Agent. The MMA  supports both Windows and Linux operating systems independently of where they run: on-premise, Azure or other clouds. The agent may be installed on Windows or Linux VMs by using one of the following methods:

  • Manual installation: following a wizard or using an existing software distribution tool.
  • Azure Automation Desired State Configuration (DSC): you can include the MMA agent as part of your desired state configuration.
  • Scripting or Automation: either using PowerShell, Bash or other automation tools such as Ansible, Puppet, Chef, etc.
  • VM extension: the Log Analytics agent virtual machine installs the Log Analytics agent on Azure virtual machines, and enrolls virtual machines into an existing Log Analytics workspace. 

 

There is a very convenient and fast approach for making sure all of your Azure VMs are onboarded into Azure Sentinel, using an Azure Policy that can audit settings inside the machine and then using the VM extensions installation as a remediation task if the agent is not present. However, it was only available for Azure VMs ; for non-Azure deployments, we had to do a manual installation, what if we could have a similar experience for those servers as well?

 

Azure Arc for servers allows us to extend Azure Policies and extensions outside the boundaries of Azure, this makes non-Azure VMs first class citizens in Azure management plane. On this article we will focus on how you can not only automate the MMA installation using extensions but also leveraging Azure Policies to warrantee all servers are reporting events to Azure Sentinel.

 

Azure Arc agent on boarding

 

To apply policies to non-Azure VMs you must first deploy the Azure Arc agent, so it is registered into ARM. Having this agent and using Azure as the management control plane will also come with a set of new benefits:

  • Have a full inventory of your resources
  • Organize them into Resource Groups and Subscriptions to follow your cloud native practices
  • Apply tagging, RBAC and locks
  • Each Azure Arc connected machine will get a resource ID in ARM, so you will be able to scope your Log Analytics queries
  • Deploy extensions at scale including Microsoft monitoring Agent, Desired State Configuration and Custom Script extension.

You could deploy this agent manually or automate the installation with tools like:

  • Windows Admin Center
  • Azure automation service
  • System Center Configuration Manager (SCCM)
  • System Center Virtual Machine Manager (SCVMM)
  • Other automation tools: Ansible, Puppet, Chef, etc.

Azure Policies with Azure Arc

 

Now that external resources are connected to Azure, you can start managing them in a similar way you’d do in Azure. A very common way to make sure all of your VMs are reporting events to Sentinel would be to set up an Azure Policy at subscription or Resource Group level so, why not do the same for non-Azure Linux and Windows systems?

 

This article, shows the steps needed to set up a policy that will act as built-in control to make sure all your servers are reporting to you Azure Sentinel Log Analytics Workspace.

 

We will set up an initiative at resource group level to make sure that all Azure Arc enabled servers are reporting to Azure Sentinel, including on-prem and other cloud servers. Since we can install extensions on Azure Arc enabled VMs, we can use deployIfNotExist policies to automatically remediate non-compliant resources and install the MMA as an extension.

 

First, I created a Linux Virtual machine in Google Cloud Platform and installed the Azure Arc agent. As a result, you will be able to see this GCP VM on Azure as an Azure Arc connected VM that is now mapped to a Resource Group, Subscription and Region.

 

01.png

 

02.png

 

Now we can set up an Initiative to the Resource Group, In this case we will set up the initiative called “Enable Azure Monitor” and the scope will be the Resource Group (you could also apply it to the entire subscription).

 

03.png

 

As part of the initiative, we need to provide the Log Analytics workspace we want to use to configure the agent to connect to. In this case, we will provide the information of the Log Analytics workspace where we have enabled Azure Sentinel.

 

04.png

 

Then we also have the option to define how the policy should act in case it finds resources that are non-compliant, using a remediation task. The initiative we are using is of the type deployIfNotExists, therefore it has the option to modify the not compliant resource so it can be put into a compliant state. The remediation task we will use is “Deploy Log Analytics agent to Linux Arc machines”, the task will leverage the Azure Arc agent to deploy MMA as an extension.

 

05.png

Once you have assigned the initiative, you will see that it will be evaluated (it may take 30 minutes to run the scan) and show that the server on GCP is not compliant.

 

06.png

 

After we run the remediation task, we will see that the extension is deployed to the VM and we have a compliant Azure Policy report, with a Linux VM reporting to our Sentinel Log Analytics workspace.

 

07.png

 

08.png

 

We can also verify that we are collecting hearbeats and security events from the VM.

 

09.png

 

Additional Resources:

 

5 Comments
Bronze Contributor

Great article!  Thanks for updating with the latest Azure Arc information.

Brass Contributor

This is incredibly useful, cheers!

Bronze Contributor

Should I be able to see these ARC enabled servers when looking at the Azure Virtual machines listing in the Log Analytics workspace (because I don't)?  This the listing you get by going to the Settings in Azure Sentinel, click Wokspace settings at the top of the page, and then click the Azure virtual Machines (VMs) link.

Microsoft

@Gary Bushey on that view, only Azure VMs are listed. However, you can see the Azure Arc connected machines in the Agent Management view on the same Workspace Settings. 

Brass Contributor

Dear All,

I wish you all  are in the pinkest of health. Let me first take this opportunity to express and thank you for putting great guiding documentations and Knowledge articles here as well as Linkedin. I'm commenting to highlight somethings which would make onboarding faster given all the pre-requisites are called out properly.

Below are the issues I faced while onboarding GCP project and connecting them to Azure Arc. Please guide if I can submit these update for below mentioned article.

 

1. Finding one- Unique ID not needed to update in Microsoft Defender for Cloud -

In the below link the point mentioned where it directs to add a "unique numeric ID presented at the end of the Cloud Shell script" should be used back in the Environment setting on the Azure Arc. 

 

Fix - In case, you have followed the documentation word by word and committed this change, then undo it or if you've encountered this comment first then skip that manual step. It is not a required step and the script generated out of the selected settings in the Azure portal while adding the GCP project in Environment setting is enough to do the needed configuration automatically in terms of Service Account name and ID.

https://learn.microsoft.com/en-gb/azure/defender-for-cloud/quickstart-onboard-gcp?pivots=env-setting...

Although, most of the pre-requisite are mentioned, e.g.
(I) OSConfig Agent API should be pre-enabled;
(II)  Metatags should be set to true for the onboarding GCP Project. 

Issue - The service account name should I did not find this one and it was what causing the issue after observing, we found that the Service account under "API and identity management" of the server was not added.

Fix -
To make sure the resources in onboarded GCP project reflect in Azure ARC it is necessary to tag those resources to a Service account ID in the GCP portal. To add it, stop the server first and use the details mentioned in image 1 (this will be custom to your own GCP project) in image 2 under API & Identity management. It can take upto several hours to reflect the changes on Azure ARC portal.
Image 1
Harshit12_0-1664531728942.png

 

Image 2

Harshit12_1-1664531931877.png


CC - @thomasmaurer @LauraNicolas 

Version history
Last update:
‎Nov 02 2021 06:00 PM
Updated by: