Blog Post

Microsoft Defender for Cloud Blog
6 MIN READ

Onboarding your AWS/GCP environment to Microsoft Defender for Cloud with Terraform

BojanMagusic1's avatar
BojanMagusic1
Icon for Microsoft rankMicrosoft
Apr 18, 2023

Important Note:

Meanwhile Defender for Cloud's UI added support for Terraform. We've decided to remove the Terraform templates previously available on GitHub, because the template required for the security connector is constantly changing. Rather than have organizations use outdated templates on GitHub, we prefer them to use the UI to get the most recent versions, which include the required permissions for new capabilities, roles corresponding to onboarded features , and more. The UI generates both CloudFormation and Terraform templates, which can be copied and manually configured at the customer’s preference. Moreover, the UI itself allows you to set different values than the default values to meet your organization’s naming requirements.

 

Objective 

The purpose of this article is to provide you with step-by-step guidance on how to use Terraform templates, to onboard your AWS/GCP environment to Microsoft Defender for Cloud. Terraform is an Infrastructure as Code (IaC) tool you can use to build, change, and version your public cloud infrastructure safely and efficiently. In addition to being a widely used tool, an advantage of using Terraform to onboard your environment to Defender for Cloud, is that you can use it for both AWS and GCP.  Using Terraform to onboard your AWS/GCP environment to Defender for Cloud, allows you to automate the onboarding process and integrate it into your existing processes.

 

If you need to install Terraform, you can start at https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli. Guidance on how to get started with Terraform in Defender for Cloud, you can start at Deploy Microsoft Defender for Cloud via Terraform - Microsoft Community Hub.

 

Introduction 

Have you ever asked yourself how to onboard your AWS or GCP environment to Defender for Cloud with Terraform? This article provides you with guidance on how to use Terraform templates that have already been created to onboard your AWS/GCP environment to Microsoft Defender for Cloud. By the time you finish reading this article you’ll learn how to accomplish the following scenarios:

  • 1st Scenario: Imagine as a security admin in your organization, you want to onboard several AWS accounts not grouped under an AWS management account to Microsoft Defender for Cloud, using Terraform.
  • 2nd Scenario: Imagine as a security admin in your organization, you want to onboard all AWS accounts grouped under an AWS management account to Microsoft Defender for Cloud, using Terraform.

 

While these two scenarios mention AWS, to onboard GCP you can follow the same process. This article provides step-by-step guidance for both scenarios, starting with the first scenario.

 

Scenarios

1. Scenario: Onboarding several AWS accounts not grouped under an AWS management account to Microsoft Defender for Cloud, using Terraform.

By default, when you onboard your AWS/GCP environment at the management account-/organizational-level through the Azure portal, the security connectors that are created for each account/project are stored in the same resource group. Some organizations have requirements for security connectors to reside in different resource groups. You can use the Terraform templates provided to achieve this purpose.

 

1.1 Prepare the cloud you want to onboard:

Download the Terraform templates available from the Defender for Cloud UI. After you download the Terraform templates there are three steps you need to do.

  • Decide whether you want to onboard AWS or GCP environment to Defender for Cloud. If you want to onboard your AWS environment.
  • After you download the Terraform templates, create the variable files (.tfvars), containing the necessary variables. Most of the variables are provided in the templates you cloned. By default, all Defender for Cloud plans are enabled. Any parameter that can be changed in the UI onboarding process, can be changed in the Terraform templates as well.
  • After you create the variables file, specify to Terraform to use the variables file by providing Terraform with credentials for each account or project that you want to onboard. You can plan the process of creating the users and roles that you need using the following commands:

 

 

 

 

 

terraform init \ 
terraform plan –var-file <variable-file-name>.tfvars \ 
terraform apply –var-file <variable-file-name>.tfvars

 

 

 

 

 

 

After these commands run, Terraform does the rest. You need to repeat this process for every account or project you want to onboard. This ensures that credentials are provided by you for each account or project that you want to onboard. You can use the same credentials for all AWS accounts that you wish to onboard. This provides Terraform with the necessary permissions to the resources in question.

The Terraform template you downloaded, can integrate with existing CI/CD processes. If you need to onboard multiple accounts or projects you can also use CI/CD pipelines, to which you can add these Terraform templates to onboard your accounts or projects. After you complete these three steps, your destination cloud is ready to be onboarded and you can proceed with the onboarding process on the Azure side.

 

1.2 Prepare the Azure cloud:

Now let’s go into the process of telling Azure that you’re ready to onboard these accounts/projects.

 

You need to provide all the accounts/projects you want to onboard and provide Terraform the Azure credentials you want it to use, for creating the security connector. You can think of the security connector as the object that connects Defender for Cloud to AWS/GCP and checks if it has all the necessary permissions.

 

You can create a single variables file to onboard multiple accounts/projects. After you provide everything that you want, you need to run the following commands:

 

 

 

 

 

terraform init \ 
terraform plan –var-file <variable-file-name>.tfvars  \ 
terraform apply –var-file <variable-file-name>.tfvars

 

 

 

 

 

After you run these commands, Terraform creates all the security connectors inside of the same resource group. This effectively allows you automate the onboarding process of your accounts, projects, and everything else.

 

 

Some organizations have requirements for security connectors to reside in different resource groups. You can achieve this by modifying the Terraform template you downloaded. Now let's look at the second scenario.

 

2. Scenario: Onboarding all AWS accounts grouped under an AWS management account to Microsoft Defender for Cloud, using Terraform.

Have you ever asked yourself how you can onboard all AWS account under a AWS management account to Defender for Cloud using Terraform. You can achieve this scenario by performing the following actions. 

 

2.1 Prepare the cloud you want to onboard:

Download the Terraform templates from the Defender for Cloud UI. 

 

After you download the Terraform templates there are three steps you need to do:

  • Decide whether you want to onboard AWS or GCP environment to Defender for Cloud. 
  • After you download the Terraform templates, create the variable files (.tfvars), containing the necessary variables. 
  • After you create the variables file, specify to Terraform to use the variables file by providing Terraform with credentials for each account or project that you want to onboard. You can plan the process of creating the users and roles that you need using the following commands:

 

 

 

 

terraform init \ 
terraform plan –var-file <variable-file-name>.tfvars  \ 
terraform apply –var-file <variable-file-name>.tfvars

 

 

 

 

 

 

After you apply it, Terraform does the rest. You need to do this process only once to onboard all accounts/projects grouper under a management account or organization. The Terraform templates you cloned, can integrate with your existing CI/CD process.

 

After you complete these three steps, your destination cloud is ready to be onboarded and you can proceed with the onboarding process on the Azure side.

 

2.2 Prepare the Azure cloud:

Now let’s go into the process of telling Azure that you’re ready to onboard these accounts/projects. 

 

You need to provide the management account/organization you want to onboard and you provide Terraform the Azure credentials you want it to create the security connector with. You can think of the security connector as the object that connects Defender for Cloud to AWS/GCP and checks if it has all the necessary permissions.

 

You can create a single variables file to onboard accounts/projects below a management account/organization. After you provide everything that you want, you need to run the following commands:

 

 

 

 

terraform init \ 
terraform plan –var-file <variable-file-name>.tfvars  \ 
terraform apply –var-file <variable-file-name>.tfvars

 

 

 

 

 

After you run these commands, Terraform creates the security connector.

 

If you need to validate in the UI that the security connector(s) has been successfully created, log-in to the Azure portal and navigate to the Defender for Cloud Dashboard. Then select Environment Settings. The page that opens should display the security connectors for the accounts/projects you onboarded with Terraform templates.

 

Conclusion

Terraform templates provide you with a flexible way to automate the onboarding process of your AWS/GCP environment. In addition to being a widely used tool, an advantage of using Terraform is that you can use it for both AWS and GCP, Additionally, you can integrate the Terraform templates covered in this article, with your existing CI/CD process.

 

Contributors

Ilay Alog, Site Reliability Engineer II,

Tomer Hoter, Senior Site Reliability Engineer,

Omer Hasson, Site Reliability Engineer

 

Reviewers

Lior Arviv, Senior Product Manager,

Ilay Alog, Site Reliability Engineer II

 

 

 

Updated Apr 08, 2024
Version 5.0
  • pojala's avatar
    pojala
    Copper Contributor

    One can do a test AWS connector manually, check the resource details with `az resource`  (something like `az resource show --ids '/subscriptions/xxx/resourceGroups/xxx-rg/providers/Microsoft.Security/securityConnectors/whatevername'`) and use that for AzAPI terraform resource.   Not fun though as the documentation for the API calls is rather minimal. 

    Also in hierarchical case it can be rather impossible to manage through code as e.g. with GCP when you do the connector to the GCP organization, there will be automatically new securityConnectors for the projects it finds.  At least with GCP the names also seem to vary between projectId (projectName) and projectId (orgName_projectName) which is rather annoying.

    I guess the Defender/Sentinel teams comes from Windows application world and is not that familiar how to manage cloud environments, e.g. the terraform from the UI is considered a script with no management for the terraform state and if they had ever used terraform, they'd know what will happen if try to run terraform again against existing resources.  A bit unprofessional for an organization of the size and resources like Microsoft.
    (I'm not going into the details what the GCP IAM Logs Azure Functions does, or what the terraform for Pub/Sub Audit log connector does on Sentinel..   Audit logs indeed when the log sink is created without any filters)

  • Correct. We've decided to remove the terraform templates from GitHub, because the template required for the security connector is constantly changing. Rather than have organizations use outdated templates, we prefer them to use the UI to get the most recent versions, which include the required permissions for new capabilities, roles corresponding to onboarded features , and more. 

     

    The UI generates both CloudFormation and Terraform templates, which can be copied and manually configured at the customer’s preference. Moreover, the UI itself allows you to set different values than the default values to meet your organization’s naming requirements.

  • pojala's avatar
    pojala
    Copper Contributor

    At least the Terraform template that the UI generates for GCP could use some work.  At least a few weeks ago it was broken with duplicated resource, the code itself doesn't really follow any best practises at all and looks like copy & paste work from half a dozen different people with different coding style.  Also the instructions to deploy are broken in the sense that you just run the terraform code to deploy it but don't manage the state.

    We have already refactored what it needs to do everything through modules and just give the details, service accounts, roles, etc. through parameters but in general it would be useful if it would not only generate code but also create a simple markdown document what is needed -- so that one doesn't run completely unknown code to any cloud environment.  Likewise there should be documentation e.g. what OS Policies are being deployed and other details as the code mostly just creates a bunch of workload identity pools and gives them permissions to do this and that.  From enterprise customer perspective, I'd be reluctant to deploy it.

  • mfelisze's avatar
    mfelisze
    Copper Contributor

    Hi BojanMagusic1,
    The terraform templates from MS Defender for Cloud UI provides only terraform resources for AWS.

    There are no terraform resources for connector in azurerm provider.

    The url in Objective section (Deploy Microsoft Defender for Cloud via Terraform - Microsoft Community Hub) provides the instructions how to setup the MS Defender for Cloud itself, not the connector.

    The alternative is azapi provider and Connector API, but I cannot find option to create connector for entire AWS Organization via it.

    If I did not miss something, it looks like the only option to integrate entire AWS Organization with MS Defender for Cloud is to use UI + Cloudformation.