1. Handling Iteration over Sensitive Values in a Map of Objects
Before diving into the use case, it's important to understand the basics of handling sensitive input variables in Terraform. You can refer to the HashiCorp Developer Documentation on Protect sensitive input variables | Terraform | HashiCorp Developer for initial understanding.
In Terraform, managing sensitive values in variables like strings or objects is straightforward. However, things become more complex when working with a map(object).
Iterating over sensitive values within such structures can be challenging because Terraform does not allow direct iteration over sensitive variables using for_each and that's when the real fun starts.
Here, we'll explore a solution to this problem with a practical example, such as managing Automation Account credentials.
Approach
- Define Two Variables
- One variable (credential_var) will be of type map(object) to hold the actual sensitive data.
- Another variable (credential_var_iterator) will be a non-sensitive set(string) to facilitate iteration using for_each.
Variable Declaration
#-----------------------------------------------------------------
# - Automation Account Credential Variables
#-----------------------------------------------------------------
variable "credential_var" {
type = map(object({
name = string
username = string
password = string
description = optional(string, null)
}))
description = <<-EOT
(Optional) credential_var supports below values:
name = "(Required) Specifies the name of the Credential. Changing this forces a new resource to be created."
username = "(Required) The username associated with this Automation Credential."
password = "(Required) The password associated with this Automation Credential."
description = "(Optional) The description associated with this Automation Credential."
EOT
default = null
sensitive = true
}
#--------------------------------------------------------------------------
# - Automation Account Credential Iterator Variables
#--------------------------------------------------------------------------
variable "credential_var_iterator" {
type = set(string)
description = <<-EOT
(Optional) credential_var supports below values:
name = "(Required) Specifies the name of the Credential. Changing this forces a new resource to be created."
username = "(Required) The username associated with this Automation Credential."
password = "(Required) The password associated with this Automation Credential."
description = "(Optional) The description associated with this Automation Credential."
EOT
default = []
}
Explanation
Once variables are declared, the objective is to iterate over credential_var_iterator, which is of type set(string), but fetch the values from the actual map(object) variable, i.e., credential_var.
This variable contains sensitive values, so direct iteration is not allowed. Instead, you use the non-sensitive iterator to access the sensitive data indirectly within your resource or module definitions.
This approach ensures that sensitive values are securely managed while still enabling the required iteration logic for complex scenarios. By separating the iteration mechanism from the sensitive data, you maintain both security and flexibility.
Resource Creation
#------------------------------------------------------------
# - Create Automation Account Credential
#------------------------------------------------------------
resource "azurerm_automation_credential" "this" {
for_each = var.credential_var_iterator != null && var.credential_var != null ? var.credential_var_iterator : []
name = var.credential_var[each.key].name
resource_group_name = var.resource_group_name
automation_account_name = azurerm_automation_account.this.name
username = var.credential_var[each.key].username
password = var.credential_var[each.key].password
description = var.credential_var[each.key].description
}Parameter Calling (like in TFvar file)
credential_var = {
"cred1" = {
name = "credential1"
username = module.azure-prdsvc-terraform-keyvaultsecret-username.resource.value
password = module.azure-prdsvc-terraform-keyvaultsecret-password.resource.value
description = "firstcred"
}
"cred2" = {
name = "credential2"
username = module.azure-prdsvc-terraform-keyvaultsecret-username.resource.value
password = module.azure-prdsvc-terraform-keyvaultsecret-password.resource.value
description = "secondcred"
}
}
credential_var_iterator = ["cred1", "cred2"]In the above example, instead of passing the password as plain text in the credential_var, it is highly recommended to use a Key Vault secret module to securely store and fetch sensitive values like passwords. This approach enhances security by keeping sensitive data out of your Terraform code and state files.
Summary
- Challenge: Terraform restricts direct iteration over sensitive variables (e.g., map(object)) using for_each.
- Solution:
- Separate sensitive data and iteration logic into two variables:
- A sensitive map(object) for actual data.
- A non-sensitive set(string) for iteration.
- Separate sensitive data and iteration logic into two variables:
- Implementation:
- Use the non-sensitive variable to iterate.
- Access sensitive data indirectly within resources.
- Security Best Practice: Store sensitive values like passwords in Azure Key Vault to avoid exposing them in Terraform code or state files.
2. Nested Looping with For_each
Terraform supports the for_each for looping over maps or sets of values to dynamically create resources. However, nesting for_each within another for_each is not possible directly.
To overcome this limitation, we can use nested loops with Terraform's locals and functions like flatten and distinct to create complex mappings and dynamically iterate over them.
In this section, we'll explore how to create multiple virtual networks (VNets) with multiple subnets for each VNet, addressing the challenges of dynamically mapping subnets to VNets.
The Use Case
- Create multiple virtual networks.
- For each virtual network, create multiple subnets dynamically with single input
Challenge
When creating subnets, we must associate them with the correct virtual network, basically mapping must exist between a subnet and a virtual network. However, using a single for_each is insufficient because it doesn’t inherently allow mapping subnets to their respective virtual networks.
Solution Approach
- Use a local block to create a combined list of Virtual Networks (VNets) and their subnets
- Use nested for loops to iterate over variables of virtual network and subnets respectively
- Apply Terraform functions (flatten and distinct) to prepare the data structure
- Convert the resulting list into a map for looping using with for_each loop
- Iterate over resultant map in final iteration to create multiple subnets under each VNet
Step-by-Step Implementation
A. Variable Declaration
Define variables for VNets and subnets as maps of objects for structured data input.
variable "virtual_networks" {
type = map(object({
address_space = list(string)
name = string
}))
default = {}
}
variable "subnets" {
type = map(object({
name = string
}))
default = {}
}B. Create Virtual Networks
Use the for_each construct to dynamically create virtual networks based on the input variables.
resource "azurerm_resource_group" "this" {
name = "nested-loop-resource-group"
location = "West Europe"
}
resource "azurerm_virtual_network" "this" {
for_each = var.virtual_networks
name = each.value.name
address_space = each.value.address_space
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
}C. Generate VNet-Subnet Mapping
Use a local block with nested loops to generate a combined list of VNets and their subnets.
This is achieved using the flatten function to merge nested lists and the distinct function to eliminate duplicates, ensuring an optimized and clean data structure.
Learn more about these functions:
locals {
# Nested loop over both maps, and flatten the result.
vnet-subnet = distinct(flatten([
for vnet in var.virtual_networks : [
for snet in var.subnets : {
virtual_network_name = vnet.name
subnet_name = snet.name
address_prefixes = vnet.address_space
}
]
]))
}D. Understand flatten and distinct
Intermediate Result (Before flatten):
[
[
{ virtual_network_name = "vnet1", subnet_name = "subnet1", address_prefixes = ["10.0.0.0/16"] },
{ virtual_network_name = "vnet1", subnet_name = "subnet2", address_prefixes = ["10.0.0.0/16"] }
],
[
{ virtual_network_name = "vnet2", subnet_name = "subnet1", address_prefixes = ["10.1.0.0/16"] },
{ virtual_network_name = "vnet2", subnet_name = "subnet2", address_prefixes = ["10.1.0.0/16"] }
]
]
After flatten:
Combines nested lists into a single list.
[
{ virtual_network_name = "vnet1", subnet_name = "subnet1", address_prefixes = ["10.0.0.0/16"] },
{ virtual_network_name = "vnet1", subnet_name = "subnet2", address_prefixes = ["10.0.0.0/16"] },
{ virtual_network_name = "vnet2", subnet_name = "subnet1", address_prefixes = ["10.1.0.0/16"] },
{ virtual_network_name = "vnet2", subnet_name = "subnet2", address_prefixes = ["10.1.0.0/16"] }
]
After distinct:
Removes duplicate entries. In this case, the list remains unchanged.
[
{ virtual_network_name = "vnet1", subnet_name = "subnet1", address_prefixes = ["10.0.0.0/16"] },
{ virtual_network_name = "vnet1", subnet_name = "subnet2", address_prefixes = ["10.0.0.0/16"] },
{ virtual_network_name = "vnet2", subnet_name = "subnet1", address_prefixes = ["10.1.0.0/16"] },
{ virtual_network_name = "vnet2", subnet_name = "subnet2", address_prefixes = ["10.1.0.0/16"] }
]E. Convert to a Map for for_each
Terraform’s for_each requires a map or a set of strings. Convert the flattened list to a map.
for_each = {
for values in local.vnet_subnet :
"${values.virtual_network_name}.${values.subnet_name}" => values
}Resulting Map:
{
"vnet1.subnet1" = { virtual_network_name = "vnet1", subnet_name = "subnet1", address_prefixes = ["10.0.0.0/16"] },
"vnet1.subnet2" = { virtual_network_name = "vnet1", subnet_name = "subnet2", address_prefixes = ["10.0.0.0/16"] },
"vnet2.subnet1" = { virtual_network_name = "vnet2", subnet_name = "subnet1", address_prefixes = ["10.1.0.0/16"] },
"vnet2.subnet2" = { virtual_network_name = "vnet2", subnet_name = "subnet2", address_prefixes = ["10.1.0.0/16"] }
}F. Create Subnets Dynamically
Use the map in the for_each argument to create subnets for each Virtual Network.
resource "azurerm_subnet" "this" {
for_each = { for values in local.vnet-subnet : "${values.virtual_network_name}.${values.subnet_name}" => values }
name = "${each.value.virtual_network_name}-${each.value.subnet_name}"
resource_group_name = azurerm_resource_group.this.name
virtual_network_name = each.value.virtual_network_name
address_prefixes = each.value.address_prefixes
depends_on = [azurerm_resource_group.this]
}Explanation:
- Key:
- A unique key is generated for each object, e.g., "vnet1.subnet1", "vnet2.subnet2".
- This ensures Terraform can track resources uniquely.
- Value:
- The object itself is stored as the value.
Summary
- Challenge: Dynamically associating multiple subnets with each set of virtual networks using for_each loop
- Key Functions:
- flatten: Combines nested lists into one.
- distinct: Ensures no duplicate entries.
- Approach:
- Create a combined list of VNets and subnets using nested loops.
- Convert the list into a map for iteration via for_each loop.
- Dynamically create several subnets under every Virtual Network.
With this approach, Terraform efficiently handles complex nested resources creation hence coming for_each limitation.
"We invite fellow developers and Terraform enthusiasts to contribute their insights and expertise to this blog. Share your advanced Terraform techniques and solutions for navigating complex scenarios and help make this resource even more valuable for the community. Together, let's empower developers to unblock!"
Microsoft