Integrating Azure Front Door WAF with Azure Container Apps

[cbellee]

Many customers require Web Applications & APIs to only be accessible via a private IP address with a Web Application Firewall on the internet edge, to protect from common exploits and vulnerabilities. Azure Front Door provides global routing and WAF capabilities to satisfy this requirement.

Azure Container Apps ingress can be exposed on either a Public or Private IP address. One option is to put Azure Front Door in front of an ACA public endpoint, but currently there is no way (other than in application code) to restrict access to the ACA public IP address from a single Azure Front Door instance. Azure App Service Access restrictions supports this scenario, but unfortunately, there is currently no equivalent access restriction for Azure Container Apps.

To work around this limitation, Azure Private Link Service can be provisioned in front of an internal ACA load balancer. A Private endpoint (NIC with private IP in a virtual network) is connected to the Private Link Service and an Azure Front Door Premium SKU instance can then be used to connect to the private endpoint (known as a Private Origin in AFD). This configuration removes the need to inspect the value of the "X-Azure-FDID" header sent from AFD since only a single AFD instance is connected to the private endpoint, guaranteeing traffic to the ACA environment occurs only from that specific AFD instance. The overall architecture is captured in the diagram below.

In order to create this architecture, we will cover the high-level steps outlined below.

1. Deploy an internal Azure Container App environment

2. Create an Azure Front Door Premium instance, origin group & route

3. Create an Azure Private Link Service (PLS) instance

4. Deploy an Azure Container App instance

5. Finally, approve the private endpoint connection to PLS

All steps above have been codified into an Azure Bicep deployment and shell script. To deploy the sample, you will need an Azure subscription and Bash or PowerShell console with the Az CLI installed. The Bicep templates and scripts referenced in this article are available on my GitHub, here.

First, let's review the Bash shell script used to deploy the Bicep template. I also included a PowerShell script in my GitHub repo, which is almost identical, if that's your preferred shell.

#!/bin/bash

LOCATION='australiaeast'
PREFIX='frontdoor'
RG_NAME="${PREFIX}-aca-rg"

# create resource group
az group create --location $LOCATION --name $RG_NAME

# deploy infrastructure
az deployment group create \
	--resource-group $RG_NAME \
	--name 'infra-deployment' \
	--template-file ./main.bicep \
	--parameters location=$LOCATION \
	--parameters prefix=$PREFIX

# get deployment template outputs
PLS_NAME=`az deployment group show --resource-group $RG_NAME --name 'infra-deployment' --query properties.outputs.privateLinkServiceName.value --output tsv`
AFD_FQDN=`az deployment group show --resource-group $RG_NAME --name 'infra-deployment' --query properties.outputs.afdFqdn.value --output tsv`
PEC_ID=`az network private-endpoint-connection list -g $RG_NAME -n $PLS_NAME --type Microsoft.Network/privateLinkServices --query [0].id --output tsv`

# approve private endpoint connection
echo "approving private endpoint connection ID: '$PEC_ID'"
az network private-endpoint-connection approve -g $RG_NAME -n $PLS_NAME --id $PEC_ID --description "Approved" 

# test AFD endpoint
curl https://$AFD_FQDN

The script first defines 3 environment variables used throughout the script - LOCATION, PREFIX & RG_NAME. Modify these as you see fit for your environment. A resource group is created using a reference to the $RG_NAME variable, then on line 11, the Bicep template is deployed to the resource group.

Once the deployment has completed, 3 deployment template outputs are collected and used as input to the private endpoint approval command on line 25. 

Let's break down the resources the deployed by the template. 

• Input parameters are defined to control the deployment location, prefix and container image uri.

param location string = 'australiaeast'
param prefix string = 'contoso'
param imageName string = 'mcr.microsoft.com/azuredocs/containerapps-helloworld:latest'

var suffix = uniqueString(resourceGroup().id)
var vnetName = '${prefix}-vnet-${suffix}'
var frontDoorName = '${prefix}-afd-${suffix}'
var wafPolicyName = '${prefix}wafpolicy'
var workspaceName = '${prefix}-wks-${suffix}'
var appName = '${prefix}-app-${suffix}'
var plsNicName = '${prefix}-pls-nic-${suffix}'
var plsName = '${prefix}-pls-${suffix}'
var appEnvironmentName = '${prefix}-env-${suffix}'
var originName = '${prefix}-origin-${suffix}'
var originGroupName = '${prefix}-origin-group-${suffix}'
var afdEndpointName = '${prefix}-afd-ep-${suffix}'
var loadBalancerName = 'kubernetes-internal'
var defaultDomainArr = split(appEnvironment.properties.defaultDomain, '.')
var appEnvironmentResourceGroupName = 'mc_${defaultDomainArr[0]}-rg_${defaultDomainArr[0]}_${defaultDomainArr[1]}'

• A virtual network with two subnets is created. The 'infrastructure-subnet' is used by the ACA environment to host an internal Azure load balancer and the 'privatelinkservice-subnet' is used to host the Azure Private Link Service.

resource vnet 'Microsoft.Network/virtualNetworks@2022-07-01' = {
  name: vnetName
  location: location
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/16'
      ]
    }
    subnets: [
      {
        name: 'infrastructure-subnet'
        properties: {
          addressPrefix: '10.0.0.0/23'
          delegations: []
          privateEndpointNetworkPolicies: 'Disabled'
          privateLinkServiceNetworkPolicies: 'Enabled'
        }
      }
      {
        name: 'privatelinkservice-subnet'
        properties: {
          addressPrefix: '10.0.2.0/28'
          delegations: []
          privateEndpointNetworkPolicies: 'Disabled'
          privateLinkServiceNetworkPolicies: 'Disabled'
        }
      }
    ]
    virtualNetworkPeerings: []
    enableDdosProtection: false
  }
}

• A log analytics workspace is created to host the ACA application & System logs

resource wks 'Microsoft.OperationalInsights/workspaces@2021-12-01-preview' = {
  name: workspaceName
  location: location
  properties: {
    sku: {
      name: 'pergb2018'
    }
    retentionInDays: 30
    features: {
      enableLogAccessUsingOnlyResourcePermissions: true
    }
    workspaceCapping: {
      dailyQuotaGb: -1
    }
    publicNetworkAccessForIngestion: 'Enabled'
    publicNetworkAccessForQuery: 'Enabled'
  }
}

• Next, an Azure Container App environment is deployed. Notice the 'properties.appLogsConfiguration' and 'properties.vnetConfiguration' sections where the Log Analytics workspace and infrastructure subnet are specified, respectively.

resource appEnvironment 'Microsoft.App/managedEnvironments@2022-06-01-preview' = {
  name: appEnvironmentName
  location: location
  sku: {
    name: 'Consumption'
  }
  properties: {
    vnetConfiguration: {
      internal: true
      infrastructureSubnetId: vnet.properties.subnets[0].id
      dockerBridgeCidr: '10.2.0.1/16'
      platformReservedCidr: '10.1.0.0/16'
      platformReservedDnsIP: '10.1.0.2'
      outboundSettings: {
        outBoundType: 'LoadBalancer'
      }
    }
    appLogsConfiguration: {
      destination: 'log-analytics'
      logAnalyticsConfiguration: {
        customerId: wks.properties.customerId
        sharedKey: listKeys(wks.id, wks.apiVersion).primarySharedKey
      }
    }
    zoneRedundant: false
  }
}

• An Azure Container App is then provisioned into the ACA environment. In this example we are deploying the ACA HelloWorld application from the Microsoft Container Registry (mcr.microsoft.com/azuredocs/containerapps-helloworld:latest).

resource containerApp 'Microsoft.App/containerApps@2022-06-01-preview' = {
  name: appName
  location: location
  identity: {
    type: 'None'
  }
  properties: {
    managedEnvironmentId: appEnvironment.id
    configuration: {
      activeRevisionsMode: 'Single'
      ingress: {
        external: true
        targetPort: 80
        exposedPort: 0
        transport: 'Auto'
        traffic: [
          {
            weight: 100
            latestRevision: true
          }
        ]
        allowInsecure: false
      }
    }
    template: {
      containers: [
        {
          image: imageName
          name: appName
          resources: {
            cpu: '0.25'
            memory: '0.5Gi'
          }
        }
      ]
      scale: {
        maxReplicas: 10
      }
    }
  }
}

• An Azure Private Link Service is deployed using a separate Bicep module file (./modules/pls.bicep). Notice that the 'appEnvironmentResourceGroupName' parameter expects the 'MC_' prefixed resource group name that's automatically created when a custom virtual network is specified at ACA environment deployment time.
• The PLS deployment will also create a NIC in the 'privatelinkservice-subnet', to which the Azure Front Door backend will connect later in the deployment.

param name string
param location string
param appEnvironmentResourceGroupName string
param loadBalancerName string
param subnetId string

resource loadBalancer 'Microsoft.Network/loadBalancers@2022-07-01' existing = {
  name: loadBalancerName
  scope: resourceGroup(appEnvironmentResourceGroupName)
}

resource privateLinkService 'Microsoft.Network/privateLinkServices@2022-07-01' = {
  name: name
  location: location
  properties: {
    autoApproval: {
      subscriptions: [
        subscription().subscriptionId
      ]
    }
    visibility: {
      subscriptions: [
        subscription().subscriptionId
      ]
    }
    fqdns: []
    enableProxyProtocol: false
    loadBalancerFrontendIpConfigurations: [
      {
        id: loadBalancer.properties.frontendIPConfigurations[0].id
      }
    ]
    ipConfigurations: [
      {
        name: 'ipconfig-0'
        properties: {
          privateIPAllocationMethod: 'Dynamic'
          subnet: {
            id: subnetId
          }
          primary: true
          privateIPAddressVersion: 'IPv4'
        }
      }
    ]
  }
}

output id string = privateLinkService.id
output name string = privateLinkService.name

• The Azure Front Door Premium instance and it's dependent Endpoint, Origin, Origin Group and Route resources are now created. 
  • Endpoint - defines a new publicly accessible Global AFD endpoint
  • Origin Group - defines the AFD load balancing and health probe settings
  • Origin - associates the ACA container app ingress hostname & header with the Azure Private Link Service
  • Route - binds the Endpoint to the Origin Group

resource frontDoor 'Microsoft.Cdn/profiles@2022-11-01-preview' = {
  name: frontDoorName
  location: 'Global'
  sku: {
    name: 'Premium_AzureFrontDoor'
  }
  properties: {
    originResponseTimeoutSeconds: 30
    extendedProperties: {
    }
  }
}

resource afdOriginGroup 'Microsoft.Cdn/profiles/origingroups@2022-11-01-preview' = {
  parent: frontDoor
  name: originGroupName
  properties: {
    loadBalancingSettings: {
      sampleSize: 4
      successfulSamplesRequired: 3
      additionalLatencyInMilliseconds: 50
    }
    healthProbeSettings: {
      probePath: '/'
      probeRequestType: 'GET'
      probeProtocol: 'Https'
      probeIntervalInSeconds: 60
    }
    sessionAffinityState: 'Disabled'
  }
}

resource afdEndpoint 'Microsoft.Cdn/profiles/afdendpoints@2022-11-01-preview' = {
  parent: frontDoor
  name: afdEndpointName
  location: 'Global'
  properties: {
    autoGeneratedDomainNameLabelScope: 'TenantReuse'
    enabledState: 'Enabled'
  }
}

resource afdRoute 'Microsoft.Cdn/profiles/afdendpoints/routes@2022-11-01-preview' = {
  parent: afdEndpoint
  name: 'route'
  properties: {
    customDomains: []
    originGroup: {
      id: afdOriginGroup.id
    }
    originPath: '/'
    ruleSets: []
    supportedProtocols: [
      'Http'
      'Https'
    ]
    patternsToMatch: [
      '/*'
    ]
    forwardingProtocol: 'MatchRequest'
    linkToDefaultDomain: 'Enabled'
    httpsRedirect: 'Enabled'
    enabledState: 'Enabled'
  }
  dependsOn: [
    afdOrigin
  ]
}

resource afdOrigin 'Microsoft.Cdn/profiles/origingroups/origins@2022-11-01-preview' = {
  parent: afdOriginGroup
  name: originName
  properties: {
    hostName: containerApp.properties.configuration.ingress.fqdn
    httpPort: 80
    httpsPort: 443
    originHostHeader: containerApp.properties.configuration.ingress.fqdn
    priority: 1
    weight: 1000
    enabledState: 'Enabled'
    sharedPrivateLinkResource: {
      privateLink: {
        id: privateLinkService.outputs.id
      }
      privateLinkLocation: location
      status: 'Approved'
      requestMessage: 'Please approve this request to allow Front Door to access the container app'
    }
    enforceCertificateNameCheck: true
  }
}

• Finally, we define a WAF policy and associate it with the AFD Endpoint.

resource wafPolicy 'Microsoft.Network/FrontDoorWebApplicationFirewallPolicies@2022-05-01' = {
  name: wafPolicyName
  location: 'Global'
  sku: {
    name: 'Premium_AzureFrontDoor'
  }
  properties: {
    policySettings: {
      enabledState: 'Enabled'
      mode: 'Prevention'
      requestBodyCheck: 'Enabled'
    }
    managedRules: {
      managedRuleSets: [
        {
          ruleSetType: 'Microsoft_DefaultRuleSet'
          ruleSetVersion: '1.1'
          ruleGroupOverrides: []
          exclusions: []
        }
        {
          ruleSetType: 'Microsoft_BotManagerRuleSet'
          ruleSetVersion: '1.0'
          ruleGroupOverrides: []
          exclusions: []
        }
      ]
    }
  }
}

resource afdSecurityPolicy 'Microsoft.Cdn/profiles/securitypolicies@2022-11-01-preview' = {
  parent: frontDoor
  name: '${prefix}-default-security-policy'
  properties: {
    parameters: {
      wafPolicy: {
        id: wafPolicy.id
      }
      associations: [
        {
          domains: [
            {
              id: afdEndpoint.id
            }
          ]
          patternsToMatch: [
            '/*'
          ]
        }
      ]
      type: 'WebApplicationFirewall'
    }
  }
}

One the template has deployed successfully, the 3 template output parameters are collected and used as input to the ''az network private-endpoint-connection approve" Az CLI command to approve the Private Endpoint connection to the Private Link Service, on line 25.

Once approved, you will be able to access the Azure Container app via a browser using the AFD endpoint URL saved in the AFD_FQDN environment variable.

$ echo https://$AFD_FQDN
https://frontdoor-afd-ep-rczv4qasdrrms-akcehrf2dncngxfa.z01.azurefd.net