5 tips for IIS on containers: #3 Hardcoded configuration

[Vinicius Apolinario]

We’re getting into the third topic of our blog post series about IIS on Windows containers. In case you missed, check out the blog on SSL certificate lifecycle management and IIS app pools and websites.

Today, we’re covering hardcoded configurations on IIS. This blog post covers the concept on IIS, but the idea is that any type of workload going into a container and running at scale on a Kubernetes environment should follow these best practices.

Thinking of workload vs. instances

When dealing with VMs, it’s common to think about the virtualized hardware and storage that goes into that VM. Since a VM is an entire OS, and we treat them as regular servers, whenever an application needs to persist state and data, we simply attach a virtual disk to that VM and call it a day. The idea with containers is a bit different. With containers, you should change the mindset to never rely on a specific instance of container. If one container goes down, another container should be able to be brought up and pick up where the previous one left off. For that to work, any state or data should be stored outside of the container and presented to any/all container instances running that workload. In the containers and Kubernetes world, this is called persistent storage. It helps in not depending on specific instances of containers and it also helps on scale-up or down scenarios.

That idea of workload mindset vs. instance also brings in another aspect: Not only should the state and data not be inside the containers, but also there should be no hardcoded configuration that restricts the container from running on a specific environment. For example, let’s say you have a dev/test database for your application, and another database for production. This is common as you don’t want to mess up your production data while testing your application. However, changing this configuration could be something simple or more complex depending on how your app (and framework) works. For IIS, web applications using ASP.Net use a config file called web.config. Because all configuration (such as database connection string) lives in the file, the setting is pretty much hardcoded into the instance. That means: If you want to change the database connection string for an ASP.Net web application on IIS, you have to manually change the file.

So, the question for the case of IIS on containers is: How do I change the configuration of the web.config file so it gets the environment-specific deployment configuration? The answer to that question is in using environment variables to modify the web.config file when the container is deployed.

Web.Config transformation and environment variables

Environment variables are not a common concept that many IT admins are familiar with. Of course, you might be familiar with it, especially for cases on which you want to call a specific tool via CLI without having to navigate to that folder and calling the tool by its full name. In that case, environment variables help a lot. (For example, when you type ping on CMD or PowerShell, you don’t need to navigate to the folder on which ping.exe resides, as the system folder is an environment variable)

For IIS, the idea is that we could provide to the container as many environment variables as needed for your app to work depending on the environment it’s going to be deployed. We could have an environment variable for DB connection string, and provide a different value for it depending if it’s going to dev/test or production, on-premises or cloud, etc.

However, modifying the content of web.config has its own challenges. Newer versions of .Net have native mechanisms to modify the file based on the environment variables that are present in the system, but older versions of .Net Framework don’t. For that reason, a common way to do this, is to use a script that reads environment variables and replaces the values on the web.config file using the native Microsoft XML Document Transformation tool.

Changing the web.config using PowerShell scripts

To understand what is happening below, you need to understand the order in which a container is started and executed. Every container instance runs based on a container image. The container image in turn, is created based on a docker file, which is a step-by-step, text file that describes how the image should be built. What we will do is to add an instruction to a docker file to call a start-up script that will be started with the container.

ENTRYPOINT ["powershell.exe", "./Startup.ps1"]

The command above should come after the instructions to deploy your ASP.Net web application, usually as the last on your docker file. Next, we need to populate the Startup.ps1 script:

.\Set-WebConfigSettings.ps1 -webConfig c:\inetpub\wwwroot\Web.config
If (Test-Path Env:\ASPNET_ENVIRONMENT)
{
    \WebConfigTransformRunner.1.0.0.1\Tools\WebConfigTransformRunner.exe \inetpub\wwwroot\Web.config "\inetpub\wwwroot\Web.$env:ASPNET_ENVIRONMENT.config" \inetpub\wwwroot\Web.config
}
while ($true) { Start-Sleep -Seconds 3600 }

The above does three things: First, it calls another script on which its sole purpose is to create another file, alongside the web.config one, with the environment variables to be used. That’s the Set-WebConfigSettings.ps1 script. The second, is that AFTER the previous script ran, it will call the WebConfigTransformRunner tool (a Nuget package that needs to be restored as part of building your image) to modify the web.config file based on the content of the XML file from the previous step. The third thing it does is to stay on a permanent loop. The reason we do this is because containers need a durable entry point on the docker file we created before. If the program called on the docker file ends (such as our script that just finished) Docker (or the container runtime) will terminate the container as it understands there’s nothing else to do. By keeping this script in a constant loop, we avoid that.

Now, all we need is the content of the Set-WebConfigSettings.ps1:

param (
    [string]$webConfig = "c:\inetpub\wwwroot\Web.config"
)

$doc = (Get-Content $webConfig) -as [Xml];
$modified = $FALSE;

$appSettingPrefix = "APPSETTING_";
$connectionStringPrefix = "CONNSTR_";

Get-ChildItem env:* | ForEach-Object {
    if ($_.Key.StartsWith($appSettingPrefix)) {
        $key = $_.Key.Substring($appSettingPrefix.Length);
        $appSetting = $doc.configuration.appSettings.add | Where-Object {$_.key -eq $key};
        if ($appSetting) {
            $appSetting.value = $_.Value;
            Write-Host "Replaced appSetting" $_.Key $_.Value;
            $modified = $TRUE;
        }
    }
    if ($_.Key.StartsWith($connectionStringPrefix)) {
        $key = $_.Key.Substring($connectionStringPrefix.Length);
        $connStr = $doc.configuration.connectionStrings.add | Where-Object {$_.name -eq $key};
        if ($connStr) {
            $connStr.connectionString = $_.Value;
            Write-Host "Replaced connectionString" $_.Key $_.Value;
            $modified = $TRUE;
        }
    }
}

if ($modified) {
    $doc.Save($webConfig);
}

The above script creates a separate XML file with the environment variables it found. It looks for environment variables with a prefix of “APPSETTING_” and “CONNSTR_”.

To summarize what all of the above does:

1. When the container instance is created, a Startup.ps1 will run.
2. Startup.ps1 calls Set-WebConfigSettings.ps1, which in turn creates an XML file with the specific environment variables it found.
3. Once the file is created, WebConfigTransformRunner will replace the settings on the web.config file.

From a container standpoint, everything is in place for the environment variables to be used to replace the configuration on the web.config file. The only thing missing are the environment variables itself.

Passing on environment variables to Windows containers on AKS

When you deploy a container, one of the options you have is to pass on environment variables to it. On Docker, you can pass on those variables when you launch the container by using:

docker run -d mycontainerimage:v1 -env MyVar=value1

On Kubernetes, the environment variables should be listed on your YAML file as part of the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: iissample
  labels:
    app: iissample
spec:
  replicas: 1
  template:
    metadata:
      name: iissample
      labels:
        app: iissample
    spec:
      nodeSelector:
        "kubernetes.io/os": windows
      containers:
      - name: iissample
        image: <imagename>
        resources:
          limits:
            cpu: 1
            memory: 800M
          requests:
            cpu: .5
            memory: 400M
        ports:
          - containerPort: 80
        env
        - name: CONNSTR_CatalogDBContext
          valueFrom:
            secretKeyRef:
              name: akvsecretscertwin
              key: CATALOG_DB_CONTEXT
        - name: APPSETTING_StorageConnectionString
          valueFrom:
            secretKeyRef:
              name: akvsecretscertwin
              key: STORAGE_CONNECTION_STRING
  selector:
    matchLabels:
      app: iissample

The example above shows the YAML file to deploy an application and passes on the environment variable with the right prefix so the container will catch it at deployment time. However, rather than providing the value for that environment variable right away, we stored it on Azure Key Vault, which is being used as a Kubernetes secret. Kubernetes secrets are used to store sensitive application configurations, such as usernames and passwords. Since the connection string for a database requires that, we used this option instead. This option requires that you also pass on a secret YAML with the description of the secret to be used:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: azure-kvname-win
spec:
  provider: azure
  secretObjects: # [OPTIONAL] SecretObject defines the desired state of synced K8s secret objects
    - data:
        - key: CATALOG_DB_CONTEXT
          objectName: CATALOG_DB_CONTEXT # name of the mounted content to sync. this could be the object name or object alias
        - key: STORAGE_CONNECTION_STRING
          objectName: STORAGE_CONNECTION_STRING     
      secretName: akvsecretscertwin
      type: Opaque

  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "<enter client ID>" #assign managed identity client_id
    keyvaultName: <enter key-vault name> # the name of the KeyVault
    cloudName: AzurePublicCloud # [OPTIONAL for Azure] if not provided, azure environment will default to AzurePublicCloud
    objects:  |
      array:
        - |
          objectName: CatalogDBContext
          objectAlias: CATALOG_DB_CONTEXT
          objectType: secret
          objectVersion: ""
        - |
          objectName: StorageConnectionString
          objectAlias: STORAGE_CONNECTION_STRING
          objectType: secret
          objectVersion: ""       
          
    resourceGroup: "<ResourceGroup>" #provide azure key vault resourceGroup
    subscriptionId: "<subscriptionId>"
    tenantId: "<TenantId>"

On Azure Key Vault, you should specify these variables as secrets that can be retrieved by the AKS nodes, so the application can be properly deployed.

Conclusion

While the process above may sound a bit complex, it’s actually very straightforward: We’re using environment variables with the configuration we want to change for the application depending on the environment we’re deploying to. That means not hardcoded configuration is in the container image, so we have more flexibility to work with the container. This concept should be true for any type of applications, but here we showed how to apply this for ASP.Net applications running on Windows containers on AKS.

If you need a complete, end-to-end example of this running on a more concrete sample application, check out the Windows Containers Demo repo on GitHub. The Ticket Desk and eShop app use this very concept to containerize existing web applications on IIS with Windows containers.

I hope this series is useful to you! Let us know in the comments below.