Containers have become popular for application development and deployment due to their portability and flexibility. As more and more apps choose containerization as a means of app modernization, it is important to secure container images ensuring they remain safe from image tampering or modification.
Today we published Windows container images signed by notation, and they are now available in Microsoft Artifact Registry (Nano Server, Server Core, Windows, and Server).
In this blog, we will explore the concept of container image signing, the benefits of signing container images, how we implemented signing in Windows containers, and how it can be verified.
Basics of container image signing
The process of signing container images is similar to how application packages are signed, using a digital signature that includes information about the image, such as its hash value, and is created using a private key, thus providing the means to ensure authenticity and integrity. When a container image is signed, a digital signature is attached to it, which can be verified using a corresponding public key from a trusted source, such as a certificate authority, to ensure that it has not been tampered with and comes from a trusted source.
Signing Windows container images provides several benefits, including:
1. Authenticity: Signing ensures that the image comes from a trusted source and has not been tampered with.
2. Integrity: Signing guarantees that the image has not been modified since it was signed.
3. Trust: Signing builds trust between the image publisher and the users who download and use the image.
Implementing Windows container image signing
Before we detail how we sign Windows Container Images as part of our Continuous Integration and Continuous Delivery (CI/CD) pipeline, let's first understand what comprises a container image. Every container image has a manifest and one or more layers:
Container Manifest: Each container is identified and described using a JSON called a container manifest that is downloaded when the container image is pulled from the container registry. The container manifest will:
- Define the various layers that are included in the container, which in this case hold all the actual bits of the Windows OS
- Include layer descriptors like the digest and size
In short, the manifest can be considered a descriptor for a container's contents. Even if a container layer is slightly changed by either renaming or removing the filesystem or flipping a setting on or off, etc., that layer will have an entirely different sha256 digest listed in the container manifest.
Signing Payload: The descriptor of the container manifest is the signing payload. If a container manifest defines the contents of a container, the signing payload defines the contents of the container manifest. The size and digest are those of the actual manifest string.
To sign the Windows Container images, we go through the following process:
1. First, we generate a payload.json file that summarizes information about the container, including the container digest and size, which can be used for validation purposes:
a) The container manifest digest uses a hashed sha256 value to represent the container’s different individual layers listed inside the container manifest.
b) The signed container manifest will include the sha256 digest. If there’s a difference between the declared sha256 and the sha256 of the pulled manifest, the signature validation will fail, as there could be a difference in the layers or the config file.
2. Next, we sign this payload.json manifest with a certificate held in an Azure Key Vault Managed HSM (Hardware Security Module). We submit the manifest file to our signing service, which controls the certificate, and that certificate is used to add a signature to the manifest file.
3. After the manifest is signed, we use the ORAS CLI to upload that signed manifest for the given Windows Image.
./oras attach myregistry.azurecr.io/repositoryname/hello-world:v0.0.1 --artifact-type
'application/vnd.cncf.notary.signature' ./payload.json:application/cose -a
'io.cncf.notary.x509chain.thumbprint#S256=[\"digest here\"]'
This command declares the following:
a) The image to attach the signed manifest to:
myregistry.azurecr.io/repositoryname/hello-world:v0.0.1
b) The artifact type of the signed manifest:
'application/vnd.cncf.notary.signature'
c) The signed manifest itself:
./payload.json:application/cose
d) The thumbprint of the certificate which can verify this signed manifest:
"io.cncf.notary.x509chain.thumbprint#S256=[\"digest here\"]"
4. With the payload generated, signed, and uploaded to the Microsoft Container Registry, customers can now use the notation CLI in the following section to validate the authenticity of the uploaded Windows Container Image using the signed manifest we have attached.
Verifying Windows container image signing:
Our implementation design also allows customers to verify the signed image by setting up notation following the steps below:
1. Install notation including the prerequisites
2. Use notation to add the certificate used for the signed container image:
a) Download the certificate to verify the signed container image and save it locally with a desired file name (e.g., my_msft_signing.crt)
b) Verify the certificate
# Get the absolute path for the certificate
$CERTIFICATE_ABSOLUTE_PATH = $(Resolve-Path ".\my_msft_signing.crt").Path
# Test the certificate for validity. In this case, we are using –AllowUntrustedRoot since we have not installed the certificate into the trusted root store
Test-Certificate $(New-Object Security.Cryptography.X509Certificates.X509Certificate2 $CERTIFICATE_ABSOLUTE_PATH) -AllowUntrustedRoot
c) Add the certificate to your notation trusted store
notation cert add --type ca --store microsoft my_msft_signing.crt
3. Setup a trust policy to specify trusted identities that sign the artifacts, and the level of signature verification to use. Update the trust policy to reflect the registry scope and trust stores and save the file as “trustpolicy.json” in the appropriate notation directory structure. As an example:
$TRUST_POLICY_JSON_STRING = @"
{
"version": "1.0",
"trustPolicies": [
{
"name": "mcrtrustpolicy",
"registryScopes": [ "mcr.microsoft.com/windows/servercore" ],
"signatureVerification": {
"level" : "strict"
},
"trustStores": [ "ca:microsoft" ],
"trustedIdentities": [
"*"
]
}
]
}
"@
# Save the file under %AppData%\notation\trustpolicy.json with UTF-8 (no BOM) encoding
$TRUST_POLICY_JSON_STRING | Out-File -encoding ASCII "$env:AppData\notation\trustpolicy.json"
- The container image signing can now be verified.
notation verify mcr.microsoft.com/windows/servercore:ltsc2022
Closing
Container image signing is a security practice that helps establish container image authenticity and integrity. By signing Windows container images, we ensure that the images pulled from our registry are secure and have not been tampered with, thereby improving the secure supply chain posture. We recommend signing Windows-based container images by following the steps here. We strive to learn from your valuable feedback and engagement with the broader community.