Running multimedia AI models on Container Apps with Serverless GPU (A100 & T4)

A video format is available for watching.

Prerequisites

- An Azure account with sufficient permissions to create resources.

- Terraform installed on your local machine.

Infrastructure Provisioning

1. Clone the Github repository and navigate to the project directory.
2. Initialize Terraform and apply the configuration to provision the necessary Azure resources, including a resource group, virtual network, log analytics workspace, container app environment, storage account, and container app for downloading models.

terraform init
terraform apply --auto-approve

The following resources will be created:

ComfyUI Deployment

The ComfyUI application is deployed as a containerized workload on Azure Container Apps. The deployment includes a job that downloads the necessary models for ComfyUI to function properly.

1. The aca_job_download_models.tf file defines a job that runs a container with the necessary commands to download the models for ComfyUI. The job is configured to run on Consumption worksload profile and has a timeout of 1200 seconds.

2. The download-models-comfyui.sh script contains the commands to download the models from Hugging Face and save them to the appropriate directory in the ComfyUI application.

Monitoring and Analytics

The Azure Log Analytics workspace is set up to collect logs and metrics from the container app environment. You can use Azure Monitor to view and analyze the logs and metrics for your ComfyUI deployment.

To view the properties and the usage of the GPU behind Container Apps, the command nvidia-smi is helpful.

Start using ComfyUI

Now that ComfyUI is provisioned, accessible on the FQDN exposed by Container Apps and the models are downloaded, you can run the Text to Image workflow in ComfyUI. You can also change the parameters as needed like the prompt.

When ready, click the Run blue button at the top right to start generating the image. It will take some time depending on the size of the image and the complexity of the prompt. Then you should see the generated image in the output node.

Using ComfyUI for Text to Video

To use ComfyUI for Text to Video generation, you can select a Text to Video template from the Workflows section. Choose Wan 2.2 Text to Video as an example. This will open the workflow to generate a video based on a text input.

Important Notes

The storage account key is required to create the storage link in your Container Apps environment. Container Apps does not support identity-based access to Azure file shares. For that it is mandatory to disable Secure Transfer at the Storage Account (more details).

Because of an issue with the Terraform provider, it won't create the Serverless GPU (A100 & T4) workload profiles. You will need to create them manually in the Azure Portal after running terraform apply.

Azure File Shares supports both SMB and NFS. Container Apps also supports both.

To mount NFS Azure Files, you must use a Container Apps environment with a custom VNet. The Storage account must be configured to allow access from the VNet either using Service Endpoint or Private Endpoint (more details)

The NFS protocol can only be used from a machine inside of a virtual network, that is why we use a Private Endpoint.

🔍 SMB vs NFS — What’s the Difference?

SMB (Server Message Block) and NFS (Network File System) are two protocols used to provide shared file storage over a network.

They serve similar purposes but have different strengths, performance characteristics, and typical use cases. NFS is native for Linux.

Consumption profile details

In Serverless GPU profiles, the GPU cost is in addition to the active usage vCPU and RAM prices for your Container App. You pay for the entire GPU cost, even if your Container App only uses a fraction of the GPU's resources. But, for CPU and Memory, you only pay for the resources your Container App actually reserves. To reduce cost, it is very important to right-size the vCPU and Memory for your Container App when using Serverless GPU profiles. You can use Azure Monitor to track the actual resource usage of your Container App and adjust the vCPU and Memory accordingly.

To get the supported profiles for a specific region, you can use the Azure CLI command:

az containerapp env workload-profile list-supported --location swedencentral -o table
# Location       Name
# -------------  -------------------------
# swedencentral  D4
# swedencentral  D8
# swedencentral  D16
# swedencentral  D32
# swedencentral  E4
# swedencentral  E8
# swedencentral  E16
# swedencentral  E32
# swedencentral  Consumption
# swedencentral  Flex
# swedencentral  Consumption-GPU-NC24-A100
# swedencentral  Consumption-GPU-NC8as-T4

Here is the vCPU, Memory and GPU consumption for the NC A100 v4 and NC T4 v3 Serverless GPU profiles with ComfyUI when running typical workloads.

You can notice that ComfyUI doesn't consume the entire compute power in terms of vCPU and Memory. That is why in Terraform, it is specified that the resource request is less than what the VM offers. That allows to reduce the cost.

Disclaimer

The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.