Azure Image Testing for Linux (AITL)

As cloud and AI evolve at an unprecedented pace, the need to deliver high-quality, secure, and reliable Linux VM images has never been more essential. Azure Image Testing for Linux (AITL) is a self-service validation tool designed to help developers, ISVs, and Linux distribution partners ensure their images meet Azure’s standards before deployment. 

With AITL, partners can streamline testing, reduce engineering overhead, and ensure compliance with Azure’s best practices, all in a scalable and automated manner. Let’s explore how AITL is redefining image validation and why it’s proving to be a valuable asset for both developers and enterprises.

Before AITL, image validation was largely a manual and repetitive process, engineers were often required to perform frequent checks, resulting in several key challenges:

• Time-Consuming: Manual validation processes delayed image releases.
• Inconsistent Validation: Each distro had different methods for testing, leading to varying quality levels.
• Limited Scalability: Resource constraints restricted the ability to validate a broad set of images.

AITL addresses these challenges by enabling partners to seamlessly integrate image validation into their existing pipelines through APIs. By executing tests within their own Azure subscriptions prior to publishing, partners can ensure that only fully validated, high-quality Linux images are promoted to production in the Azure environment.

How AITL Works?

AITL is powered by LISA, which is a test framework and a comprehensive opensource tool contains 400+ test cases. AITL provides a simple, yet powerful workflow run LISA test cases:

1. Registration: Partners register their images in AITL’s validation framework.
2. Automated Testing: AITL runs a suite of predefined validation tests using LISA.
3. Detailed Reporting: Developers receive comprehensive results highlighting compliance, performance, and security areas. All test logs are available to access.
4. Self-Service Fixes: Any detected issues can be addressed by the partner before submission, eliminating delays and back-and-forth communication.
5. Final Sign-Off: Once tests pass, partners can confidently publish their images, knowing they meet Azure’s quality standards.

Benefits of AITL

AITL is a transformative tool that delivers significant benefits across the Linux and cloud ecosystem:

• Self-Service Capability: Enables developers and ISVs to independently validate their images without requiring direct support from Microsoft.
• Scalable by Design: Supports concurrent testing of multiple images, driving greater operational efficiency.
• Consistent and Standardized Testing: Offers a unified validation framework to ensure quality and consistency across all endorsed Linux distributions.
• Proactive Issue Detection: Identifies potential issues early in the development cycle, helping prevent costly post-deployment fixes.
• Seamless Pipeline Integration: Easily integrates with existing CI/CD workflows to enable fully automated image validation.

Use Cases for AITL

AITL designed to support a diverse set of users across the Linux ecosystem:

• Linux Distribution Partners: Organizations such as Canonical, Red Hat, and SUSE can validate their images prior to publishing on the Azure Marketplace, ensuring they meet Azure’s quality and compliance standards.
• Independent Software Vendors (ISVs): Companies providing custom Linux Images can verify that their custom Linux-based solutions are optimized for performance and reliability on Azure.
• Enterprise IT Teams: Businesses managing their own Linux images on Azure can use AITL to validate updates proactively, reducing risk and ensuring smooth production deployments.

Current Status and Future Roadmap

AITL is currently in private preview, with five major Linux distros and select ISVs actively integrating it into their validation workflows. Microsoft plans to expand AITL’s capabilities by adding:

• Support for Private Test Cases: Allowing partners to run custom tests within AITL securely.
• Kernel CI Integration: Enhancing low-level kernel validation for more robust testing and results for community.
• DPDK and Specialized Validation: Ensuring network and hardware performance for specialized SKU (CVM, HPC) and workloads

How to Get Started?

For developers and partners interested in AITL, following the steps to onboard.

Register for Private Preview

AITL is currently hidden behind a preview feature flag. You must first register the AITL preview feature with your subscription so that you can then access the AITL Resource Provider (RP). These are one-time steps done for each subscription.

Run the “az feature register” command to register the feature:

az feature register --namespace Microsoft.AzureImageTestingForLinux --name JobandJobTemplateCrud

Sign Up for Private Preview – Contact Microsoft’s Linux Systems Group to request access.  Private Preview Sign Up

To confirm that your subscription is registered, run the above command and check that properties.state = “Registered”

Register the Resource Provider

Once the feature registration has been approved, the AITL Resource Provider can be registered by running the “az provider register” command:

az provider register --namespace Microsoft.AzureImageTestingForLinux

*If your subscription is not registered to Microsoft.Compute/Network/Storage, please do so. These are also prerequisites to using the service. This can be done for each namespace (Microsoft.Compute, Microsoft.Network, Microsoft.Storage) through this command:

az provider register --namespace Microsoft.Compute

Setup Permissions

The AITL RP requires a permission set to create test resources, such as the VM and storage account. The permissions are provided through a custom role that is assigned to the AITL Service Principal named AzureImageTestingForLinux. We provide a script setup_aitl.py to make it simple. It will create a role and grant to the service principal.

Make sure the active subscription is expected and download the script to run in a python environment.

https://raw.githubusercontent.com/microsoft/lisa/main/microsoft/utils/setup_aitl.py

You can run the below command:
python setup_aitl.py -s "/subscriptions/xxxx"

Before running this script, you should check if you have the permission to create role definition in your subscription.

*Note, it may take up to 20 minutes for the permission to be propagated.

Assign an AITL jobs access role

If you want to use a service principle or registration application to call AITL APIs. The service principle or App should be assigned a role to access AITL jobs. This role should include the following permissions:

az role definition create --role-definition '{

   "Name": "AITL Jobs Access Role",

   "Description": "Delegation role is to read and write AITL jobs and job templates",

   "Actions": [

      "Microsoft.AzureImageTestingForLinux/jobTemplates/read",

      "Microsoft.AzureImageTestingForLinux/jobTemplates/write",

      "Microsoft.AzureImageTestingForLinux/jobTemplates/delete",

      "Microsoft.AzureImageTestingForLinux/jobs/read",

      "Microsoft.AzureImageTestingForLinux/jobs/write",

      "Microsoft.AzureImageTestingForLinux/jobs/delete",

      "Microsoft.AzureImageTestingForLinux/operations/read",

      "Microsoft.Resources/subscriptions/read",

      "Microsoft.Resources/subscriptions/operationresults/read",

      "Microsoft.Resources/subscriptions/resourcegroups/write",

      "Microsoft.Resources/subscriptions/resourcegroups/read",

      "Microsoft.Resources/subscriptions/resourcegroups/delete"

   ],

   "IsCustom": true,

   "AssignableScopes": [

       "/subscriptions/01d22e3d-ec1d-41a4-930a-f40cd90eaeb2"

   ]

}'

 

You can create a custom role using the above command in the cloud shell, and assign this role to the service principle or the App.

All set! Please go through a quick start to try AITL APIs.

Download AITL wrapper

AITL is served by Azure management API. You can use any REST API tool to access it. We provide a Python wrapper for better experience.

The AITL wrapper is composed of a python script and input files. It calls “az login” and “az rest” to provide similar experience like the az CLI. The input files are used for creating test jobs.

1. Make sure az CLI and python 3 are installed.
2. Clone LISA code, or only download files in the folder. lisa/microsoft/utils/aitl at main · microsoft/lisa (github.com).
3. Use the command below to check the help text.

python -m aitl job –-help

python -m aitl job create --help

 Create a job

Job creation consists of two entities: A job template and an image.   The quickest way to get started with the AITL service is to create a Job instance with your job template properties in the request body.

Replace placeholders with the real subscription id, resource group, job name to start a test job. This example runs 1 test case with a marketplace image using the tier0.json template. You can create a new json file to customize the test job.

The name is optional. If it’s not provided, AITL wrapper will generate one.

python -m aitl job create -s {subscription_id} -r {resource_group} -n {job_name} -b ‘@./tier0.json’

The default request body is:

{

    "location": "westus3",

    "properties": {

        "jobTemplateInstance": {

            "selections": [

                {

                    "casePriority": [ 0 ]

                }

            ]

        }

    }

}

This example runs the P0 test cases with the default image.  You can choose to add fields to the request, such as image to test. All possible fields are described in the API Specification – Jobs section. 

The “location” property is a required field that represents the location where the test job should be created, it doesn’t affect the location of VMs. AITL supports “westus”, “westus2”, or “westus3”.

The image object in the request body json is where the image type to be used for testing is detailed, as well as the CPU architecture and VHD Generation. If the image object is not included, LISA will pick a Linux marketplace image that meets the requirements for running the specified tests.  When an image type is specified, additional information will be required based on the image type. Supported image types are VHD, Azure Marketplace image, and Shared Image Gallery.

-          VHD requires the SAS URL.

-          Marketplace image requires the publisher, offer, SKU, and version.

-          Shared Image Gallery requires the gallery name, image definition, and version.

Example of how to include the image object for shared image gallery. (<> denotes placeholder):

{

  "location": "westus3",

  “properties: {

    <...other properties from default request body here>,

    "image": {

            "type": "shared_gallery",

            "architecture": "x64",

            "vhdGeneration": 2,

            "gallery": "<Example: myAzureComputeGallery>",

            "definition": "<Example: myImage1>",

            "version": "<Example: 1.0.1>"

    }

  }

}

Check Job Status & Test Results

A job is an asynchronous operation that is updated throughout the job’s lifecycle with its operation and ongoing tests status.

A job has 6 provisioning states – 4 are non-terminal states and 2 are terminal states. Non-terminal states represent ongoing operation stages and terminal states represent the status at completion. The job’s current state is reflected in the `properties.provisioningState` property located in the response body.

The states are described below:

Test results are updated in near real-time and can be seen in the ‘properties.results’ property in the response body. Results will begin to get updated during the “Running” state and the final set of result updates will happen prior to reaching a terminal state (“Completed” or “Failed”). For a complete list of possible test result properties, go to the API Specification – Test Results section.

Run below command to get detailed test results.

python -m aitl job get -s {subscription_id} -r {resource_group} -n {job_name}

The query argument can format or filter results by JMESquery. Please refer to help text for more information.

For example,

• List test results and error messages.

python -m aitl job get -s {subscription_id} -r {resource_group} -n {job_name} -o table -q 'properties.results[].{name:testName,status:status,message:message}'

• Summarize test results.

python -m aitl job get -s {subscription_id} -r {resource_group} -n {job_name} -q 'properties.results[].status|{TOTAL:length(@),PASSED:length([?@==`"PASSED"`]),FAILED:length([?@==`"FAILED"`]),SKIPPED:length([?@==`"SKIPPED"`]),ATTEMPTED:length([?@==`"ATTEMPTED"`]),RUNNING:length([?@==`"RUNNING"`]),ASSIGNED:length([?@==`"ASSIGNED"`]),QUEUED:length([?@==`"QUEUED"`])}'

Access Job Logs

To access logs and read from Azure Storage, the AITL user must have “Storage Blob Data Owner” role. You should check if you have the permission to create role definition in your subscription, likely with your administrator.  For information on this role and instructions on how to add this permission, see this Azure documentation.

To access job logs, send a GET request with the job name and use the logUrl in the response body to retrieve the logs, which are stored in Azure storage container. For more details on interpreting logs, refer to the LISA documentation on troubleshooting test failures.  To quickly view logs online (note that file size limitations may apply), select a .log Blob file and click "edit" in the top toolbar of the Blob menu. To download the log, click the download button in the toolbar.

 Conclusion

AITL represents a forward-looking approach to Linux image validation bringing automation, scalability, and consistency to the forefront. By shifting validation earlier in the development cycle, AITL helps reduce risk, accelerate time to market, and ensure a reliable, high-quality Linux experience on Azure.

Whether you're a developer, a Linux distribution partner, or an enterprise managing Linux workloads on Azure, AITL offers a powerful way to modernize and streamline your validation workflows.

To learn more or get started with AITL or more details and access to AITL, reach out to Microsoft Linux Systems Group