Announcing CI/CD Enhancements for Azure Load Testing
We are excited to announce a significant update to our Azure Load Testing service, aimed at enhancing the experience of setting up and running load tests from CI/CD systems, including Azure DevOps and GitHub. This update is a direct response to customer feedback and is designed to streamline the process, making it more efficient and user-friendly. Key Features and Improvements: Enhanced CI/CD Integration: Developers and testers can now configure application components and the metrics to monitor directly from a CI/CD pipeline. This integration allows monitoring the application infrastructure during the test run. You can make the following changes to your load test YAML config. appComponents: - resourceId: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/microsoft.insights/components/appComponentResource" resourceName: appComponentResource #Optional kind: web # Optional metrics: - name: "requests/duration" namespace: microsoft.insights/components aggregation: "Average" - name: "requests/count" aggregation: "Total" namespace: microsoft.insights/components - resourceId: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/samplerg/providers/microsoft.insights/components/appComponentResource" resourceName: appComponentResource #Optional kind: web # Optional metrics: - name: "requests/duration" aggregation: "Average" namespace: microsoft.insights/components - name: "requests/count" aggregation: "Total" namespace: microsoft.insights/components Pass/Fail Criteria on Server Metrics: Users can set pass/fail criteria on server metrics from a CI/CD pipeline, providing more granular control over test outcomes. This feature helps in maintaining high performance standards by automatically flagging any performance issues. You can make the following changes to your load test YAML config. failureCriteria: clientMetrics: - avg(responseTimeMs) > 300 - percentage(error) > 50 - getCustomerDetails: avg(latency) > 200 serverMetrics: - resourceId: /subscriptions/abcdef01-2345-6789-0abc-def012345678/resourceGroups/sample-rg/providers/Microsoft.Compute/virtualMachines/sample-vm metricNamespace: Microsoft.Compute/virtualMachines metricName: Percentage CPU aggregation: Average condition: GreaterThan value: 80 - resourceId: /subscriptions/abcdef01-2345-6789-0abc-def012345678/resourceGroups/sample-rg/providers/Microsoft.Compute/virtualMachines/sample-vm metricNamespace: Microsoft.Compute/virtualMachines metricName: Available Memory aggregation: Average condition: LessThan value: 20 Parameter Overrides: The ability to override parameters of a load test configuration YAML from the Azure DevOps task or GitHub action adds flexibility and customization to the testing process. Output Variables: The Azure DevOps task now includes output variables that can be consumed in downstream steps, jobs, and stages. This feature to take further actions on the load test results within the pipeline. Pipeline Cancellation: If a pipeline in Azure Pipelines or a workflow in GitHub is cancelled, any load test triggered by the pipeline/action will also be cancelled. This ensures avoiding costs for unnecessary tests. Traceability and Results Viewing: Users can trace a test run back to the pipeline that ran the test from Azure portal. This helps in end-to-end traceability to understand what changes might have triggered the test failure. Conclusion These enhancements are designed to provide a more integrated and efficient load testing experience for our users. We believe that these updates will help developers, testers, and DevOps engineers to better manage their load testing processes, ensuring high performance and reliability of their applications. We look forward to your feedback and are excited to see how these new features will improve your CI/CD workflows. Stay tuned for more updates and happy testing!Azure Load Testing Celebrates Two Years with Two Exciting Announcements!
[Update on March 18, 2025: AI-powered load test generation, referred to in the third section below, is in preview now!] Azure Load Testing (ALT) has been an essential tool for performance testing, enabling customers across industries to run thousands of tests every month. We are thrilled to celebrate its second anniversary with two major announcements. In this blog post, we will delve into the remarkable capabilities of ALT and reveal the exciting developments that will redefine load testing for you. Why do customers love ALT? ALT is a powerful service designed to ensure that your applications can handle high traffic and perform optimally under peak load. Here are some key features of ALT: Large-scale tests: Simulate over 100,000 concurrent users. Long-duration tests: Run tests for up to 24 hours. Multi-region tests: Simultaneously simulate users from any of the 20 supported regions. Continuous tests: Catch performance regression early by integrating with Azure Pipelines, GitHub Actions, or other CI/CD systems. Comprehensive test results: Correlate server-side metrics with client-side metrics for end-to-end insights. Analytics and insights: Quickly and easily identify performance bottlenecks with detailed analytics. Pricing Changes: Listening to You We have heard your feedback and are excited to announce significant pricing changes, effective March 1, 2025: No monthly resource fee: We have eliminated the $10 monthly resource fee to help you save on overall costs. 20% price reduction: The cost per Virtual User Hour (VUH) for >10,000 VUH is reduced from 7.5 cents to 6 cents. Additionally, we are introducing a feature to set a consumption limit per resource. This will enable central teams, such as the Performance Center of Excellence, to effectively manage and control the costs incurred by each team. These changes reflect our commitment to making ALT more accessible and cost-effective, ensuring that you can optimize your applications without worrying about budget constraints. Locust-Based Tests: Offering You a Choice In another exciting development, we are delighted to announce the availability of Locust-based tests. This addition allows you to leverage the power, flexibility, and developer-friendly nature of the Python-based Locust load testing framework, in addition to already supported Apache JMeter load testing framework. We are also working on making it easy for you to generate tests by leveraging AI. With our integration with GitHub Copilot, you will be able to simply start with a Postman Collection or an HTTP file and leverage the copilot to generate Locust-based tests. Stay tuned! This update opens new possibilities for you, providing a choice of load testing frameworks and making it easy to generate tests. In Summary As we celebrate the second anniversary, we are committed to continually improving and evolving the service to meet your needs. With the introduction of half a dozen features (1. consumption limits, 2. Locust-based tests, 3. support for multiple test files, 4. scheduling, 5. notifications, 6. support for managed identity) apart from pricing changes, we are confident that ALT will continue to be an indispensable tool for your performance testing arsenal. We are excited about all the 50+ updates over two years and look forward to seeing how they enhance your testing processes. Thank you for being a part of our journey, and we can't wait to see what you achieve with ALT. If you would like to share how you were able to leverage ALT for an interesting scenario, email me at shon dot shah at microsoft dot com or post your feedback at https://aka.ms/malt-feedback. Happy load testing!AI-Powered Load Testing in VS Code with Azure Load Testing & GitHub Copilot
There's a better way than writing load test scripts by hand. The new Azure Load Testing extension for Visual Studio Code (Preview), now integrated with GitHub Copilot, automatically generates realistic, Locust-based load tests. It seamlessly handles authentication, API request sequencing, response validation, and test data—helping you save time and ensure realistic performance testing. With this AI-driven tool, you can: Instantly generate Locust test scripts from Postman collections, Insomnia collections, or .http files. Easily enhance tests with GitHub Copilot, like adding random data or dynamic user flows. Quickly iterate by running tests locally before scaling up. Easily execute large-scale tests in Azure Load Testing to uncover performance bottlenecks. Spend less time wrestling with test scripts and more time optimizing your app's scalability. Key Features 🔹 AI-Generated Locust Scripts in Seconds Skip manual script creation. In just a few clicks, provide a Postman collection, Insomnia collection, or .http file to generate a complete load test script. Copilot then: Sequences API requests, passing values from one request to the next to simulate real workflows Integrates authentication and headers without hardcoding credentials Validates requests and handles errors gracefully with validations and logs Aggregates response metrics for detailed success and failure rates. Cleans up test data when the test finishes Produces reusable scripts you can refine locally and scale to Azure Load Testing 🔹 Refine with AI-Powered Customization Need to adjust your test script? Jusk ask: 💬 "Randomize request payloads" 💬 "Use data from a CSV file for username/password" 💬 "Retry requests for HTTP errors" 💬 "Define a custom load test shape for simulating spikes" Copilot suggest the changes, and you can apply them with a single click—no need to manually edit the script. 🔹 Run Tests Locally for Fast Iteration Quickly validate your Locust script in VS Code before scaling: Run Locust locally right from within VS Code Monitor performance and errors in real time in the Locust web UI Fine-tune and troubleshoot quickly By iterating locally, you ensure your tests work correctly before moving to large-scale testing. 🔹 Scale with One Click in Azure Load Testing Ready for bigger tests? Move your scripts to Azure Load Testing with simple YAML-based configuration: Set virtual users and load parameters in VS Code Choose regions to simulate distributed traffic Track client and server performance metrics in real time to pinpoint bottlenecks Azure Load Testing can handle many thousands of concurrent users, so you can find issues before they hit production. 🔹 Secure Test Secrets with Azure Key Vault Performance tests often require API keys or tokens. The Azure Load Testing extension integrates with Azure Key Vault for secure secret management during test runs. 🚀 Get Started Stop struggling with manual load testing scripts. Let GitHub Copilot and Azure Load Testing do the heavy lifting, so you can focus on building scalable, high-performing apps. 🔹 Install the Azure Load Testing extension for VS Code (Preview) and start testing smarter today!Run Locust-based Tests in Azure Load Testing
We are excited to announce support for Locust, a Python-based open-source performance testing framework, in Azure Load Testing. As a cloud-based and fully managed service for performance testing, Azure Load Testing helps you easily achieve high scale loads and quickly identify performance bottlenecks. We now support two load testing frameworks – Apache JMeter and Locust. You can use your existing Locust scripts and seamlessly leverage all the capabilities of Azure Load Testing. Locust is a developer friendly framework that lets you write code to create load test scripts as opposed to using GUI based test creation. You can check-in the scripts into your repos, seek peer feedback, and better maintain the scripts as they evolve – just like you would do for your product code. As for extensibility, whether it is sending metrics to a database, simulating realistic user behavior, or using custom load patterns, you can just write Python code and achieve your objective. You can also use Azure Load Testing to integrate locust based load tests in to your CI/CD workflows. Very soon, you will be able to get started from Visual Studio Code (VS Code) and leverage the power of AI to get a Locust script generated. You can then run it at scale using Azure Load Testing and get the benefits of a managed service, all from within the VS Code experience. User feedback in action During the preview phase, many of you tried out Locust in Azure Load Testing and provided us invaluable feedback. We have put that into action and improved the offering to further enhance your experience. We have ensured that your experience of getting a Locust script working with Azure Load Testing is frictionless with zero to minimal modifications needed in your test scripts. This ensures that you can seamlessly run the same scripts in your local environment with lower load and on Azure Load Testing with high-scale load. You can now install the dependencies required for your test script by specifying them in a ‘requirements.txt’ file and uploading it along with your test script. If your test requires any supporting Python modules in addition to your test script, you can now upload multiple python files and specify the main test script from which the execution should begin. If you use a Locust configuration file to define load or any other configuration for your load test, you can just upload your .conf file along with your test script. The precedence order followed by Locust to override the values is honored. Locust plugins are already available on the Azure Load Testing test engines. You can use them without having to separately upload or configure the plugins. You have multiple options to integrate Locust load tests into your automation flows. You can use CI/CD integration, Azure CLI, or REST APIs. Very soon, you’d also be able to use Azure SDKs. Using Locust scripts with Azure Load Testing All the capabilities of Azure Load Testing that help you configure your tests, generate high scale load, troubleshoot your tests, and analyze test results are supported for Locust-based tests. Let’s see this in action using a simple example of a user browsing multiple pages in a web application. You can create a Locust script for this scenario by writing a few lines of Python code. Once you run the script in your local environment and ensure that it is working as expected, you can run the same on Azure Load Testing. To create a Locust-based test in Azure Load Testing, On the ‘Test plan’ tab, select ‘Locust’ as the load testing framework and upload your test script. You can also upload any supporting artifacts here. Figure 2: Test framework selection On the ‘Load’ tab, configure the load that you want to generate. You can specify the overall number of users required and the spawn rate in load configuration. Azure Load Testing automatically populates the number of test engines required. You can update the count, if required. You can also define the overall load required and the load pattern in your Locust script, or in a Locust configuration file. In that case, you can select the engine instances required to generate the target load. You also have the options to parameterize your test script, monitor app components and define test criteria. Once you create the test and run it, you can see a rich test results dashboard that shows the performance metrics for the overall user journey as well as for specific pages. You can slice and dice the metrics to better understand performance and identify any anomalies. You can also correlate the client-side metrics with the server-side metrics from your app components to easily identify performance bottlenecks. Get started Get started with Locust on Azure Load Testing today and let us know how it enhanced your performance testing journey. Stay tuned for more exciting updates! You can learn more about using Locust with Azure Load Testing here. Have questions or feedback? Drop a comment below or share your feedback with us in the Azure Load Testing community!Supporting Managed Identity based authentication flows in Azure Load Testing
At Microsoft, we prioritize security above anything else. As part of the Secure Future Initiative (SFI), one of the key guidelines from an identity and access security perspective is to replace secrets, credentials, certificates, and keys with more secure authentication, such as managed identities for Azure resources. With managed identities (MIs), credentials are fully managed, and they can’t be accidentally leaked. As you move towards using managed identities, our job at Azure Load Testing is to ensure that you can seamlessly run load tests on flows using MI based authentication. Introducing the support for MI based authentication scenarios in Azure Load Testing. Managed identity based authentication is typically used in communication between services. The target Azure resource authenticates using a managed identity that has access to do the specific operation on the resource. A common practice to run load tests in flows requiring authentication is to bypass authentication or to hardcode the access tokens. However, this is not the best practice. It is a less secure approach, and you can’t get a true sense of the performance either. Running load tests in this kind of scenario require the load generating agent (like an Azure Virtual Machine) to have the managed identity assigned so that the authentication request is successful. With Azure Load Testing, you can simply select the managed identity that you intend to use and your test script, with the authentication flow included, runs seamlessly. Let’s see this in action through a sample scenario. Load test scenario Let’s take the scenario of getting the details of an Azure Load Testing resource through a GET API call. In this case, we will authenticate using a managed identity. Running a load test for this scenario can be done in these steps: Create a load test for the test scenario Grant the required RBAC permissions to the managed identity being used Configure the test in Azure Load Testing and run it Creating the test script Let’s use Apache JMeter to create a load test script for this scenario. For this scenario to work, we need to first fetch a Microsoft Entra token using the managed identity and then use the access token in the GET API request. We first make a HTTP REST call to the Azure Instance Metadata Service (IMDS) endpoint to fetch the access token, extract it using a JSON extractor and save the token in a property using a JSR223 PostProcessor. This is how your JMeter script would look like for this scenario. Detailed guidance on setting up this test script is available here. Grant permissions to the managed identity To run this test script in Azure Load Testing, you can use either a system-assigned identity or a user-assigned managed identity. If you intend to use a system-assigned identity, enable it on the Azure Load Testing resource from which you want to run the load test. Then, provide the required RBAC permission for this identity on the target resource for the GET API call. If you want to use a user-assigned identity, you can select the identity that you want to use and assign it to the Azure Load Testing resource. Configuring and running the test in Azure Load Testing To configure a test to use a managed identity for authentication, you just need to select the identity that you want to use. The rest of the steps to create a test do not change. On the ‘Test plan’ tab of the test creation flow, in the ‘Managed Identity for authentication scenarios’ section, select the required managed identity. If your test scenario requires multiple authentication calls using different managed identities, you can select user-assigned identity and select the multiple identities needed. When you create and run this test, the results dashboard will look the same as that of any other load test in Azure Load Testing. You will be able to see the performance metrics for the authentication API call and the GET API call on the resource separately. In this sample scenario, we used the JMeter and a HTTP call to the IMDS endpoint for authentication. You can also use Locust framework and any other method of authentication using a managed identity to create the test script. In this blogpost, we saw how Azure Load Testing can help you set up and run load tests for scenarios involving authentication using a managed identity. Do try out this capability and let us know your feedback. You can learn more about this feature here. Have questions or feedback? Drop a comment below or share your feedback with us in the Azure Load Testing community!Introducing Notifications in Azure Load Testing: Stay Updated in Real-Time
We are thrilled to introduce Notifications in Azure Load Testing to ensure you never miss critical updates in your performance testing workflows. With notifications, you can stay informed about test runs, schedules, and results effortlessly, enabling smoother collaboration and faster response times. Why Notifications Matter? In the world of performance testing, staying on top of test progress and results is crucial. The new Notifications feature in Azure Load Testing simplifies this process by: Keeping You Informed: Receive real-time updates when test runs start, complete, or fail. Enabling Automation: Use webhooks to trigger automated actions or integrate with tools like Azure Logic Apps. Fostering Collaboration: Ensure team members stay aligned with email alerts for test and schedule updates. Key Features of Notifications 1. Supported Events Azure Load Testing supports notifications for key events, including: Test Run Started: Be notified when a test run begins. Test Run Completed: Get alerts when a test run finishes, including failed tests or results. Schedule Events: Stay informed about schedule completions or when schedules are disabled. 2. Notification Channels Azure Load Testing leverages Action Groups to deliver notifications via multiple channels like email, webhook, logic app, to name a few. 3. Configurable Rules You can tailor notifications to meet your specific needs: Configure notification rules for all tests or specific ones. Choose from up to five Action Groups per rule to target different recipients or channels. How to Configure Notifications Step 1: Create a Notification Rule In the Azure portal, navigate to your Load Test Resource. Select Notifications from the left navigation menu and click Create notification rule. Provide a name for your notification rule and select the scope: All Tests: Applicable to all current and future tests. Specific Tests: Select up to 20 tests for this rule. Choose the events you want to monitor, such as Test Run Started or Test Run Completed. Add an existing Action Group or create a new one for delivering notifications. Click Save to finalize your notification rule. Step 2: Link Notification Rules to Tests Navigate to a specific test in the Azure portal and click Notifications. Select an existing notification rule to associate with the test or create a new one. Click Apply to save your changes. Step 3: Edit or Delete Notification Rules To edit a rule, select it from the Notifications page, make the necessary changes, and save. To delete a rule, select it and click Delete. Getting Started Notifications in Azure Load Testing are designed to provide a seamless experience for staying updated on your testing workflows. Whether you want to monitor test progress, automate workflows, or enhance collaboration, Notifications have you covered. Start configuring Notifications today in the Azure portal and make the most of your load testing experience! For a detailed guide, visit our documentation. Please share your feedback on our developer community. Stay tuned for more updates and features!Automate Your Load Tests: Introducing Scheduled Load Tests in Azure Load Testing
Why Schedule Load Tests? Automating your load tests through scheduling enhances performance validation in multiple ways: Consistent performance monitoring – Set up tests to run daily, weekly, or monthly, ensuring continuous assessment of your application's stability and scalability. Reduced development disruptions – Schedule tests to run during off-peak hours, minimizing interference with active development and live production environments. Key Features of Scheduled Load Testing Azure Load Testing’s scheduling functionality offers flexible options to meet different testing needs: Customizable scheduling – Set up tests to run once, hourly, daily, weekly, or monthly. Cron-based scheduling – Define complex recurrence patterns using cron expressions. Controlled test runs – Set an end condition based on a specific date or number of occurrences. Failure handling – If three consecutive test runs fail, the schedule is automatically disabled, ensuring you can resolve issues before resuming testing to avoid unnecessary costs. How to Schedule a Load Test in Azure Load Testing Setting up a scheduled test is straightforward: Navigate to your Load Testing Resource in the Azure portal. Select the test you want to schedule. Click on 'Add Schedule' in the Schedule pane. Configure Scheduling Parameters: Start date & time Recurrence pattern (hourly, daily, weekly, monthly, or cron-based) End condition (specific date, number of occurrences, or never-ending) Save the Schedule, and the test will run automatically as per the configured recurrence. Monitoring and Managing Scheduled Tests View schedules and the status in the Schedule pane. Modify, pause, or resume an existing schedule as needed. Track scheduled test runs under the Test Runs grid, where they are labeled as ‘Scheduled’ or filter to see only the scheduled runs. Get Started Today Let us know how scheduling load tests enhances your testing strategy, and stay tuned for more exciting updates! You can learn more how to schedule load tests here. Have questions or feedback? Drop a comment below or share your feedback with us in the Azure Load Testing community!
Introducing Support for Multiple JMeter Files and Fragments in Azure Load Testing
We are excited to announce a significant update to Azure Load Testing that allows you to use multiple JMeter files and fragments in your test configurations. This feature empowers users to design more modular, flexible, and scalable performance tests, ensuring comprehensive testing for complex applications. What’s New? Previously, Azure Load Testing supported a single JMeter file for defining test scenarios. With the new update, you can now: Include multiple JMeter test files in a single load test configuration. Use JMeter fragments to define reusable components, such as authentication workflows or common request sequences. Seamlessly manage test modularity, enabling easier collaboration and maintenance. This approach improves test execution and maintenance, especially for large-scale and complex applications. Why Use Multiple JMeter Files and Fragments? Modern applications are often complex, with multiple APIs, microservices, and diverse user workflows. A single, monolithic JMeter script can become difficult to manage, scale, or reuse across projects. Here’s how the new capability addresses these challenges: 1. Modularity for Large Workflows Using multiple JMeter files allows you to break down test scenarios into smaller, manageable units. For instance, separate files can be created for: User login workflows API testing for specific microservices Database queries This approach enhances clarity and reduces errors, especially in large-scale applications. 2. Reusability with JMeter Fragments JMeter fragments are reusable components of test plans. By creating fragments for common functionalities (like token generation or error handling), teams can: Avoid duplicating efforts across different test scenarios. Quickly adapt test plans to changing requirements. 3. Collaboration Across Teams With modular scripts, different team members can work on specific JMeter files or fragments simultaneously, improving collaboration and accelerating test development. 4. Easier Debugging and Maintenance Debugging a modular test setup is more straightforward, as you can isolate and troubleshoot specific files or fragments without affecting the entire test plan. How to Create a Test with Multiple JMeter Files Follow these steps to create a performance test using multiple JMeter files in Azure Load Testing: 1. Prepare Your JMeter Files Organize your test logic into multiple JMeter .jmx files. For shared functionality, such as authentication, create reusable JMeter fragments. 2. Upload Files to Azure Load Testing Navigate to the Azure Load Testing portal. Create a new test or update an existing one. In the Test Plan section, upload all JMeter files and fragments as part of the configuration. Under the file relevance section, designate the main test script from which execution should begin. Child test scripts or fragments are treated as configuration files and can be included within a zipped folder for better organization. However, the main script file must be uploaded separately, outside of the zip. 3. Set Up Parameters and Certificates Ensure test parameters and any required certificates (managed via Azure Key Vault) are properly configured. 4. Execute and Monitor the Test Run your load test and monitor metrics through the Azure Load Testing dashboard. You can analyze throughput, response times, and resource utilization to identify performance bottlenecks. Get Started Today This update opens up new possibilities for performance testing with Azure Load Testing. Whether you’re working on a microservices architecture, testing complex workflows, or fostering team collaboration, the support for multiple JMeter files and fragments is here to streamline your testing process.
