Azure Load Testing is a fully managed load testing service that enables you to easily generate high-scale load without worrying about test infrastructure set up. As a Product Manager, I get many questions about the pricing for the product. In this article, I will explain how pricing for Azure Load Testing works. The currency mentioned here is US dollar. These are the prices at the time of publishing this post. Over time, prices might change and there could be a mismatch with what customers see. For all pricing calculation and estimates, use the pricing calculator.
Cost factors when using Azure Load Testing
In this section I’ll talk about the different steps involved in using Azure Load Testing and the cost implications for each. As a developer or tester to use Azure Load Testing you need to do the following.
1. Create a load testing resource: To run tests on Azure Load Testing, you need to create a load testing resource. You can enter the details of the resource and the region in which you want to create the resource. This resource provides a centralized place to view and manage load tests, test runs, and test results. Since one resource can have multiple tests, it is recommended to have one resource per project or development team. You can have multiple resources if you want to restrict access to a resource.
Cost implication: For each load testing resource that is present in a subscription during any part of a month you are charged a monthly fee of $10, which also includes 50 virtual user hours (VUH). We’ll learn more about the 50 VUH in Step 3.
2. Create a test: The next step would be to create a load test. You can either or byuploading a JMeter script
Cost implication: Creating a test does not have any cost implications. When you create a test, you define the load and the duration of the test. These parameters influence the cost of running the test, as you'll learn in the next step. Creating a quick test from Azure portal would also run the test, which will result in incurring charges.
3. Run a test: When you create your tests, you define the number of concurrent users you want to simulate and the duration of the simulation. You can run the test to generate the desired load to your application. The number of virtual users (VUs) depends on the following.
- Threads configured in JMeter script: In the JMeter script, the number of threads configured would define the number of users that are simulated per engine. The number of virtual users is added across all the thread groups running at a given point of time.
- Number of engines: When you upload the JMeter script on Azure Load Testing and select multiple engines while creating the test, the script executes on every engine. This means that the virtual users are multiplied by the number of engines.
Cost implication: The cost of running a test is calculated based on the virtual user hours consumed (VUH). A Virtual User Hour = (# of virtual users simulated * the duration of the simulation in minutes) / 60 minutes. In simple terms, virtual users is the number of concurrent users simulated and hours is the duration for which these users were simulated.
We count the # of active virtual users every 10 seconds and you are charged for the equivalent number of fractional Virtual User Hours you use. The first 50 VUH is included in the resource cost as mentioned in Step 1.
|
Tier
|
VUH per month
|
Price
|
|
Tier 1
|
Up to 50 VUH
|
Included in resource charge, $10 per month
|
|
Tier 2
|
51 – 10,000 VUH
|
$0.15/VUH
|
|
Tier 3
|
Above 10,000 VUH
|
$0.075/VUH
|
4. Analyze test results: You can view your performance metrics real time while the test is running. Once the test completes, you can view the test summary, test results and logs. The test results are retained for a certain period of time which can be referred to here.
Cost implication: There is no cost associated with storing and viewing test results.
Calculating costs
In this section, I’ll take the example of Contoso Traders, who have been using Azure Load Testing for quite some time now.
They have one load testing resource and have configured a few tests as shown below.
Test 1: This is a performance regression test which runs in the test environment within the deployment pipeline. This test runs with average parallel users using the application everyday – 1,000 virtual users for 15 minutes. This runs weekly, so there are four test runs per month.
Test 2: This test is used to determine a system's behavior under peak conditions – 10,000 virtual users for 30 minutes. This runs twice a month.
Let’s calculate the VUH consumption for this customer.
|
|
Virtual users per test (A)
|
Test Duration in minutes (B)
|
Test runs per month (C)
|
Virtual User Hours per month (A*B*C)/60
|
|
Test 1
|
1,000
|
15
|
4
|
1,000
|
|
Test 2
|
10,000
|
30
|
2
|
10,000
|
|
Total
|
|
|
|
11,000
|
Now, let’s calculate the total price for this scenario:
Load testing resource: $10
Tiered VUH price:
|
|
VUH (A)
|
Price per VUH (B)
|
Total Price for the tier (A*B)
|
|
Tier 1
|
50
|
$0
|
$0
|
|
Tier 2
|
9,950
|
$0.15
|
$1,492.50
|
|
Tier 3
|
1,000
|
$0.075
|
$75
|
|
Total
|
11,000
|
|
$1,567.50
|
Total price = Load testing resource charge + VUH charge
= $10 + $1567.50
= $1,577.50
Conclusion
As you saw above, you can leverage Azure Load Testing to optimize your infrastructure before deploying in production, planning for the customer traffic you are expecting, while paying only for what you use. Azure Load Testing also auto-aborts the test if the error percentage exceeds 90% for a 60 second window. This is to make sure that customers are not charged for an incorrectly configured test.
You can learn more about Azure Load Testing pricing and use the pricing calculator to calculate for your usage.
You would be surprised to see that the actual pricing may vary depending on your Azure agreement. Make sure to log in to the pricing calculator and select the licensing program from the dropdown menu to create an estimate for your Azure agreement.
You can also contact an Azure sales specialist for more information on pricing or to request a price quote.
