First published on MSDN on Aug 01, 2018
Authored by Edwin Hernandez
In this post, we would like to talk about Performance Testing Tools. The market for Performance Testing has evolved over time with some newcomers and also test tools that have been around for 10-20 years and have been updating their features to rise up to the challenges of modern applications.
One important note is that I am only going to list full-featured Performance testing tools that are used for End-to-End Load testing, Endurance validation and capacity planning of Enterprise-level applications. With the popularity of Cloud computing, many of these platforms offer Load Tests integrated to their platforms (AWS, Azure, Google cloud). For the most part this is limited to URL or API testing, where you specify a list of URLs for the platform to use to stress a website and produce results. This only works on open websites that do not have authentication and do not need to be tested with complex Business processes (test cases). Some of these platforms have an option to upload a JMeter script or a VS webtest instead of specifying URLS. This will be covered below.
LoadRunner is the oldest and most mature performance test tool in the market, it has been around since 1994, it really picked up on version 5 or 6 in 1998-9. This tool was originally created by Mercury Interactive, which was acquired by HP in 2006. Later in 2016 HP sold its software division to Micro-focus. LoadRunner is unique in that its core features are divided into 5 different software installations: VuGen, Controller, Generator, Agent process and Analysis. It has a wide range of technologies and communication protocols that it can test, however it is not very well adjusted to modern technologies like Cloud computing.
This tool by Apache has been around since 1999 and gained popularity throughout the 2000s, taking market share from LoadRunner mainly because it is a free, open-source alternative that can be expensively customized. It's important to point out that JMeter even though designed and used mainly for load testing, it is also used for Unit and Functional testing of FTP, LDAP, WebServices and database connections.
JMeter feels a bit dated because of its IDE and some of its features . Reporting is limited when compared to LoadRunner. Small companies love it not only because it's free and runs on any OS that supports Java but also because it can be used for unit test, functional test and monitoring which makes it kind of a swiss-knife of testing. This obviously attracts teams that want to save time and effort learning more than one tool.
Included by Microsoft in every release of Visual Studio since 2005. It's core features were updated in 2008 and 2012, since then most updates have been focused on Cloud Load Generation, Testing with VS Team Services and it's integration with Azure and VSTS for DevOps and continuous release scenarios.
WebLoad is a full-featured enterprise-level load testing tool by RadView. It has many similar features compared to LoadRunner or Visual Studio Load Tests while it has a lower cost than LoadRunner (but it's not free or open-source as JMeter is). It is a pretty popular LoadRunner competitor and has been gaining market in the last years.
Originally created by Segue, acquired by Borland where it gained its most notoriety, it was acquired by Micro-Focus in 2009. (Micro-Focus now has two of the top offering in the Load Testing market, the other on being LoadRunner). It is a tool very similar to LoadRunner in features but these are not at the same level on quality or maturity.
SmartBear has test suite that includes TestComplete for UI testing, SoapUI for API functional testing and LoadUI for load testing specialized on APIs.
SOASTA specializes in cloud load generation, a cheaper price model (when compared to LoadRunner) and faster scripting and testing. It is one of the most recent robust tools in the market.
Solid features, easy to use, embed in Eclipse, good for ERPs and IBM products. Limited Protocols, lacks on Execution and monitoring.
We struggled to find good market research on Performance Testing tools because of two factors:
Based on what I've found it looks like JMeter and LoadRunner are the two major players in Load Testing:
Consider the following:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.