TOP 20 tools for load testing in 2017

Yuriy 08/05/2017

The best tools for load testing will be considered in this article. You can find here information about free as well as commercial tools to make your load testing more effective.

Free load testing tools

Let’s start from the fully free distributed tools.

Apache JMeter™

Apache JMeter

Apache JMeter is the most often used open source tool for load testing. JMeter originally was designed for testing web application or FTP application. At present, it also can be used for the functional test, database server test etc. JMeter doesn’t require state of art infrastructure for load testing and provides support for multiple load injectors managed by a single controller.

Let’s consider how it usually works. The completed JMeter workflow is illustrated in the picture below.

The completed workflow of JMeter as show in figure below

The main features and advantages of JMeter.

Price.  JMeter is the open source tool, that allows everybody to use the source code for the development.

Platform independence. As JMeter is 100%  Java desktop app, so it can be run on multiple platforms.

Installation.  No special installation is needed. Everything you should do to run JMeter is to copy and run the *.bat file.

Friendly GUI (Graphic User Interface). Actually, it could be used in 3 modes: GUI Mode, Server Mode, and Command Line Mode. It takes a little time to get familiar with JMeter GUI.

 Test case result visualization. The test result can be displayed in different ways: table, chart, log file and tree.

Multi-threading framework. JMeter helps to provide the parallel and simultaneous sampling of different functions by a separate thread group.

High extensibility level. You have an ability to write your own test cases. Besides that, special visualization plugins are supported for extending your testing.

Simulation. JMeter allows to simulate multiple users with parallel threads and create a heavy load against web application under test.

Multi-protocol support. It supports all basic protocols such as HTTP, SOAP, LDAP, JDBC,  JMS, and FTP.

Script test. Jmeter can be integrated with Selenium and Bean Shell for automated testing.

“Record” and “Playback Record” options. The user activity on the browser could be recorded and simulated in the web application using JMeter.

Integrated real-time support. Apache Tomcat collectors allow providing real-time monitoring.

Easy continuous integration with open source libraries. JMeter is integrated with open source libraries for Jenkins, Maven, and Gradle.

JMeter also has some weak points:

  • There is no ability to record HTTPS communication.
  • It is not capable of application server related statistics monitoring.
  • There is no option of the AJAX traffic interception.
  • The very limited features of the Reporting framework.

 

The Grinder

the grinder-logo

The Grinder is another popular Java-based framework for load testing. Jython, the special Java implementation of the Python, is the default script language. Testsripst also could be written in Clojure.

The main parts of The Grinder:

  • The Grinder Console is the basic interactive development environment (IDE) for test suites developing and editing. It monitors results in real time and controls various agents of The Grinder.
  • The Grinder Agents are the special headless load generators. They can have a number of Workers to create the load.

You can see the system under the load test in The Grinder at the picture below.

the grinder load tseting

The Grinder’s key features:

  • The Grinder is the cross-platform tool. It could be run anywhere you can set up a Java Virtual Machine.
  • Multiple protocols support.
  • Rapid and powerful script development
  • Simple, sliding performance graphs for all transactions in one tab.
  • The users has an ability to write the own plug-ins to a documented interface
  • Flexible parameterization.
  • The user has an ability to load Java API as a test from HTTP web servers, SOAP and REST services.
  • A lot of flexibility for loading and executing third party libraries is offered by The Grinder.

Integration with JIRA.

The Grinder’s disadvantages:

  • Memory failures could be in the large file download scenarios.
  • The Grinder console can not to dynamically adjust the levels of the load being generated by the agents.
  • There are no special additional analysis tools.
  • The special knowledge of Jython or Clojure is needed.

 

Gatling

Gatling-logo

Gatling is the powerful Scala-based “weapon” for load testing. It has two executables: one for test case recording and the other for test case execution.

Actually, Gatling structure is defined in four parts:

  • HTTP protocol configuration helps to define the base URL you will be running your tests against. Besides that, user agent, language header, and connection also could be defined.
  • Headers definition provides the headers for the request which is sent to the server.
  • Scenario definition specifies a group of actions to be executed in order to simulate a user interaction with your application.
  • Simulation definition is aimed to specify the number of users that will concurrently execute load scenario for a particular period of time.

The main features of the Gatling:

  • It works with whichever operating system and any browser.
  • Gatling supports colorful and informative graphical reports containing valuable information for analysis.
  • It can execute its test cases in different test clouds.
  • Gatling could be easily integrated with Jenkins through the Jenkins-plugin and runs your tests through Gradle and Maven with the help of Maven plugin and Gradle plugin.
  • Gatling uses a small amount of memory.
  • It reduces the debugging phase time.
  • Errors and drawbacks are detected in the early part of the development cycle.

The Gatling’s disadvantages:

  • API has been radically changed. So, there is a problem with running test cases written in the old version.
  • Gatling doesn’t allow you to distribute the load between machines.
  • It cannot be scaled horizontally.

 

Locust

Locust is a code-driven, distributed load testing suite built in Python. It is often used for load testing websites (or other systems) and indicating how many concurrent users a system can handle.

Locust based on the original idea. According to it, the swarm of locusts will attack your website during the test. You can define the behavior of each locust (user). The swarming process is monitored from a web UI in real-time and helps you to identify the weak points of your code.

Locust’s key features:

  • Any system can be tested with Locust.
  • It is the cross-platform and easily extendable tool.
  • Hundreds of thousands of users’ behavior can be simulated with Locust.
  • Locust has the very reusable code.
  • It has the high level of customization. There is an ability to combine several requests as one scenario by grouping requests with specific custom points.

The limitations of the Locust:

  • The loss of statistics. Locust gathers and displays stats until requested number of users has been reached. After that, all the statistics are reset and it starts gathering new ones.
  • Testing in Locust never stops. There is no ability to tell it how many hits each user should simulate.
  • It does not get all available resources. For example, Locust will not request an URL unless you explicitly tell it to do this.

 

ApacheBench (ab) – Apache HTTP server benchmarking tool

ApacheBench

ApacheBench is probably the simplest tool for load testing. It is the command line program, which was originally designed for the Apache HTTP Server testing. Indeed, ApacheBench could be effective in testing any web server.

If you are the user of macOS or Linux, you already have it on your computer. If your OS is Windows, just install Apache and you will see ab.exe in your apache/bin folder.

The main merits of the ApacheBench:

  • It is simple to use and easy to learn the tool.
  • No additional installation is needed.
  • ApacheBench is the cost-effective and time-saving tool.
  • There is an ability to set the total number of requests, the number of parallel executed queries, and the time limit for all queries.

Demerits of the ApacheBench:

  • Poor functionality.
  • There is no ability to evaluate the performance of the project from all sides.
  • HTPS protocol is not supported.

 

Taurus

taurus

Taurus is the open source multi-tool testing platform developed by the BlazeMeter. Taurus stands for Test Automation Running Smoothly. The main idea of its creation was to enforce existing free load testing tools like JMeter, Selenium, Grinder, and Gatling.

The main advantages of the Taurus:

  • Taurus has the simple setup and upgrading.
  • YAML or JSON makes easier the versioning of scripts.
  • Taurus is easy combined with JMeter, Selenium, Grinder, and Gatling, Significantly expanding their capabilities.
  • Real-time reports are available during test case execution.
  • Pre-existing scripts could be used in the Taurus.
  • It is closed integrated with the BlazeMeter.
  • Flexible pass/fail criteria are defined in an easy way.
  • Taurus is run everywhere Python and Java can be run.
  • It is integrated into CI tools like Jenkins.

Some disadvantages of the Taurus:

  • Taurus is the resource-intensive tool.
  • Changing hardware configuration is out of its scope.
  • Long-time endurance test cases are not supported.

 

Siege by Joe Dog

siege_joe_dog

Siege is the special utility for the load testing of the HTPP and HTPS protocol. It also can be used as the web server benchmarking tool. Siege is the GNU/Linux-based tool, but now it has been successfully ported to AIX, Solaris, HP-UX, and BSD.

Siege’s key features:

  • HTTP, HTTPS, FTP protocols, cookies, and simple authentication are supported.
  • It is easy compiles on Unix-based OS.
  • Siege is the multithreaded tool.
  • There is an ability of users load emulation.
  • It could be run with multiple IP addresses from the same machine.
  • The simple scenario test cases execution is supported.

Some weak points of the Siege:

  • It couldn’t be run on Windows. Though, you can use Cygwin to do it.
  • Siege is the resource-intensive tool.
  • It is not suitable for serious and large-scale testing in hundreds and thousands of threads.

Commercial load testing tools with free editions

The next group of load testing tool we are going to consider is commercial tools which have free editions. Of course, almost the most commercial tools for load testing have the free trial period. But in this chapter, we will talk about load testing tools, which have some kind of free used editions.

As a rule, there are different kinds of limitations of these load testing tools usage. For example, the number of users, number of executed test cases per month, test duration time etc.

WebLOAD Reviews

webload

WebLOAD is the load testing and analysis tool is distributed by RadView Software. It is real powerful and effective tool for the web apps testing as well as for the mobile testing. Test cases are generated in JavaScript and can be enhanced and edited using various tools in the IDE.

WebLOAD is distributed both in the free and commercial way. It should be mentioned that WebLOAD Free Edition includes almost the all features from the commercial version. The limitations concerned to the number of virtual users (maximum 50 persons) and some exception for Oracle Forms, Flex/AMF, streaming/multimedia features.

So, the small teams have an ability to use its features. Here are some of them:

  • Major web technology support.
  • The user has an ability to generate load on-premise or in the cloud.
  • UI wizards help you to enhance the script.
  • Over 80 configurable report templates are provided by WebLOAD analytics dashboards.
  • The user has an ability to share test case execution results across distributed teams.
  • WebLOAD can be easily integrated with many popular tools: Selenium, Jenkins, Perfecto Mobile, New Relic, and Dynatrace etc.
  • Potential bottlenecks are automatically detected.
  • A wide range of mobile testing tools
  • Combination on-premise and cloud virtual user load.
  • The high-level customer support.
  • Powerful correlation engine.

Some weak points of the WebLOAD:

  • There is no Citrix support.
  • WebLOAD focuses mostly on developers than QA specialist. So, special technical skills are required to its configuration.
  • Upgrade not able to use previous scripts.
  • SAP GUI in not supported.

 

HP Performance Tester (LoadRunner)

LoadRunner

Loadrunner by Hewlett-Packard Enterprise is one of the most widely used load testing tools.

The components of LoadRunner:

  • The Virtual User Generator (VuGen) is aimed to record a script and it enables to develop Vuser (virtual user) scripts for a various application types and communication protocols.
  • The Controller is the main component for controlling the load simulation.
  • Agents Machines (Load Generators or Load Injectors) are the machines virtual users spread across.
  • LoadRunner Analysis and Monitoring is the detailed performance of your test cases.
  • LoadRunner Books Online are the complete study materials for HP LoadRunner.

The main parts of load testing process in the LoadRuner:

  • Plan load test
  • Create Vuser Scripts
  • Define Scenario
  • Run Scenario
  • Analyze results

LoadRunners advantages:

  • Interactive user transaction simulation.
  • A wide range of applications support.
  • Powerful mobile testing tools.
  • It reduces the systems requirements and human intervention.
  • LoadRunner offers record and playback or scripting.
  • Effective utilization tracking tool.
  • Single point execution.
  • Optimal usage of load generator farm.
  • LoadRunner quickly and accurately detects the root cause of application performance problems.

Some disadvantages of the LoadRunner:

  • Multiple specific scenarios cannot be launched from the command line.
  • It requires continuous support for technical assistance during execution.
  • A lot of disk space is required for LoadRunner installation.
  • The Controller and VUGen work only in Windows.

 

Rational Performance Tester by IBM

rational_perfomance_tester_ibm

Rational Performance Tester (RPT) is a performance and load testing tool developed by IBM Corporation. It can be used for a web application or a server based application where there is a process of input and output is involved. RPT creates a demo of the original transaction process between the user and the web service.

Key merits of the Rational Performance Tester:

  • A broad base of applications such as HTTP, SAP, Siebel, SIP, TCP Socket and Citrix are supported.
  • No special programming knowledge is required.
  • It provides the diagnostics of Websphere and Weblogic application servers.
  • It is run with large multi users test cases.
  • Real-time reporting for immediate performance problem identification.
  • The flexible graphical test schedule where the load can be specified by proportional groups of users.
  • Environment and Platform support.
  • The wide ranges of the root cause analysis tools.

Some demerits of the Rational Performance Tester:

  • Ramp up or Ramp down of resources is not possible in the Rational Performance Tester.
  • It doesn’t support Java Applet based applications.
  • There is no ability for scheduling.

 

BlazeMeter

blazemeter

BlazeMeter is one of the most wide used platforms for the load testing. It is offered both in free and commercial editions. You can run maximum 10 tests with no more than 50 users for free. The only one load generator is available in the free version of the Blazemeter.

Key features of the Blazemeter:

  • Blazemeter is 100% compatible with Apache JMeter.
  • It is easy integrated with other popular open source tools: Selenium, Locust, and Gatling.
  • There is an ability to run test cases with the up to 1 million users.
  • There is the direct access to the Blazemeter from the web.
  • The traffic from mobile devices with varied signal strengths and various networks can be simulated by Blazemeter.
  • Scalable low testing from anywhere.
  • You can download the testing results for off-line using.
  • It supports Sandbox tests for validation the scripts for the limited number of users for the limited amount of time.
  • Blazemeter has high-customized and well-illustrated reports which are easy to be share.

Some of Blazemeter’s limitations:

  • There is no ability to modify the users’ load during running the test case.
  • You cannot use Blazemeter with Flex, Silverlight or Multimedia protocols.
  • Endurance test cases, which run for more than 3 hours are not supported.

 

NeoLoad by Neotys

neoload

NeoLoad is the popular tool for analysis the performance of the website or application under the heavy load. It works with the high efficiency both with desktop and mobile applications. NeoLoad is written in Java. Test case execution for not more than 50 virtual users is free.

The key features and benefits of NeoLoad are:

  • It supports the latest web and mobile technologies.
  • NeoLoad cross-platform capabilities allow the loads to be generated on a wide variety of platforms (Unix, Windows, Linux…).
  • NeoLoad gets a summary of the test case execution and then examining its details using the graphs and statistical tables. Statistics is got from your servers (CPU, Memory, etc).
  • Real world testing scenarios help you to catch performance problems early.
  • Cloud integration, live monitoring, and real device integration is supported not only on Android or iOS but also on Windows Phone and Blackberry.
  • An ergonomic graphical interface that reduces the test execution time.
  • There is an ability to define Virtual User behavior and set scenario parameters such as load policy (constant, ramp-up, peak or custom) or the number of Virtual Users to be simulated in the test.
  • It supports Selenium script conversion
  • Performance monitors (CPU, memory, disk usage, and so on) are set for your servers.
  • Page content is checked under the load.
  • There is an ability to record any mobile application directly from any mobile device with the proxy mode or DNS tunnel mode.
  • It records HTTP traffic between browser and server.

Some drawbacks of NeoLoad:

  • It does not function in Mac OSX.
  • There is no access page DOM
  • There is no facility to create start and end transactions to capture the response time.

 

LoadUI by SmartBear

loadUI

LoadUI is the famous tool used mostly for the web services load testing. It is distributed both in free and licensed version. If you want to use it with higher efficiency, you should use it with another free SmartBear product SoapUI.

The main advantages of LoadUI:

  • Multiple performance strategies.
  • Drag and drop interface.
  • API support.
  • Parallel and isolated load testing execution.
  • Real-time feedback about your test cases results.
  • The ability to add new data even as the application is being tested.
  • Automatically updated interface.
  • It works with the major operating system.
  • LoadUI visual reports are presented in a very easy to understand graphical format.
  • There is an ability to create, configure and modify test cases during the execution.

Some weak points of the LoadUI:

  • Scripts created in Pro versions can’t be used in free versions.
  • Old projects could be imported with some limitations.
  • It is weak and limited for web load testing.

 

CloudTest by Soasta

CloudTest by Soasta

CloudTest is the comprehensive tool for providing load and performance tests of web and mobile and apps. It can be hosted on one or more physical servers as well as in the cloud.

Amazon Web Services and Rackspace cloud services providers are used by CloudTest for traffic simulation.

CloudTest Lite is its free trial version. Of course, there are a lot of limitations in this edition:

  • No 24/7 support.
  • There is no global, cloud-based load generation
  • The maximal number of concurrent virtual users is 100 persons.
  • Only one load generator could be used etc.

Anyway, this is powerful even in the free edition version.

The main advantages of the CloudTest:

  • It provides end-to-end mobile performance testing.
  • Real-time analytics.
  • Test cases are quickly built, edited, executed and analyzed.
  • It is easy integrated with JMeter and other often used automated testing tools.
  • The high accuracy of the simulating web and mobile user traffic.
  • It offers 360° performance reports of the real user monitoring.
  • There is an ability to add or drop users during the test case execution.

Some disadvantages of the CloudTest:

  • The free edition is too limited.
  • SLA configuration is not available in CloudTest.
  • There are no hardware control capabilities.
  • There is no ability to use CloudTest for automation Flex, Multimedia or Silver light based apps.

 

LoadStorm

loadstorm-logo

LoadStorm is the easy-to-use and cost-effective tool which can be used for load testing of the web and mobile applications. It has open source and commercial editions. You can run test cases with no more than 50 virtual users for free.

LoadStorm’s key features:

  • No special installation is needed.
  • It takes a few minutes to record a script.
  • Worldwide distributed cloud platform for test case running
  • Well-customized reports.
  • Analytics in the real-time mode.
  • Performance intelligence during running test cases.
  • Constant on-line support
  • It provides traffic geographic distribution.

Some drawbacks of the LoadStorm:

  • LoadStorm can test only web applications that can be reached from the Amazon cloud.
  • OS Windows is required for its using.

 

Smart Meter

smartmeter load testing tools

Smart Meter has the very powerful free version, though it is a commercial load and performance testing tool. Unlike to other commercial tools from our list, you have an ability to run test cases with 50 virtual users for free. Also, you have 2 load generators for your testing.

The main advantages of the SmartMeter:

  • Test reports, which are generated automatically, contain the full information about the test and its results.
  • Multifunctional test scenario recorder.
  • There is an ability to provide load testing from different locations.
  • It is easy integrated with Jenkins, Bamboo, and Maven etc.
  • Multiple monitors display test cases execution results in real time mode.
  • There is an ability to combine load test cases with Selenium.
  • Permanent test environment monitoring.

Some limitations of the SmartMeter:

  • It is not easy to learn tool.
  • The maximal test case execution time in the free version is only 15 min.

 

Fully commercial testing tools

Finally, commercial tools with only paid version will be considered. Naturally, all of them have some short free trial period. But it is cannot be defined as open source editions.

Silk Performer from Micro Focus

silk_performer

Silk Performer (formerly known as Borland) is the powerful tool for the load testing of web, mobile and enterprise apps. It helps to identify the weak points of the application under the peak customer load using integrated diagnostics and trending reports.

Key features of the Silk Performer:

  • It supports all major web environments: HTML5, AJAX, Responsive Web Design, Adobe Flash/Flex, and Microsoft Silverlight.
  • A wide range of the mobile web & native application testing tools.
  • Minimum hardware resources are required for virtual user simulation.
  • It generates well-customized reports with tables and graphs.
  • Silk Performer has user-friendly correlation and parameterization.
  • It supports integrated server monitoring.
  • Visual root cause analysis helps to detect the bottlenecks.
  • It has the simple script correlations.
  • Unlimited cloud scalability.
  • There is an ability to test your applications across a variety of the mobile platforms and connection standards: iOS, Android, BlackBerry, GPRS, HSPA+, EDGE, and LTE.

Some weak points of the Silk Performer:

  • There is a poor integration with deep diagnostics tools for end to end performance monitoring.
  • It is designed only for Windows.
  • Much memory is required for load testing with the Silk Performer.
  • There could be a problem with the setting of the ramp down pattern.

 

WAPT

wapt load testing tools

WAPT is the popular tool, which is equally effective for load testing web services, mobile applications, and all types of websites from e-shops to corporate ERP and CRM systems.

Making just a few clicks, you can create and execute test cases with thousands of virtual users. WAPT allows you to control over how to customize these virtual users.

The main merits of WAPT:

  • It is easy to install.
  • Fast test recording and design.
  • WAPT supports RIA technologies and mobile apps.
  • Advanced error reporting based on different indicators: network level errors, HTTP response codes, timeouts, server delays, and response validation by page content.
  • Web sites security testing.
  • There is an ability to include the JavaScript code execution into the virtual user profiles.
  • Well-illustrated and detailed reports are available even at the runtime.
  • Integrated log viewer allows you to browse step by step the each executed session.

Demerits of WAPT:

  • OS Windows is required to install this tool.
  • WAPT has no scripting ability.

 

Testing Anywhere by Automation Anywhere

Testing Anywhere focuses on the websites, controls, objects, and GUI front-ends load testing.

There are five methods of test case creation in the Testing Anywhere:

  • Web recording
  • Object recording
  • Image recognition
  • Smart recording
  • Editor

Key features of the Testing Anywhere:

  • No programming knowledge is required.
  • Test cases are easily built and edited.
  • There is an ability to perform migration testing.
  • Visual test cases automation
  • Automatic language objects identification. It works with Java, Net, WPF, Silverlight, HTML, Flash, and Flex.
  • Testing Anywhere offers the single platform for testing.

Some limitations of the Testing Anywhere:

  • It supports the only Windows operating system.
  • There could be some problems with testing iOS apps.
  • The testing platform is not provided for applications written in the older languages.

 

StormRunner Load by Hewlett Packard Enterprise

stormrunner

StormRunner Load is the latest load testing tool among product offered by Hewlett Packard Enterprise. It can be used both for the web and mobile testing. StormRunner is the cloud/SaaS based service, which makes it simple to access and leverage.

StormRunner’s advantages:

  • It is easy and simple to use.
  • HTTP/HTML (both web and mobile), SAP Web, Java, Flex, TruClient Web, TruClient Native Mobile and TruClient Mobile Web protocols are supported by the StormRunner.
  • Minimalistic and responsive interface.
  • High scalability. There is an ability to run test cases with more than one million geographically distributed web and mobile users.
  • Virtual users can be removed or added during the test case execution.
  • Load testing infrastructure is automatically deployed.
  • It has the tight integration with Jenkins, Dynatrace, Gatling, Docker, AWS Code Pipeline, and New Relic.
  • There is an ability of performance center servers decommission.
  • It offers load generators at several different geographical locations.
  • Fast detection of the problem.
  • Well-customized analytical report via Network Virtualization.

Some limitations of the StormRunner:

  • There is no ability to use network analytics with JMeter scripts.
  • There is no FTP for bulk uploads.
  • There is no option to schedule tests to start at a later date.
  • Assets other than scripts couldn’t be attached.

So, now you are informed about the best tools for the load testing. As you have read, all of them have their own merits and limitations of the usage. Some of them are preferable for small teams, other for big developers companies. Anyway,  it depends on varieties of factors which one should be chosen in the particular situation.

Do you agree with our list?  Maybe you know the tool deserves a place in our rating. Share your opinions and experience in the comments below.