header

Best automation testing tools

Yana Gusti 08/05/2017

What is Automation Testing?

While the manual testing executed by successive steps, the automation testing using automation tools, which can be conveniently administered test data, test system to compare the expected and actual results and generate detailed test reports.
Because the development cycles require performing the same tests repeatedly testing requires a significant investment of money and resources.

By using test automation tool you can record test suit and replay it as needed. After the automated test set, no need for human intervention and for through is possible to improve return on investment test automationThe main goal of automation testing is to reduce the number of tests to run manually.

Benefits of Automation Testing

 Following are benefits of automated testing:

  • Faster than the manual testing
  • Wider test coverage of application features
  • Reliable in results
  • Ensure Consistency
  • Saves Time and Resources
  • Improves accuracy
  • Human Intervention is not required while execution
  • Increases effectiveness
  • Reusable test scripts
  • Test thoroughly and Frequently

Drawbacks of Automation Testing

  • False sense of quality
  • Not reliable
  • Automation is not testing
  • Maintenance Time and Effort
  • Slow feedback
  • Not many bugs found

Automation testing frameworks

Selenium

logo testing tool

Selenium is a Java application that can analyze files of a specific structure in order to find commands for manipulating the browser and commands for performing certain actions and checks.
Being one of the most popular automated testing tools for WEB on different platforms, Selenium has support from some browsers that make Selenium a native part of their browser.
Selenium was originally developed by Jason Huggins in 2004 as an internal tool at ThoughtWorks.

Stable release 3.0 ( October 13, 2016)

Pricing: Free, open source.

Features

  • Selenium is pure open source, freeware and portable tool.
  • Selenium supports a variety of languages that include Java, Perl, Python, C#, Ruby, Groovy, Java Script, and VB Script. etc.
  • Selenium supports many operating systems like Windows, Macintosh, Linux, Unix etc.
  • Selenium supports many browsers like Internet Explorer, Chrome, Firefox, Opera, Safari etc.
  • Selenium can be integrated with ANT or Maven kind of framework for source code compilation.

Read more about Selenium

Watir

logo watir

 

Watir is an automation testing tool that using to testing of web apps. It is an open source library of Ruby and anyone can get it on GitHub. It’s very helpful for automating web browsers and allows you to create simple tests
that easy maintain.
Unlike testing tools based on the HTTP protocol that simulates browser requests, Watir simulates user interaction with the browser using the OLE protocol, which is based on the COM architecture.Ruby supports OLE, which allows the automation of Microsoft Internet Explorer.

Stable release 5.0 (October 5, 2013).

Pricing: Free, open source.

Features

  • Tests any language-based web application
  • Cross-browser testing
  • Compatible with business-driven development tools like RSpec, Cucumber, and Test/Unit
  • Tests web page’s buttons, forms, links, and their responses

Read more about Watir

SoapUI

Soup UI Logo

SoapUI is a crossplatform engineering tool for functional testing, which was created for automatically be used to test the interface, so SOAP and REST for secure access to the oldest addons.It is built entirely on the Java platform and uses Swing for the user interface. This means that SoapUI is crossplatform.

Today, SoapUI also supports IDEA, Eclipse, and NetBeans. SoapUI can test SOAP and REST web services, JMS, AMF, as well as make any HTTP(S) and JDBC calls.

 Stable release 5.3 (December 5, 2016)

Pricing: Free, open source.

Features

  • User-Friendly GUI
  • Easy for Functional Testing
  • Vulnerability Testing
  • Load Testing using LoadUI
  • Automation with Groovy

Read more about SoapUI

Test-driven development 

JUnit

JUnit has been very important in test-driven development and testing framework at all. This testing framework using for testing applications written in Java and help to easy design repeatable tests. A research survey performed in 2013 across 10,000 Java projects hosted on

A research survey performed in 2013 across 10,000 Java projects hosted on GitHub found that JUnit was the most commonly included external library. Each library was used by 30.7% of projects.

Stable release 4.12 (December 4, 2014)

Pricing: Free, open source.

Features

  • Simple framework for writing automated, self-verifying tests in Java
  • Support for test assertions
  • Test suite development
  • Immediate test reporting

Read more about JUnit

TestNG

TestNG is also for the Java programming language application. This testing framework was inspired by JUnit and NUnit. It provides few new features for writing tests that make it more powerful convenient and easy to use.
TestNG was created to cover multiple types of tests, such as unit, functional, end-to-end and integration tests.

Stable release 6.9.12(June 21, 2015)

Pricing: Free, open source.

TestNG’s main features include:

  • Annotation support.
  • Support for parameterized and data-driven testing
  • Support for multiple instances of the same test class
  • Flexible execution model. TestNG can be run either by Ant via build.xml, or by an IDE plugin with visual results. There isn’t a TestSuite class, while test suites, groups and tests selected to run are defined and configured by XML files.
  • Concurrent testing: run tests in arbitrarily big thread pools with various policies available, and test whether the code is multithread safe.
  • Embeds BeanShell for further flexibility.
  • Default JDK functions for runtime and logging.
  • Dependent methods for application server testing.
  • Distributed testing: allows distribution of tests on slave machines.

Read more about TestNG

NUnit

NUnit is an open environment for a unit-testing framework for .NET. It was ported from the Java language (the JUnit library). The first versions of NUnit were written in J #, but then all the code was rewritten in C # using such .NET innovations as attributes.
There are also known extensions of the original NUnit package, most of them also with open source. NUnit.Forms complement NUnit by testing the Windows Forms user interface elements. NUnit.ASP performs the same task for interface elements in ASP.NET.

Stable release 3.2 ( 5 March 2016)

Pricing: Free, open source.

Features

  • Tests can be run from a console runner, within Visual Studio through a Test Adapter, or through 3rd party runners.
  • Tests can be run in parallel.
  • Strong support for data-driven tests.
  • Supports multiple platforms including .NET Core, Xamarin Mobile, Compact Framewor, and Silverlight.
  • Every test case can be added to one or more categories, to allow for selective running.

Read more about NUnit

GoogleTest

Google Test is a library for the C++ programming language apps that based on the xUnit architecture. It’s executing testing individual parts of the program such as classes, functions, modules. They are checked separately from each other, in isolation and it called unit testing.

It can be compiled for a variety of POSIX and Windows platforms, allowing unittesting of C sources as well as C++ with minimal source modification. One of the biggest advantages of this tool is a possibility to run tests as one and all at once at the same time, which makes testing more convenient and versatile for testers and coders.

Stable release 1.8( August 22, 2016)

Pricing: Free, open source.

Features

  • An XUnit test framework.
  • Test discovery.
  • A rich set of assertions.
  • User-defined assertions.
  • Death tests.
  • Fatal and non-fatal failures.
  • Value-parameterized tests.
  • Type-parameterized tests.
  • Various options for running the tests.
  • XML test report generation.

Read more about GoogleTest

Testoob

Testoob is an acronym which stands for Python Testing Out Of (The) Box. This tool is an advanced unit testing framework for Python language apps. It’s very convenient because easy integrates with existing PyUnit (module “unit test”) test suites. You can get it on GitHub.

Stable release 1.12 (9 September 2006)

Pricing: Free, open source.

Features:

  • Work with PyUnit tests as-is, no changes necessary
  • Filter which tests to run with regular expressions
  • Output test results as XML/HTML/PDF
  • Test skipping
  • Color output on a terminal
  • Run pdb post-mortem on failed tests
  • Profiling support
  • Run tests in parallel in threads or processes and more!

Read more about TestToob

csUnit

csUnit Logo

csUnit is a framework for unit testing. It works with compliant language applications. csUnit was designed to work with any .NET compliant languages.

It has specifically been tested with C#, Visual Basic .NET, Managed C++, and J#. The framework is open source and comes with a flexible license that allows costfree inclusion in commercial closedsource products as well.
The tool offers a native GUI application, a command line, and add-ins for Visual Studio 2005 and Visual Studio 2008.

Stable release 2.6 (March 28, 2009)

Pricing: Free, open source.

Features

  • Standard unit testing features such as test, fixture, setup, teardown, ignore, an expected exception, etc.
  • Easy to use graphical user interface
  • Recipes for combining several test assemblies into one test suite
  • Add-in for Visual Studio 2005 and Visual Studio 2008
  • Search capabilities across tests, output, and statistics
  • Statistics per test to create performance baseline
  • Categories to group tests for execution
  • Support for applications targeting .NET 2.0, .NET 3.0, and .NET 3.5
  • Works with any .NET language (C#, VB.NET, Managed C++, etc.)
  • Runs on Windows Vista SP 1 or Windows XP SP 3

Read more about csUnit

HtmlUnit

Htmlunit is a web browser which was created in Java and ensures access to structure and detail within the received web pages.
Using this tool you can imitate emulates the behavior of the browser like fill the fields, submit the form and clicking hyperlinks to websites.
It can deal with HTTPS security, basic HTTP authentication, automatic page redirection and other HTTP headers.
The goal is to simulate real browsers such as Chrome, Firefox ESR 38 and Internet Explorer.

Stable release 2.23 ( July 27, 2016)

Pricing: Free, open source.

Features

  • Provides high-level API, taking away lower-level details away from the user.
  • Compared to other WebDriver implementations, HtmlUnitDriver is the fastest to implement.
  • It can be configured to simulate specific Browser.

Read more about HtmlUnit

PHPUnit

php-unit-logo-big

One of the most popular unit testing framework for PHP projects is PHPUnit. At the first time, it was an instance of xUnit architecture and after became popular with JUnit. This framework was designed by Sebastian Bergmann and hosted on GitHub. The main goal of the tool allowed developers to find mistakes in their code quickly. And they are able to assert that no code regression has occurred in other parts of the code base.

In order to show that the individual parts are correct, this tool makes their goal of unit testing is isolate each part of the program. As follows, a unit test can find problems in the first times of development cycle. PHPUnit can output test results in a number of different formats, including JUnit XML, Test Anything Protocol, JSON, and TestDox.

Stable Release 6.0 (February 3, 2017)

Pricing: Free, open source.

Features

  • PHPUnit is the standard; most frameworks use it (like Zend Framework (1&2), Cake, Agavi, even Symfony is dropping their own Framework in Symfony 2 for PHPUnit).
  • PHPUnit is integrated into every PHP IDE (Eclipse, Netbeans, Zend Stuide, PHPStorm) and works nicely.
  • PHPUnit works fine with every continuous integration server since it outputs all standard log files for code coverage and test reports.
  • PHPUnit is actively maintained, stable and works great for every codebase, every scenario and every way you want to write your tests.

Read more about PHPUnit

Simple Test

SimpleTest is an open source unit test framework for the PHP programming language and was created by Marcus Baker. The test structure is similar to JUnit/PHPUnit. SimpleTest supports mock objects and can be used to automate the regression testing of web applications with a scriptable HTTP Client that can parse HTML pages and simulate things like clicking on links and submitting forms.

Stable release 1.1.7 ( September 21, 2015)

Pricing: Free, open source.

Read more about SimpleTest

Business-driven development 

Cucumber

It is a testing business development driven that helps testers to test the software. Cucumber does automated tests for acceptance that writing in BDD style. As such, Cucumber allows the execution of feature documentation written in business-facing text. Capybara, which is a part of the Cucumber testing framework, is an example of a web-based test automation software.

Cucumber is written in the Ruby programming language and was originally used exclusively for Ruby testing as a complement to the RSpec BDD framework. Cucumber now supports a variety of different programming languages through various implementations. For example, Cuke4php and Cuke4Lua are software bridges that enable testing of PHP and Lua projects, respectively. Other implementations may simply leverage the Gherkin parser while implementing the rest of the testing framework in the target language.

Stable release 1.3.15 (May 9, 2014)

Pricing: Free, open source.

<VIDEO>

Read more about Cucumber

Behave

behave_logo2

Behavior-driven development (or BDD) is an agile software development technique that encourages collaboration between developers, QA and non-technical or business participants in a software project.

Stable release 1.2.6

Pricing: Free, open source.

Main features

  • Establishing the goals of different stakeholders required for a vision to be implemented
  • Drawing out features which will achieve those goals using feature injection
  • Involving stakeholders in the implementation process through outside–in software development
  • Using examples to describe the behavior of the application, or of units of code
  • Automating those examples to provide quick feedback and regression testing
  • Using ‘should’ when describing the behavior of software to help clarify responsibility and allow the software’s functionality to be questioned
  • Using ‘ensure’ when describing responsibilities of software to differentiate outcomes in the scope of the code in question from side-effects of other elements of the code.
  • Using mocks to stand-in for collaborating modules of code which have not yet been written

Read more about Behave

Automation testing tools

SeeTestAutomation

SeeTestAutomation is the special tool that allows running automated mobile app testing not only on iOS or Android but also on BlackBerry and Windows Phones on real devices as well on emulators.

Pricing: Free, open source.

Main features

  • Perform mobile test automation using real devices and emulators
  • Create a test on one device and run it on any other device model or operating system
  • Increase coverage with full peripheral testing capabilities – GPS, Touch ID, camera and audio
  • Start your mobile test automation quickly with a simple plug, record and play tool
  • Enhance your mobile test automation by Integrating into all your major ALM environments
  • Integrate with WebDriver (Selenium), HP UFT/QTP, JUnit, Microsoft Visual Studio and more
  • Integrate with any continuous integration environment – HP Quality Center (ALM), Jenkins and more

Read more SeeTestAutomation

Rapise

Rapise logo

Rapise is the next generation software test automation platform that leverages the power of open, extensible architecture to provide the most rapid and flexible functional testing tool on the market today. Rapise’s unique Learn and Go testing method is much more time efficient than traditional record-and-playback processes. Objects can be edited during the learning process instead of waiting for all objects to be recorded then edited.

Rapise provides powerful built-in support for the following environments: Java, Google Web Toolkit, Yahoo User Interface, Web applications including AJAX, Windows Forms including Infragistics, DevExpress, Telerik, Adobe Flash/Flex including AIR, Microsoft .NET, Win32 applications and WPF. With Rapise you can use the same environment to test all these technologies.

Rapise’s cross-browser testing capabilities are best in class, with support for multiple versions of Microsoft Internet Explorer, Mozilla Firefox and Google Chrome. Choose Rapise to test web applications in all of the most widely used browsers. Record or create one test script and execute the same script without modification across the major browsers. Built-in support for GWT and YUI. Fully Integrated with SpiraTest for Test Management.

Stable release 5.0(January 12, 2017)

Pricing: Free, open source.

Read more about Rapise

Ranorex

Ranorex is a GUI test automation framework for testing of desktop, web-based and mobile applications. Ranorex is provided by Ranorex GmbH, a software development company for innovative software test automation solutions. Ranorex does not have a scripting language of its own, instead of using standard programming languages such as C# and VB.NET as a base.

Stable release 6.2.1 (January 18, 2017)

Pricing: Free, open source.

The main advantages of Ranorex using:

  • Easy installation for non-programmers.
  • “Click & Go” XPath editor.
  • UI object map editor.
  • Support for Dynamic IDs.
  • Easy maintenance.
  • Easy execution of tests with executable (.exe) files.
  • Professional Support Services.

<VIDEO>

Read more about Ranorex

qTest eXplorer

qTEst logo

qTest encompasses the full lifecycle from requirements and test case design, through test execution and defect tracking, to real-time reports on status and quality-metrics. This SaaS solution allows you to get up and running in a matter of minutes. It supports both scripted and unscripted (exploratory) testing approaches.

Stable release 8.1 

Pricing: Free, open source.

Main Features:

  • Support for Agile testing
  • Intelligent multi-screen capture
  • Auto recording of all clicks and screen actions
  • Cross application capture
  • Ability to add notes and annotation
  • Test case and defect submission

 Read more about qTest eXplorer

Telerik Test Studio

Telerik Test Studio is a software testing tool that based on Windows. It was designed for web and desktop functional testing, software performance testing, load testing and mobile application testing.
This tool can be used as a plugin for Visual Studio as a standalone app that uses the same repositories and file formats. Test Studio supports HTML, AJAX, Silverlight, ASP.NET MVC, JavaScript, and WPF. Test Studio supports test execution in Internet Explorer, Firefox, Safari, and Chrome.

Stable release 2012.2 (R2 2012)(September 20, 2012)

Pricing: Free, open source.

Test Studio’s features include:

  • Scriptless test recording and playback
  • Cross-browser test execution – Internet Explorer, Firefox, Chrome and Safari (web browser)
  • Support for HTML, AJAX, Silverlight, WPF and ASP.NET MVC application testing
  • Element abstraction and reuse
  • Integration with Visual Studio 2010 / 2012, MS Team Foundation Server
  • NUnit, MbUnit, XUnit integration
  • Sentence-based UI validation
  • Continuous integration with Microsoft Build Server, CruiseControl, and TeamCity
  • Visual Debugger
  • Test customization in C# and VB.NET
  • Exploratory Testing
  • Manual Testing
  • Integration with HP Quality Center
  • Built-in testing framework
  • DOM Explorer
  • Fiddler web debugger integration
  • Automated data-driven testing
  • Bug-tracking tools integration

Get from http://www.telerik.com/teststudio

Read more about TestStudio

For API automated testing

Apache JMeter

A load testing tool developed by the Apache Software Foundation. Although JMeter was originally developed as a web application testing tool, it is now able to perform load tests for JDBC connections, FTP, LDAP, SOAP, JMS, POP3, IMAP, HTTP, and TCP.

To create a large number of queries using several computers when managing this process from one of them. The architecture that supports plug-ins of third-party developers allows you to supplement the tool with new features.

The program implements mechanisms for authorizing virtual users, user sessions are supported. Logging of test results and various visualization of results in the form of diagrams, tables, etc. are organized. Apache JMeter is a solution for load testing for analyzing and measuring the implementation of various services with a focus on web apps.

Stable release 3.1 (November 19, 2016)

Pricing: Free, open source.

Features

  • Being an open source software, it is freely available.
  • It has a simple and intuitive GUI.
  • JMeter can conduct load and performance test for many different server types − Web – HTTP, HTTPS, SOAP, Database via JDBC, LDAP, JMS, Mail – POP3, etc.
  • It is a platform-independent tool. On Linux/Unix, JMeter can be invoked by clicking on the JMeter shell script. On Windows, it can be invoked by starting the jmeter.bat file.
  • It has full Swing and lightweight component support (precompiled JAR uses packages javax.swing.* ).
  • JMeter store its test plans in XML format. This means you can generate a test plan using a text editor.
  • It’s full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
  • It is highly extensible.
  • It can also be used to perform automated and functional testing of the applications.

Read more about JMeter 

POSTMAN

Automation postman

Postman is a Google Chrome application for interaction with the HTTP API. It provides a user-friendly graphical interface for building queries and reading responses. For people behind Postman also offer an additional package called Jetpacks, which includes some automation and testing Javascript library. Although they do not replace your focused unit tests, they breathe new life into the testing feature of the application. This makes it extremely valuable for functional testers or developers who like to check outside to inside.

Read more about PostMan

 

Conclusion

The right selection of automation tool, testing process, and team, are important players for automation to be successful. Manual and automation methods go hand-in-hand for successful testing.