Software Test Automation Tools

Automated software testing is becoming more and more important for many software projects in order to automatically verify key functionality, test for regressions and help teams run a large number of tests in a short period of time. Many teams (especially larger projects) still require a significant amount of manual functional testing in addition to automated testing, either because of the lack of sufficient resources or skills to automate all tests.

There are various tools that help software teams build and execute automated tests. Many teams are actively using unit tests as part of their development efforts to verify critical parts of their projects such as libraries, models and methods. Historically, testing user interfaces of desktop-based applications via automated tests have been more challenging, and currently available tools for this are usually commercial and quite expensive.

With the growing number of web-based applications this is changing, however, as verifying and testing web-based interfaces is easier and there are various tools that help with this, including free open source projects. Please see below for a list of popular and useful tools, projects, books and resources to get started with automated software testing.

Automated Web Testing Tools 7 Tools

Automatically testing your web application is a good way to ensure that new versions of your application don't introduce bugs and regressions. Automation of your web application testing also allows your development team to make changes and refactor code with more confident, as they can quickly verify the functionality of the application after every change.

However, actually building automated tests for web applications can be challenging because the user interface of your application might change regularly, because of incompatibilities between browsers and because you usually need to support various server or client platforms. The following tools make it easier to build and execute automated tests for your web application.

Open Source

Selenium

Selenium is a popular automated web testing tool and helps you automate web browsers across different platforms. Selenium has the support of some of the largest browser vendors who have taken steps to make Selenium a native part of their browser.

Open Source

Watir

Watir is a set of Ruby libraries for automating web browsers and allows you to write tests that are easy to read and maintain. Watir drives browsers the same way people do (it clicks links, fills in forms, presses buttons etc.) and also checks results such as whether expected text appears on the page.

Open Source

Windmill

Windmill is a web testing tool designed to help testers automate and debug web applications. It comes with a cross-browser test recorder, JavaScript integration and an interactive shell to automate web browsers.

Commercial

Ranorex

Ranorex allows you to automate your web application testing (among other things) and both record user interactions and play them back to execute your tests. Ranorex is one of the more popular commercial tools to build and run automated web and GUI tests.

Open Source

SoapUI

SoapUI is a cross-platform functional testing tool. It has been specifically designed to help automatically test APIs such as SOAP and REST interfaces to ensure the interoperability of different applications.

Open Source

Sahi

Sahi is a tool for automation of web application testing. Sahi is available as a free open source edition as well as a commercial Pro edition.

Commercial

Tellurium

Tellurium is a web automation tool that allows you to design and write your automated tests using plain English without any scripting or programming experience.

Automated GUI Testing Tools 5 Tools

Building robust automated GUI tests for desktop applications (e.g. on Windows or Mac systems) is quite difficult, as small changes to the user interface can often result in broken tests. The following tools help you build and execute robust GUI tests for various platforms and operating systems.

Commercial

Squish

Squish is a GUI testing tool for various platforms, including QT, native Windows and Mac applications. Squish allows testers and developers to build automated tests using familiar scripting languages such as JavaScript, Perl, Python and Ruby.

Commercial

Ranorex

Ranorex allows you to automate your desktop applications (among other things) and both record user interactions and play them back to execute your tests. Ranorex is one of the more popular commercial tools to build and run automated GUI and web tests.

Commercial

TestComplete

TestComplete is an automated testing tool for the Windows platform. It allows you to record, script and run GUI tests for applications built using different frameworks and languages, such as .NET or C++.

Commercial

Test Studio

Test Studio is an automated functional and load testing tool that helps you test applications on various platforms built using different frameworks and tools.

Commercial

eggPlant

eggPlant is a GUI test automation tool for professional software applications and enterprise teams. It can be used to automate different application types, such as .NET, Java and Flash applications.

Unit Testing Frameworks 9 Tools

See below for a list of popular unit testing frameworks and tools for major platforms and programming languages. These frameworks can be used by programmers to test specific functionality in libraries and applications. Unit tests can then be used to automatically test new versions and builds as part of an automated build system or deployment process.

.NET, Open Source

NUnit

NUnit is a unit-testing framework for all .Net languages. It was initially ported from JUnut to .NET and has been redesigned to take advantage of many .NET language features.

.NET, Open Source

xUnit.net

xUnit.net is a community-focused unit testing tool for the .NET Framework written by the original inventor of NUnit. xUnit.net is the latest technology for unit testing C#, F#, VB.NET and other .NET languages.

Python, Open Source

PyUnit / unittest

The Python unit testing framework, sometimes referred to as "PyUnit", is a Python language version of JUnit. It's part of the Python framework and supports test automation, sharing of setup and shutdown code for tests and various other features.

Java, Open Source

JUnit

JUnit is a simple unit testing framework to write repeatable tests in Java. JUnit has been important in the development of test-driven development and is one of the standard testing frameworks for Java developers.

Java, Open Source

TestNG

TestNG is a Java testing framework inspired by JUnit and NUnit and introduces some new functionalities that make it more powerful and easier to use. TestNG is designed to cover multiple categories of tests, including unit, functional, end-to-end and integration tests.

PHP, Open Source

PHPUnit

PHPUnit is a popular framework for unit testing in PHP projects. It provides both a framework that makes it easier to write tests as well as the functionality to easily run, execute and analyze tests and results.

PHP, Open Source

Symfony Lime

Lime is a unit and functional testing framework built for the popular Symfony PHP web application framework. The framework is designed to have readable output from tests, including color formatting, by following the Test Anything Protocol.

Ruby, Open Source

Test::Unit

Ruby comes with its own standard unit testing framework as part of the Test::Unit namespace and can be used to define basic pass/fail tests and group tests. The framework also comes with tools to run single or whole groups of tests.

Ruby, Open Source

RSpec

RSpec is a testing tool for the Ruby programming language. Born under the banner of behaviour-driven development, it is designed to make test-driven development more productive and enjoyable.

Automated Testing Cloud Services

Cloud testing services allow you to run your automated web, mobile and unit tests in different environments and on multiple machines without having to build your own testing infrastructure. The below list of services provides a good overview of popular online Selenium and continuous integration services that are easy and affordable to use.

Service Type Description Pricing
Sauce Labs Selenium & Mobile Cross-browser Selenium & mobile testing Starting @ $149/month
TestingBot Selenium Cross-browser Selenium testing Starting @ $40/month
Gridlastic Selenium Cross-browser Selenium testing Starting @ Free
CircleCI Continuous Integration Continuous integration service & unit testing Starting @ $19/month
Tddium Continuous Integration Continuous integration service & unit testing Starting @ $15/month
CloudBees Continuous Integration Continuous integration service & unit testing Starting @ Free
Mailosaur Email Testing Service to integrate emails in automated tests Starting @ Free

Software Test Automation Books 6 Books

The following books provide a good starting point to learn more about automated software testing in general, test-driven and behavior-driven development as well as integrating automated tests into your overall testing strategy.

Test Driven Development: By Example

Kent Beck, 240 Pages, Addison-Wesley Professional

Quite simply, test-driven development is meant to eliminate fear in application development. While some fear is healthy (often viewed as a conscience that tells programmers to "be careful!"), the author believes that byproducts of fear include tentative, grumpy, and uncommunicative programmers who are unable to absorb constructive criticism.

Selenium Testing Tools Cookbook

Gundecha Unmesh, 326 Pages, Packt Publishing

Selenium Testing Tools Cookbook is an incremental guide that will help you learn and use advanced features of Selenium WebDriver API in various situations for building reliable test automation. You will learn how to effectively use features of Selenium using simple and detailed examples. This book will also teach you best practices, design patterns, and how to extend Selenium.

Software Test Automation

Mark Fewster, Dorothy Graham, 600 Pages, Addison-Wesley Professional

This book describes how to build and implement an automated testing regime for software development. It presents a detailed account of the principles of automated testing, practical techniques for designing a good automated testing regime, and advice on choosing and applying off-the-shelf testing tools to specific needs. This sound and practical introduction to automated testing comes from two authors well known for their seminars, consultancy and training in the field.

Just Enough Software Test Automation

Daniel J. Mosley, Bruce A. Posey, 288 Pages, Prentice Hall

Just Enough Test Automation shows test developers and users how to design, implement, and manage software test automation. Learn from authors Dan Mosley and Bruce Posey how to implement a powerful data-driven testing framework; automate unit testing, integrate testing and system/regression testing; and facilitate manual testing with automated tools.

Experiences of Test Automation: Case Studies of Software Test Automation

Dorothy Graham, Mark Fewster, 672 Pages, Addison-Wesley Professional

Authors Dorothy Graham and Mark Fewster wrote the field's seminal text, Software Test Automation, which has guided many organizations toward success. Now, in Experiences of Test Automation, they reveal test automation at work in a wide spectrum of organizations and projects, from complex government systems to medical devices, SAP business process development to Android mobile apps and cloud migrations.

ATDD by Example:A Practical Guide to Acceptance Test-Driven Development

Markus Gärtner, 240 Pages, Addison-Wesley Professional

With Acceptance Test-Driven Development (ATDD), business customers, testers, and developers can collaborate to produce testable requirements that help them build higher quality software more rapidly. However, ATDD is still widely misunderstood by many practitioners. ATDD by Example is the first practical, entry-level, hands-on guide to implementing and successfully applying it.

Additional Test Automation Resources

In addition to the above mentioned books, there's a huge number of online resources to help you get started with test automation and test-driven development. We researched the best articles and online resources and please see the list of links below.

A guide for test automation

Software Testing has found its place in the software industry, with more and more organizations understanding the crucial role that it plays in quality software production. This article explains test automation strategies and approaches for software teams.

10 (Sometimes Painful) Lessons Learned from Test Automation

This presentation explains how eBay's quality engineering team uses automated testing in different projects and within different development methodologies (such as agile, waterfall, distributed etc.) and the lessons they learned.

What are the disadvantages of automated testing?

This is a very useful discussion about the (possible) drawbacks and disadvantages of automated testing and covers areas such as programmer productivity, tools cost, skill level, organizational challenges and more.

Introduction to Test Driven Development

This is an in-depth article about test-driven development (TDD), explains the pros and cons of TDD and contains additional links to related tools and resources. The article also tries to explain and clarify typical misconceptions of TDD and explains why and when teams should adapt test-driven development for their projects.

When to Automate Testing?

This article tries to explain when (and when not) to automate your software tests and provides a useful list of criteria. The article also discusses the need to spend testing efforts wisely to find the most important software defects.

Testing Newsletter

Subscribe to our newsletter to receive tips, community news and articles right in your inbox:

Test Automation

In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes to predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or add additional testing that would be difficult to perform manually.

Keep Reading

Keywords

  • test automation
  • test automation software
  • test automation tools
  • automated testing tools
  • automated testing software