The Test Automation Pyramid: Your essential guide for test automation

Test Automation has not only grown in popularity and usage over recent years, it has also grown in terms of potential solutions and ways to implement.

Software providers now rely heavily on the use of test automation tools and techniques for software testing in order to release their software quicker and with more quality and confidence. When following processes like DevOps, where several release candidates could be produced in a day, automation testing (and continuous testing) enable the team to get quick feedback on the viability of the release which has been produced.

It also allows decision makers to understand what risks may be associated with a given release and make choices based on reliable testing statistics and test data.

At DeeperThanBlue, we talk a lot about Test Automation. Our clients want to understand more about how to implement this effectively. The reference point for this should be the Test Automation Pyramid, the concept of which was designed by Mike Cohn in his book ‘Succeeding with Agile‘.

When looking at implementing Test Automation, we use this Test Automation Pyramid as a guide.

As you can see from the diagram, the earlier you start automating tests, the more benefit you will gain, thus increasing your ROI.

At this stage, it’s important to note that we’re not advising to not run UI Test Automation, or to not do Manual Tests. What we’re doing is showing a way to do less of these things, as they are less reliable and return less in terms of ROI.

Automating Unit Tests

When following Behaviour Driven Development (BDD) and Test Driven Development (TDD), the onus of producing better quality software is put more squarely on the developers.

The process of writing a Unit Test BEFORE any production code is written is really the best start that a team could make to producing error free code. This way, you are not only documenting your solution as easy-to-follow tests, it becomes clear when you break something which you don’t expect to. This in turn helps to avoid Regression issues.

Depending on which language you develop your software in, there is a plethora of tools to help you automate your unit tests. At this stage, it’s important to work with the developers to understand which they would be most comfortable to pick up and use going forwards.

Automating API Tests

Automating API tests can be a simple but effective way to ensure your data flow and integration points are working as expected. Automating in this layer can also be a lot quicker than having to log into a User Interface and populate the information on-screen, to trigger the API.

Automating APIs directly will also help you execute non-functional tests. This can include both security and performance testing for example.

With regards to tooling, there are some simple solutions, like Postman and SoapUI, which can be used to automate your API tests. There are also Licensed versions of these tools to help scale up and share the work across the team. As well as these no-code tools, you may opt to utilise your own API testing framework (perhaps using tools such as Rest-Assured or Karate DSL). The main benefit of coded versus no-code tools is that you will have more flexibility to adapt your tool to fit your solution.

Automating UI Tests

Automating your User Interface Tests, might not be as quick to execute as automated API or Unit tests, but it can still cover some critical test scenarios.

If you run your tests in a third party tool like Browserstack or Saucelabs, you can get a great idea of whether your solution can work with your in-scope browsers and device set. Always base these tests on analytical data from what your user base is primarily using. For example. there would be little value in testing your solution on the Chrome browser, if your test data suggests that your users do not use this browser to interact with the software.

Tooling can consist of Coded (such as Selenium integrations and Playwright) and No-Code (such as IBM’s Rational toolset, Katalon, Eggplant, Test Complete and others) solutions and it will depend on your team’s experience as to which will work for you.

As described for API testing, the coded solutions will provide more flexibility which may help you in the longer term, to adapt your testing according to the requirements.

Manual Tests

Whilst all of this automation can help you and your teams deliver better quality faster, it is still prudent to schedule manual test cases to focus on how a user may behave, as well as testing more critical scenarios or scenarios which cannot be automated.

Often teams will perform a number of different activities here, but your delivery team should be able to identify which activity or activities will suit best. Some ideas include Bug Hunts, Exploratory Testing and Checklists, for example.

• Bug Hunts

  A bug hunt is a time-boxed piece of work with a number of testers (or members of the team), where you walk through user journeys and flows, with the aim of identifying bugs, which may stop the release or reduce the quality of the software.

• Exploratory Tests

  Exploratory tests are more free-form in nature and dependant on a tester with good knowledge of the system, in order to look for issues around the more complex areas of the solution. The tester may also choose to look at specific areas which have proven to be problematic in the past.

• Checklists

  Checklist based testing is more organised that Exploratory Testing and relies less on an individual’s prior system knowledge. It also provides a clear audit of tests that have been executed and their results.

Get Expert Help

As you can see, there are a number of areas that need to be considered when implementing automated testing on software projects.

Following the principles of the automation test pyramid, the earlier in the development pipeline you invest in automated testing, the more benefit you can get to de-risk the software implementation. It is essential to hire expert help to implement automated testing to ensure the most effective outcome for your business.

Whether you are starting on your test automation journey or would like to review where you are and find ways to improve, we can help. Get in touch with the Unify team below.

• Share