A Starters Guide To Test Automation
Testing all parts, new and old, is crucial, but doing this manually is cost and labour-intensive...
Several weeks of development have passed and the new version of your product is finally ready. You tasked your team to test it for a few days, after which the errors they found were patched and the final tweaks were made. Finally, you deploy the new version to the production server.
Unfortunately, not even an hour has passed and you already hear complaints from your clients. An old feature they use daily no longer works as they expected. The changes to the code had an unintended side effect that caused a defect.
Maintaining a stable and reliable software product comes with its challenges. Each newly added feature creates many new interactions with existing code, with many new bugs and unintended side effects waiting around the corner.
Therefore, testing all parts, new and old, is crucial for your product going forward, but doing this manually is labour-intensive: effort that could be spent elsewhere. This is where Test-Automation comes in.
The Test-Automation Process
To properly maintain your codebase and release new versions of your product you need an automated development pipeline. Any time you commit a change to your source code, the pipeline is triggered and a new build will be made. This build process involves the following test-automation pattern:
First, you run unit tests on the source code level to catch coding errors early.
After that, integration tests are performed to test the interaction between different components.
Next, the application is deployed to a testing environment, on which end-to-end tests (E2E tests) are performed.
Finally, if all steps are completed you have a new stable version of your software that you can ship to your clients.
If any of the phases fails, the test-automation process stops, after which you receive a report on what tests failed and for what reason.
Tests at the source code level are called unit tests. These tests check whether the logic and flow within each component is correct. Even though unit tests are small and abstract, they form the crucial foundation of your automated tests. They do not need any setup, unlike integration or E2E tests and can be run during the build process. They are the fastest tests to perform, catching errors early.
In the ideal situation, unit tests should be the foundation of your tests. But even if they cover your entire codebase, you cannot prevent defects with unit tests alone. Software consists of more than just individual components. Each component interacts with other parts of your product, or with external libraries, databases or APIs.
An integration test does exactly what it says on the tin: it simulates the integration of different parts of your application and checks if they communicate properly. While these tests require some additional setup compared to unit tests, they still work “under the hood” of your application, giving these tests more freedom than testing from the user interface of your application.
Finally, E2E tests are written from the perspective of the user and are therefore run on a fully functioning copy of your application, by using Docker for example. They usually consist of a list of steps that you follow; if there are no hiccups along the way and the end result is what you expected, the test has passed and you move onto the next one.
If you are already performing manual tests, then these can be easily converted to automated E2E tests. Although E2E tests are valuable, they should be used sparingly. They can only be performed when your software is fully built and configured. In addition, they have a high upkeep cost, as the UI of your product is prone to change, which may require changes to your E2E tests.
Relying on End-2-End tests only
If E2E tests look at your application from the perspective of the user, why should you bother making integration or unit tests?
Say you have your entire application covered with E2E tests. You build your application, set up a test environment and run the tests. After a lengthy 2 hour test period the testing is completed and some of your tests have failed.
In such case, your developers will have to dive into the code, fix the bug and hope that the next 2 hour testing cycle is successful. E2E tests protect your users from bugs, but do not support developers in finding the bug’s cause. This is why it is important to have a good foundation of unit and integration tests.
Unit tests are run in a fraction of the time that E2E tests will take, so component level errors are detected quickly. As the scope of unit tests is so small, developers can quickly find what part of the source code caused the error and correct it accordingly.
In a similar vein there are E2E tests that can be replaced with integration tests. Say your application has a search engine with several fields. While you could cover all different search queries and configurations with E2E tests, running integration tests that make the database calls directly would perform much faster.
Test automation as an investment
Just like any other process in your development pipeline, testing your application should be formalized and automated as much as possible to reduce labour-costs and minimize the time between releases.
Reaching that point can be challenging, especially on a large existing codebase with little to no automated tests. But delaying its implementation will only lead to more technical debt. Therefore, investing in test automation is crucial going forward, allowing you to mature as a software company that offers high quality products with minimal defects to your clients.
Every case is different…
If you would like to know how to start YOUR journey towards a fully tested codebase, or want advice on how to improve your testing process…
I suggest you introduce your case and we’ll answer your question.