3 Questions you should ask yourself before automating a functional test scenario

The list of functional test automation frameworks is endless nowadays. A pitfall off this ease of automation is that testers get overenthusiastic in automating all functional test scenarios. As a result you can be busy whole day maintaining your automated regression testset. If you have a large project, the number of tests to maintain can grow exponentially during the years. The time to create test scenarios and manual/exploratory testing time decreases, because all time will be spent on refactoring your automated tests. In my experience manual and exploratory testing is still an essential part of the software testing process. Automated tests are robotic and don’t necessarily act as a real user would. Protractor for example is manipulating the browser DOM by injection some javascript to be able to wait for synchronisation. Manual testing, on the other hand, allows the tester, a real user, to be in control of the tests. Nevertheless automation is  necessary to cope with the current rapid software release cycles.

To determine which logical tests scenarios are eligible to be automated you can ask yourself the following questions.

1. Is this or part of the scenario already covered by a system integration or unit test?

The mindset of developers regarding testing is changed radically the last couple of years. More and more developers understand the importance of writing unit tests and integration tests to improve code quality. This has resulted in a lot of javascript end-to-end testing frameworks, most of them based on the Selenium WebDriver API. With those frameworks a lot of user input element functionality can be checked before the functional system test phase. For example the behaviour of a custom dropdown element can completely be tested with a end-to-end javascript testing framework like Nightwatch.js. As a tester, when starting in a new or existing project, it is always wise to check the unit test and system integration test code to determine functionality that is already covered. Also ask a developer about their vision on testing, this can lead to new insight for you and the developer. Also reviewing/skimming the quality of the tests written by developers can be of great help to improve software quality.

2. Can the functionality be tested with a more ‘low-maintainable’ test?

If I have a functional test scenario I try to put it as low as possible in the testing pyramid. If the level is not suitable for the scenario I go one level up. Think about a situation here were you want to automate the following scenario. A user enters an account number into a web application form. The server can reply with a valid or invalid response. The invalid response can result in 20 different error messages. Given this situation a tester can write functional scenarios for all error messages. This can result in possibly 20 functional testscripts. My solution for this situation would be to verify the server frontend mechanism with one functional automated test and verify if all the messages are returned correctly with one or more backend  integration testcases.

3. Is this critical system functionality?

This question is especially for starting projects, when you start writing some script, take baby steps. Think, “what is the minimalistic thing I want right away!”. Keep building on top of it whenever tasks get complicated. Functional testing scenarios often have a lot of overlap in test steps to be able to prepare certain scenarios. It will save you a lot of time in the future when generic functions are created which can be reused in the test scripts. I see the generic functions as lego bricks which can be used for automated testcase construction.

Test automation is becoming more and more important due to faster development iterations and agile methodologies. Overthinking were and how certain functionality of your application will be tested can take effort to accomplish but it saves you a lot of valuable work time in the future.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.