Automated End to end testing (E2E) is something that you as a software developer should at least have heart multiple times about. This topic is always something that has haters and supporters but it is an important one anyway. Most of the people I know are not a big fan of E2E testing because of all the effort it occurs together with the dependencies you need to consider. In this blog post I will discuss why the effort is worth it and why agile development principles need E2E testing to work in a propriate way.
Let’s start with a very provocative sentence: Without E2E testing you cannot have reliable and a working continues deployment. Continues deployment describes the process of trying to deploy new releases in a continues way, so you will not have the big bang one time but you are going to deliver software every time a feature is ready for testing. Something that is essential if you are living agile development correctly as customer integration is a core part of that way of software development. But what does that mean? After every sprint, so every 1-3 weeks you need to deliver working software to the customer. Nothing worse if you are a one man development team which delivery is just a zip with one web app folder. But this is alone a pretty big deal if you have many development teams working on a microservice architecture. In such a scenario you need some kind of automation when something is ready for deployment as well as some kind of automation in terms of how the software gets ready for the customer. And in automation you always need some kind of validation, a way to have that kind of automation is E2E testing.
First of all what does E2E testing mean? E2E testing is the process of testing the whole application. So in contrast to unit testing where you just check if a granular part of the application works in E2E testing you check if the whole application works with the interactions the user might do. In a typical web application which has a registration form that would cover an automatic interaction with the GUI which the user fills out to register as well as the storing of the user and the email confirmation process. So to cover that scenario you need the whole application being set up.
What are the parts you need to consider when you decide to doing E2E testing? First of all the whole application needs to be setup in way that E2E testing is possible. If you have a web app ids on the HTML elements are very important as it is extremely difficult to address a specific element if you don’t have that. If there are no ids you would need to formulate a xpath query which is sometimes not only difficult but also might result in multiple results. Also such queries need to be adopted when something changes in the web app.
Next it is important that you need to specify when your app should be tested automatically. That is a topic for the build pipeline which needs an extra step for E2E testing which covers the setup, the execution and the clean up of the tests. Such a build step should not be a part of the actual application build process as it needs some time and a goal for efficient automatic build pipelines is that developers get feedback quick. So a typical way is that first the application gets build and pushed to a global repository (for instance Nexus) and then the E2E testing process starts.
Also you need to specify when E2E testing is necessary. As E2E testing is an effort it is typically a specify user story task. And also not always it makes sense. The discussion whether or not the user story needs E2E testing needs to take place in a meeting where the development team, testers and the PO sits together as it needs all the different opinions. A meeting which is ideal for that is a refinement. Typically you need to do E2E testing in case you have repetitive work like signing up for an application or creating some sort of general entities. More complex processes which need to be thought througth are done by manual testers. The aim of E2E testing is not to replace manual testing but to let them do the important work!
Next question might be who is responsible for creating those E2E tests as normally you need to create code for that. In my opinion creating E2E tests is a task for software developers but in a big company you might think of having employees with special know-how in E2E testing so that difficult tests can be created by them. But software developers should also create those tests so that everybody understand why something needs to be coded in a certain way (like having ids on HTML elements). Software developers are also responsible of making sure that new features do not destroy E2E tests. That in mind when estimating a new feature it is also important to think of which test might be affected.
But what has that to do with agile development? I agree that E2E testing should also be done in other software development processes. But especially for agile development principles it is important to have automated E2E tests. To understand that, you first need to think why E2E testing is done? To increase application quality, as everytime when you do tests. In agile development high quality software is important as too many bugs will result in problems when it gets to reaching sprint goals. And manual testers have lot of work to do, so they should not retest basic functionality all the time. If manual testers can focus on complex work they will find more relevant issues and this can only be done when you reduce other tasks they might need to do. Also you can make sure that the customer will not discover basic bugs when rolling out the new release as tests have checked that before.
So that is my opinion on E2E testing. It is a lot of effort but it is still worth it. What do you think about that topic?