Software testing is an essential mechanism to guarantee the correct functioning of our applications. There are different types of tests, each of them focused on evaluating a different characteristic or with a particular approach.
Within the repertoire of tests that can be performed in a testing environment, we will analyse the end-to-end (E2E). These tests are responsible for simulating real scenarios to validate that the behaviour of a final product is as desired and meets the needs of the users who will interact with them.
What is E2E testing used for?
End-to-end tests are methods that comprehensively validate the entire workflow of our applications. From the user interface to the database and backend services, E2E allows us to detect performance, usability and accessibility issues.
They focus on reproducing the behaviour of a user within our functional applications (systems deployed in production or very similar environments) to check that the software as a whole behaves in the way we expect it to.
Scope of end-to-end testing
The scope of E2E testing covers the critical and high-risk functionalities of our application. Since they are costly to maintain, their use focuses on aspects such as the user interface and its most common actions within the application, communication between services, authentication and authorisation, third-party integrations and performance under load.
Therefore, we should not focus on obtaining an extremely high coverage by using only this type of tests. Ideally, they should be complemented by integration and unit tests, which are less costly and more recurrent.
In order to decide where to use them, it is important that we identify the key user journeys. We also need to be able to understand what the most frequent functionalities are within the system. Otherwise, we may not be detecting the malfunctioning of a critical component in the system. Clear examples of cases where it is important to implement them are those that correspond to the user stories that we have previously developed to configure our application.
Types of E2E tests
End-to-end tests can be divided into different subtypes depending on the feature they try to implement. Among the total repertoire that exists, a good set of types of E2E tests that we can perform would be the following:
- Functional testing: checks that each feature works as described in the requirements. They also check that the workflows are carried out correctly.
- Usability testing: ensures that the application is intuitive to use.
- Compatibility testing: assesses whether the application works correctly on different devices, operating systems, browsers and versions.
- Performance testing: checks the responsiveness and stability of the application under different load conditions.
- Security testing: detects whether vulnerabilities exist or whether data is being protected against unauthorised access, attack or loss.
- Disaster recovery testing: determines the ability of the system to recover in case of crashes or failures.
- Compliance testing: validates whether the system complies with established industry standards. For example, legal regulations or quality standards. These tests may only be useful in some specific environments, as they are related to audits.
As in most approaches to system testing, we can also distinguish between horizontal and vertical testing:
- Horizontal testing focuses on the complete walkthrough of the different components of the application, simulating real-life situations from a user’s point of view. The types most closely related to this category would be functional and compatibility testing, as they evaluate the interaction between the different components of the system.
- The vertical ones focus on the internal components of the individual layers of the system (backend services, databases and middleware), testing that each of them performs its job correctly. Here we find security testing, performance testing and disaster recovery testing if we analyse them from the point of view of specific isolated processes, such as specific workflows.
Both of the above approaches can be done manually or through automation.
Best practices
There are some important key points to consider when conducting our E2E tests in order to apply them properly. Perhaps the most common are the following:
- Defining test coverage: this involves establishing a scope when implementing tests so as not to carry out tests that are less important (of use cases that practically do not take place), given that they are costly to maintain.
- Identification of key scenarios: again, this consists of prioritising the most important scenarios, avoiding testing infrequent cases.
- Use known, deterministic data: data whose outcomes are predictable and can be anticipated without a change occurring. Otherwise, the outcome should be random. This could apply to modules that interact with external components where we do not know whether or not we will get a particular result.
- Verification of communication between different services: checking the behaviour of systems connected to each other. For example, APIs, databases and external systems.
- Authentication and authorisation: focused on testing user access to the system and their permissions to guarantee its security.
- Integration with third-party systems: evaluation of the behaviour between the application and third-party systems. It should only be implemented when the data to be obtained is predictable.
- Performance under high workloads: evaluation of the behaviour of the system under stress by multiple users operating on it. It is used to detect possible performance bottlenecks.
- Continuous testing: this involves regular and constant testing to ensure that the system performs flawlessly throughout its lifecycle.
Considering best practices for our E2E testing will allow us to see the overall performance of the application. On the other hand, we will be able to detect problems early in critical parts of our system. Furthermore, we will be able to determine whether the final system meets the technical and business requirements before it goes live.
Key vulnerabilities
Although very important, E2E testing has some drawbacks that can make it difficult to implement and maintain:
- They are slow and costly because they simulate the entire application flow. Therefore, it is necessary to employ more resources to use them than with other types of tests.
- Fragile to changes and interactions with the outside world. They may fail due to changes in the user interface. Or even generate less predictable results when connected to external APIs.
- Can be difficult to debug. When the test fails, there are times when the error may be related to several components of the system and may be difficult to detect.
- False positives or negatives. Sometimes, some poorly defined tests may pass even though the real functionality has bugs if they have not been detected.
- Scalability issues. There is an added complexity when applications start to grow. This is because E2E test suites become larger and more difficult to manage. This increases the test execution time and makes it difficult to run tests in parallel without interference.
Security risks
When developing E2E tests, it is important to be aware that there are some risks related to the exposure of sensitive data if some security guidelines are not followed:
- Sending sensitive data: this can occur when interacting with external APIs. It is avoided by using isolated test environments (e.g. mocking simulations), using fake data or logging all interactions with external APIs.
- Exposure of private data: Sometimes, testing may require logging where sensitive data is used. In that case, it is important to use good security practices, such as environment variables or secrets handlers.
- Denial of Service (DoS) attacks: these attacks can occur unintentionally if they are not well designed. They could exhaust server resources by going into uncontrolled loops or over-consuming the database.
- Exposure of APIs and endpoints can also occur. Therefore, it is important to set up secure environments with firewalls and authentication. In addition to implementing mocks, setting rate limiting and detecting anomalous traffic.
- Manipulation and attacks on automated tests: occurs when malicious code is injected or Cross-Site Request Forgery (CSRF) requests are forged. Also when test execution is manipulated or vulnerabilities in test environments are exploited to attack the real application.
Some relevant technologies
There are numerous tools for E2E testing. Each of them has a number of features that focus on addressing different needs. Perhaps, some of the most recurrent ones today are the following:
- Selenium: it might be more complex to use compared to other options more adapted to a less technical profile. However, it is extremely flexible and customisable. It is the most suitable tool for large projects.
- Katalon Studio: it is a very simple solution, as it has a test recorder with which it is not necessary to write code. In addition, its interface is very intuitive. However, it is not very flexible for complex tests.
- Cypress: unlike other tools, it runs inside the browser, which makes it faster and more efficient. The level of difficulty could be intermediate, as it uses JavaScript but has a user interface.
- Playwright: like Cypress, it runs in the browser, but is faster. Unlike Cypress, it runs in different tabs and offers native support for multiple browsers.
The list of tools goes on and on, and there are differences between them that we need to understand in order to select the one that best suits us. For example, whether or not they are free software, simple or complex, fast or slow, or which browsers they support.
Conclusion
As we have seen in this post, E2E testing is a very important part of software testing. They allow us to ensure the quality and reliability of our applications, simulating the behaviour of a real user and the results to be obtained.
However, it is essential to combine them with unit and integration tests, since their cost in terms of time and resources is high. I invite you to put this type of testing into practice to assess whether your applications are robust and meet your expectations.
