Software tests are tests that are performed on a piece of code or an entire software to validate that the behavior of the software is as expected. The main purpose of testing is to detect errors in the code in order to correct them and ensure the quality of the software. Having an extensive and complete set of tests that can be launched automatically is one of the main steps to have a continuous software delivery (CD).
Software testing can be performed at different levels. At the first level are unit tests. These focus on testing individual units of code, such as a function or a class. Unit tests isolate the component from other components and external systems to focus on validating only the functionality of the unit.
At the next level, integration tests are dedicated to verify the relationships between the different components of the software. These tests help, for example, to check whether two classes interact correctly with each other or whether the sequence of applying two functions one after the other returns the expected result. These tests can be performed on all or almost all software components at once or on smaller assemblies. It is often interesting to perform integration tests in isolation from external systems (e.g. a database), replacing the connection with a fake connection. These imitations of connections or components are called mock.
Finally, once all integration tests have been passed, the entire software is evaluated with a system test. These tests test how the software interacts with external systems and the environment. An example could be testing how the system behaves when one of the components communicates with a database or when another component receives an API call.
In some cases, a fourth level called acceptance testing is defined, which consists of testing that the system meets the functionality required by the end user of the software.
Python unittest library: What is it?
There are different libraries for different programming languages that provide a series of functions and classes that allow tests to be written in a simple way. They facilitate the execution and analysis of the results in case, for example, the test fails and does not return the expected result.
One such library is unittest, which is part of the Python standard libraries. To use it, a class that inherits from TestCase is generated. The methods of this class whose name begins with test will be treated as a test. In this example, the assertEqual method of TestCase is used to verify that 1 plus 2 is 3.
import unittest
class MyTests(unittest.TestCase):
def test_addition(self):
self.assertEqual(3, 1 + 2)To run the tests, the following command can be used via console:
python -m unittest discoveryBy default, this command searches for instances of TestCase in files that meet the following pattern for test_*.py and that are in python packages in the current directory. You can change the search directory and file pattern with the -s and -p options, respectively.
Running the above test returns the following result:
If the expected result is changed to 4, the failed test is displayed as follows:
Use Case
The purpose of this use case is to test an ETL (Extract, Transform, Load) pipeline application. For simplicity, the data will be a single number and each of the pipeline steps are simple functions. The application consists of the following code:
from random import randint
def get_data_from_database(connection):
# Do something with connection
# ...
return randint(1, 6) # Use random int to simulate db connection
def add_one(x):
if x is None:
return 1
return x + 1
def times_two(x):
return 2 * x
def write_data_to_database(x, connection):
# Do something with connection
# ...
return
def pipeline():
x = get_data_from_database('SourceDatabase')
x = add_one(x)
x = times_two(x)
write_data_to_database(x, 'SinkDatabase')Unit tests
This section verifies the functionality of the add_one and times_two transformations, isolating them from the other functions of the pipeline.
In the following tests add_one is tested:
import unittest
from my_app.my_app import add_one
class AddOneTest(unittest.TestCase):
def test_add_one(self):
input_data = [-4, 0, 1, 3, 10]
expected_data = [-3, 1, 2, 4, 11]
for x, y in zip(input_data, expected_data):
with self.subTest(input=x):
self.assertEqual(y, add_one(x))
def test_none(self):
x = None
y = 1
self.assertEqual(y, add_one(x))The self.subTest method is used to test different cases. The None case has been separated into a separate test since we want to see a specific functionality of the transformation.
In the following tests times_two is tested, also using the subtests:
import unittest
from my_app.my_app import times_two
class TimesTwoTest(unittest.TestCase):
def test_times_two(self):
input_data = [-4, 0, 1, 3, 10]
expected_data = [-8, 0, 2, 6, 20]
for x, y in zip(input_data, expected_data):
with self.subTest(input=x):
self.assertEqual(y, times_two(x))
def test_none(self):
x = None
self.assertRaises(TypeError, times_two, x)In this case, the value None is not specifically handled in the function. Multiplying this value by 2 is not a permitted operation in Python and raises the TypeError exception. We want this to be the behavior of the application. Then you check that this error is returned using self.assertRaises.
Integration tests
In this section the functionality of the pipeline is checked, verifying that all the components work correctly with each other. In this case, we do not want to depend on database connections and we want to run the integration test locally. To achieve this purpose, you use unittest.mock, which allows you to replace parts of the code.
In the following example, the @patch decorator is used to replace two parts of the code. This decorator passes by parameter the mock object that replaces the specified part of the code. Then we can specify the behavior that we replace using this parameter inside the test.
import unittest
from unittest import mock
from my_app.my_app import pipeline
class PipelineTest(unittest.TestCase):
@mock.patch('my_app.my_app.write_data_to_database')
@mock.patch('my_app.my_app.randint')
def test_pipeline(self, mock_randint, mock_write_data_to_database):
input_data = [-4, 0, 1, 3, 10]
expected_data = [-6, 2, 4, 8, 22]
for x, y in zip(input_data, expected_data):
with self.subTest(input=x):
mock_randint.return_value = x
mock_write_data_to_database.side_effect = \
lambda z, *args, **kwargs: self.assertEqual(y, z)
pipeline()
@mock.patch('my_app.my_app.write_data_to_database')
@mock.patch('my_app.my_app.randint')
def test_none(self, mock_randint, mock_write_data_to_database):
mock_randint.return_value = None
mock_write_data_to_database.side_effect = \
lambda x, *args, **kwargs: self.assertEqual(2, x)
pipeline()The first function that is replaced is randint, which is used in get_data_from_database (simulates a connection to a database). In this case, this function is replaced in the context of my_app.my_app, and not in the random library. This is because in the my_app.py file randint is imported and then it has no way of knowing that the function has been replaced by a mock object. In the case of this function the returned value (return_value) is replaced.
The second function that is replaced is write_data_to_database. In this case, when this function is called, the test will be evaluated with self.assertEqual (side_effect), instead of writing to the database.
In summary, the pipeline has been tested by replacing the input and output with mock objects, thus testing the add_one and times_two transformations together. The mock objects also have other functionalities to check how these objects have been called. For example, assert_not_called allows to ensure that the replaced function or object has not been called.
Conclusion
This post has introduced the basics of software testing, as well as the Python unittest library. Tests help to implement more robust code and allow for smoother deployment. Although only very simple examples have been seen and using a single programming language, the concepts apply to more complex software and the methodologies are similar to others used in other libraries and languages.
