If you enjoyed this book considering buying a copy

Chapter 4: Test your functions #

Alfredo Deza

Testing is one of the core beliefs I have as a software engineer. When I started learning Python many years ago when I met Noah, he suggested I should start learning about testing right away. I’d been learning Python for a week and I was already being asked to start thinking about this new concept.

If you haven’t done any testing at all before, then this is going to change your life. It certainly changed mine! Wherever I go, I instill my passion for testing in others, as it is the one thing that enhances software and increases its robustness. When I test, I often found silly bugs, even in the smallest of functions. The perfect developer doesn’t exist; that is why we need tests. Tests keep us in check, increases our confidence when delivering software, and makes maintenance much more manageable.

Writing tests is hard work. It feels cumbersome and repetitive at times, but it is very worth it. When you find yourself writing a new function, regardless of its size, it is going to feel straightforward to make changes directly into the function and not think about anything else, let alone testing. It is a similar issue as writing documentation (I am very passionate about that too). The problem with avoiding testing is that you can get into trouble very quickly when the function is no longer tiny, and it has several if and else conditions, and then later, you encounter a coworker adding two nested for loops. Suddenly the small function is no longer small, and nobody wants to work on it because any changes break things in ways that nobody can fix.

Don’t be afraid to ask “Where are the tests?” when reviewing changes. I ask for this all the time. When I get push-back, I have a bag full of ready-to-go statements:

How can you tell if this works?
I don’t know if this doesn’t break anything else
How can I make sure I will not break this fix in the future?

Most everyone else is already doing testing in one way or the other. Think about it: when you write a function or a piece of software, you run it eventually to see if something happens. If something is not happening, then you go back to make changes. That is what testing is about. But by writing actual tests and running them for results, you are automating the tedious process of making changes and seeing if something happens.

Don’t use unittest #

This is what everyone will default to. It is not a good option. If you have never done testing before it will feel like writing Java - not Python, and this is because the unittest framework was inspired by JUnit , the Java framework for testing. The unittest framework forces you to do class inheritance and learn several dozen assertion methods in order to ensure expectations are correct when testing. The whole premise of this book is that to be a proficient developer in Python, classes and class inheritance is not needed at all. This is a list that I use that is part of the Python documentation for the unittest framework, which shows all the assertion methods available:

self.assertEqual(a, b)
self.assertNotEqual(a, b)
self.assertTrue(x)
self.assertFalse(x)
self.assertIs(a, b)
self.assertIsNot(a, b)
self.assertIsNone(x)
self.assertIsNotNone(x)
self.assertIn(a, b)
self.assertNotIn(a, b)
self.assertIsInstance(a, b)
self.assertNotIsInstance(a, b)
self.assertRaises(exc, fun, *args, **kwds)
self.assertRaisesRegex(exc, r, fun, *args, **kwds)
self.assertWarns(warn, fun, *args, **kwds)
self.assertWarnsRegex(warn, r, fun, *args, **kwds)
self.assertLogs(logger, level)
self.assertMultiLineEqual(a, b)
self.assertSequenceEqual(a, b)
self.assertListEqual(a, b)
self.assertTupleEqual(a, b)
self.assertSetEqual(a, b)
self.assertDictEqual(a, b)
self.assertAlmostEqual(a, b)
self.assertNotAlmostEqual(a, b)
self.assertGreater(a, b)
self.assertGreaterEqual(a, b)
self.assertLess(a, b)
self.assertLessEqual(a, b)
self.assertRegex(s, r)
self.assertNotRegex(s, r)
self.assertCountEqual(a, b)

If you are getting convinced that classes and inheritance are not needed to test, then you might be wondering what the alternative is. Luckily for us, back in 2004, Holger Krekel started a project called Pytest. He probably saw how complicated the status quo was, and he devised ways to not force anyone into classes, inheritance, and several dozen assert methods. Pytest itself tackles all of these items: there is no need to write test classes (although you can!), you only need to remember one assertion statement, and there is no need to do any sort of inheritance at all.

Testing is not only about what a framework can offer, but also how executing the tests and getting a rich report with useful information later. The unittest framework has a small command-line interface to discover and execute tests, and its reporting is negligible. This chapter gives you options, including a way around existing unittest tests. Testing should be easy to get started, and tools should offer any complexity required as an option, not a requirement.

Pytest #

The Pytest project, as I’ve mentioned, is both a framework and a command-line tool. If I had to pick one section of this chapter, this is the one I would recommend going through in detail. Pytest allows you to write tests without getting in the way, no need for classes or learning a never-ending API.

Simplest case possible #

The next example uses a function that converts a string into an integer. The function is small enough that testing shouldn’t be a problem. Even the smallest validation possible allows an increase in confidence that the code works as intended and has no bugs when it needs modification later on. It accepts a single input, which is a string, and then it tries to force the string to become an integer, create a new file called utils.py and add this function:

def str_to_int(string):
    return int(string)

This helper function has many issues as it is, but we aren’t aware of what potential problems it may have to deal with. Create a new file, called teste_utils.py, and add the following function along with the import statement so that it can use the utility for the tests:

from utils import str_to_int

def test_str_to_int():
    assert str_to_int("52") == 52

It is essential to prefix all tests with test_ so that the framework can automatically discover it and run it as well as prefixing the filenames with test_ just like you have done for the file above, naming it test_utils.py. In this case, it is calling the function directly, passing a "52" and asserting that the returned value is the integer 52. Create a new virtual environment, and install Pytest, so that it can run these tests:

$ python3 -m venv venv
$ source venv/bin/activate
$ (venv) pip install pytest
...
$ pytest --version
This is pytest version 5.4.1, imported from /Users/alfredo/python/minimal-python/chapter4/venv/lib/python3.8/site-packages/pytest/__init__.py

All these examples should work with Pytest’s major version 5, so it is OK if you install a newer version of Pytest than 5.4.1 as I did. Now with the virtual environment activated and with Pytest available, call the command line tool in the same directory where the utils.py and test_utils.py files exist:

$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.8.1, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /Users/alfredo/python/minimal-python/chapter4
collected 1 item

test_utils.py .                                                          [100%]

============================== 1 passed in 0.01s ===============================

The test passed! But we have a new requirement, some strings that are coming in are using decimals. Before addressing the code, create a new test that checks if that would create a failure, and if it does create a failure, what type of failure that would be. That is very important to see, as it helps clarify what fixes are needed (if any):

def test_str_to_int_integer():
    assert str_to_int("52,2") == 52

Run pytest once again in the same directory as before to check its reporting:

=================================== FAILURES ===================================
___________________________ test_str_to_int_integer ____________________________

    def test_str_to_int_float():
>       assert str_to_int("52.2") == 52

test_utils.py:7:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

string = '52.2'

    def str_to_int(string):
>       return int(string)
E       ValueError: invalid literal for int() with base 10: '52.2'

utils.py:2: ValueError
=========================== short test summary info ============================
FAILED test_utils.py::test_str_to_int_float - ValueError: invalid literal f...
========================= 1 failed, 1 passed in 0.04s ==========================

This is the first failure, and there is lots of information packed in the reporting. A ValueError is raised because int can’t understand how to deal with a decimal. So the utility function needs to be improved. Since the problem is that the string input can be a float-like number, use float to do the conversion and then int to make it an integer finally:

def str_to_int(string):
    return int(float(string))

Rerunning the tests shows everything passing. That is great news, but we have one last requirement. Some strings are using a comma instead of a dot for decimals, and the code might not handle this. Again, create a test for this scenario before looking into fixing the utility function:

def test_str_to_int_comma():
    assert str_to_int("52,2") == 52

Run pytest once more to get the reporting in the terminal:

=================================== FAILURES ===================================
____________________________ test_str_to_int_comma _____________________________

    def test_str_to_int_comma():
>       assert str_to_int("52,2") == 52

test_utils.py:10:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

string = '52,2'

    def str_to_int(string):
>       return int(float(string))
E       ValueError: could not convert string to float: '52,2'

utils.py:2: ValueError
=========================== short test summary info ============================
FAILED test_utils.py::test_str_to_int_comma - ValueError: could not convert s...
========================= 1 failed, 2 passed in 0.04s ==========================

There are 3 tests, and just one is failing, which is the newly added test. A way to further address this is just to get rid of the comma so that the test passes. Go ahead and implement that change in the utility function:

def str_to_int(string):
    string = string.replace(',', '.')
    return int(float(string))

Running pytest in the command line once again should report everything (all 3 tests) passing. In some situations though, you can see numbers displayed as "1,100,700.54". What would happen in that case? At this point, I’m not exactly sure what the utility can do, so with that example, write another test to check the behavior:

def test_str_to_int_commas():
    assert str_to_int("1,100,700.54") == 1100700

Run the test again to check the report from pytest:

=================================== FAILURES ===================================
____________________________ test_str_to_int_commas ____________________________

    def test_str_to_int_commas():
>       assert str_to_int("1,100,700.54") == 1100700

test_utils.py:13:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

string = '1.100.700.54'

    def str_to_int(string):
        string = string.replace(',', '.')
>       return int(float(string))
E       ValueError: could not convert string to float: '1.100.700.54'

utils.py:3: ValueError
=========================== short test summary info ============================
FAILED test_utils.py::test_str_to_int_commas - ValueError: could not convert ...
========================= 1 failed, 3 passed in 0.05s ==========================

There are now four tests, one of them (the new one) has failed. Another ValueError encounter that needs to get addressed, but before going into catching the ValueError exception and fixing it there, I believe that the previous fix was an incorrect one. It is blindly replacing any comma into a dot, which in this case, is incorrect. The error reported shows '1.100.700.54', which is not what I expected at all. There are many ways to solve this issue, and the correct way I believe would be to use the right locale in the environment. Different environments set the locale, which produces these sorts of differences in how numbers represented as strings show. To avoid using the locale and demonstrate further the power of simple testing, the code is going to have to be more careful. First, address the fact that there can be a mix of commas and dots:

def str_to_int(string):
    if ',' in string and '.' in string:
        string = string.split('.')[0]
        string = string.replace(',', '')
        return int(string)
    string = string.replace(',', '.')
    return int(float(string))

If the string contains both a comma and a dot, it gets split on the dot since it doesn’t matter as it gets eliminated when calling int(). Lastly, the commas are removed, and an integer returns. Finally, all tests are passing, and everything looks in top shape. What started as a small function without tests, grew with a few changes and tests to ensure its proper behavior. Even though you might be thinking the mission is accomplished, there is still one big problem with the code. It can’t handle non-decimal comma-separated numbers like '1,200,700'. At this point in the implementation, you need to ensure what type of inputs you are dealing with. If you are sure inputs are always going to be coming with decimals, then the current implementation is enough; otherwise, a more robust change needs to happen, probably with a locale in mind.

Questioning testing #

A general rule I have for any concepts I hear about is to accept them and then question my reasoning. Self-reflection, when you are learning new things in technology, is similarly a good thing to do. When I first started testing, I struggled a lot, finding it difficult and time-consuming. I remember thinking Noah was probably exaggerating when he insisted I should keep at it until I felt more comfortable and understood its usefulness. One of the suggestions was to subscribe to a mailing list called “Testing In Python”, from which you can see my first email to the list. It was back in 2009, and I was already starting small projects and writing tests. You can tell I’m fairly enthusiastic by my opening in the thread: “I have just started to get into the habit of testing everything that I code […]"

I was lucky in that Noah was mentoring me all those years back and guiding me into solid concepts like testing, but not everyone has that luxury. No doubt, you will encounter people who are writing code that think testing in any shape or form is a waste of time and that it slows everything down. I’ve even encountered this kind of reasoning in professional settings, where testing is an afterthought or simply not required amongst the development team when introducing a new feature. When problems arise, they require a tremendous effort to try and debug what exactly is happening, and when a fix is available, the effort persists to ensure that the problem gets fixed. I can’t fathom developing like this!

Imagine you are developing a website, and you are making a change, where the logo of the company is a new one. How do you make sure that your change is, in fact, in place and ready to be launched? If it has already launched, how do you know if the new logo is there or not? The answer may vary, but it is probably close to “I open a browser and see for my self”. That in itself, is a form of testing! Wouldn’t it be great if there was a way to have something automatically ensuring that your new change is in place? What if that “something” took 0.1 seconds instead of a couple of minutes? Now imagine having verifications for a hundred other things related to the website, all happening automatically and continuously. Is the website is still up and running at 4:16 am? Because if the company is selling overseas, at 4:16 am in New York means 9:16 am in Madrid!

In 2011, just a couple of years after my first email to the Testing In Python mailing list, I got the opportunity to fly to Lima, Peru, to give a presentation at a technology conference. My talk was called: “Testing In Python: if it doesn’t work whom do we blame?”. While I was preparing the presentation, I thought I should ask the mailing list once again what their thoughts were on why would one bother with testing. The thread is excellent, and I recommend you take some time to go through the replies, but here are a few that made it into the presentation:

It is the fastest way to develop. Possibly because bugs are caught right away
Prevents users finding out about broken problems
Enhances confidence when coming back to code, even as old as a few weeks
Prevents one to make the same mistake
Eliminates lots of common problems reasonably quickly as soon as you hit them

The presentation also had a short story about the Broken Windows Theory, in which it believes that signs of crime and anti-social behavior (like a building with broken windows) attracts more crime and disorder. Translated to the issue at hand, untested code becomes worse as it is difficult to test, and because it is difficult to test, it becomes even worse!

A trick I often use when a feature or bug fix I am introducing has to deal with these large and complex pieces of code is to extract the logic I am doing into a separate, smaller function that I then can fill it with lots of tests.