Enterprise software needs tests the way buildings need fire and safety inspections. If you know certain electrical conditions or structural issues could lead to catastrophe, you look for them not just once but again and again. Likewise, if you want to guard against certain problems or conditions in your software application, writing tests for them ensures that they won’t ever escape into production.
Tests are doubly important with dynamic languages like Python. With dynamic languages, whole classes of errors surface only at runtime. Robust test suites can help smoke out those problems, and provide a way for whomever inherits the codebase to do the same.
By default, Python comes with its own unit-testing framework, and it is both useful and widely used. But many other options exist, each with its own philosophy about how to build and maintain unit tests. Following is a rundown of the default, unittest, along with the three most significant alternatives.
unittest and doctest
unittest is the default testing framework that ships with Python’s standard library, and it is used to create the test suites for Python itself. unittest isn’t as far-reaching as some third-party testing frameworks, and it isn’t meant to be — it supplies just enough features to write robust unit tests for most projects. Java developers who have used the JUnit test framework should find unittest familiar.
To create a test suite, you create Python classes derived from unittest.TestCase, with the tests being functions in those classes. The tests can be simple equality or inequality assertions, or they can be more sophisticated. For example, the assertRaises test ensures that a given exception is raised for a given test.
Tests can be grouped into a single file, or spaced out across multiple files, auto-discovered, and then executed. You can also provide fixtures (setup and teardown routines) for each test group, designate tests to skip under specific conditions, and control the execution behaviors of the tests.
The biggest advantage to using unittest is that it is universally known and understood. It’s hard to go wrong with unittest, if only because the way your test suite is written will be highly familiar to other Python programmers.
doctest, also supplied with the Python standard library, is a complementary module. doctest looks for comments in Python files that are formatted like Python interactive sessions, and attempts to run the commands contained within those comments to see if the results match. In this way the sample output in the docstring for a function can be verified along with the function’s behavior.
Note that doctest is best suited for tests of simple functions where the inputs and outputs are readily understood in the console, and for tests that don’t require lots of setup or teardown to yield results. If you try doctest on complex functions, you may end up with doctest examples that are too unwieldy to be useful.
pytest
The pytest project can replace unittest entirely or augment it. pytest applies a different philosophy to test writing, allowing a developer to quickly implement a small test suite for a project without a lot of scaffolding. As the project grows, pytest can scale up to match, allowing more testing functionality to be layered in over time.
pytest tests start off as compact functions that follow a certain naming convention in a module, not as a subclass of an existing test-type class. Tests are invoked by running pytest on the module from the command line, or by invoking pytest within code. You can create pytest fixtures to set up and tear down test conditions, and scope those fixtures as broadly or narrowly as needed (e.g., from test-session-wide to only function-wide).
The options you can add as you go are many and powerful. Hundreds of pytest plug-ins let you integrate tests with many third-party services and actions. Finally, the unittest and nose test suites can be run as part of the testing process as well, so you can integrate pytest with existing suites without having to ditch them and start from scratch.
nose and nose2
nose and its successor nose2 expand on tests written with unittest and make them easier to run. nose itself is no longer maintained; nose2 continues the project with Python 3 compatibility and some changes in functionality, but with the same underlying philosophy. We’ll concentrate on nose2 here.
nose2 differs from unittest in two key ways: It has a plug-in architecture and uses .ini-style configuration files to control how tests run. Many of the nose2 plug-ins are provided and enabled by default. The .ini file includes settings like which directories to probe and which plug-ins to use during the test run. Thus a set of nose2 tests with uncommon behaviors can be scripted precisely and re-used without hassle.
When nose2 runs, it looks for modules starting with the name test, plus any classes derived from unittest.TestCase, so it can run tests designed by way of the pytest or unittest paradigms. nose2 tests can also be configured to run when a module is executed in stand-alone fashion (something unittest and pytest also do).
Note that nose2 may not be the best choice for your first foray into testing Python apps, as it assumes some experience with creating tests. You’ll want to get your feet wet with unittest first.
behave
behave is one of a number of testing frameworks for Python that use the “behavior-driven development” paradigm, or BDD. You describe the expected behavior for your tests in a plain English “feature file,” write the tests themselves in another file, then execute behave and watch as the tests are run with annotations provided from the plain-English description.
This may be more pedantic than the usual test-writing methods, which involve writing code first. But the BDD approach is intended to make tests easier to reason about for the developer who wrote them, and easier for others—including end users and fellow developers—to understand what’s being tested for and why. behave is a good match for projects where meticulousness is a must.
behave allows you to selectively run tests from the command line by specifying “tags” or metadata that can be added to individual cases in the feature file. You can also define granular stepwise behaviors like setup and teardown procedures. And behave tests can be integrated with the Django and Flask web frameworks.
There are several other BDD frameworks for Python, such as Lettuce and Radish, but behave is among the most widely used.