Enterprise software needs tests the way buildings need fire and safety inspections. If you know certain electrical conditions or structural issues could lead to catastrophe, you look for them not just once but again and again. Likewise, if you want to guard against certain problems or conditions in your software application, writing tests for them ensures that they won’t ever escape into production.
Tests are doubly important with dynamic languages like Python. With dynamic languages, whole classes of errors surface only at runtime. Robust test suites can help smoke out those problems, and provide a way for whomever inherits the codebase to do the same.
By default, Python comes with its own unit-testing framework, and it is both useful and widely used. But many other options exist, each with its own philosophy about how to build and maintain unit tests. Following is a rundown of the default, unittest
, along with the three most significant alternatives.
unittest and doctest
unittest
is the default testing framework that ships with Python’s standard library, and it is used to create the test suites for Python itself. unittest
isn’t as far-reaching as some third-party testing frameworks, and it isn’t meant to be — it supplies just enough features to write robust unit tests for most projects. Java developers who have used the JUnit test framework should find unittest
familiar.
To create a test suite, you create Python classes derived from unittest.TestCase
, with the tests being functions in those classes. The tests can be simple equality or inequality assertions, or they can be more sophisticated. For example, the assertRaises
test ensures that a given exception is raised for a given test.
Tests can be grouped into a single file, or spaced out across multiple files, auto-discovered, and then executed. You can also provide fixtures (setup and teardown routines) for each test group, designate tests to skip under specific conditions, and control the execution behaviors of the tests.
The biggest advantage to using unittest
is that it is universally known and understood. It’s hard to go wrong with unittest
, if only because the way your test suite is written will be highly familiar to other Python programmers.
doctest
, also supplied with the Python standard library, is a complementary module. doctest
looks for comments in Python files that are formatted like Python interactive sessions, and attempts to run the commands contained within those comments to see if the results match. In this way the sample output in the docstring for a function can be verified along with the function’s behavior.
Note that doctest
is best suited for tests of simple functions where the inputs and outputs are readily understood in the console, and for tests that don’t require lots of setup or teardown to yield results. If you try doctest
on complex functions, you may end up with doctest
examples that are too unwieldy to be useful.
pytest
The pytest
project can replace unittest
entirely or augment it. pytest
applies a different philosophy to test writing, allowing a developer to quickly implement a small test suite for a project without a lot of scaffolding. As the project grows, pytest
can scale up to match, allowing more testing functionality to be layered in over time.
pytest
tests start off as compact functions that follow a certain naming convention in a module, not as a subclass of an existing test-type class. Tests are invoked by running pytest
on the module from the command line, or by invoking pytest
within code. You can create pytest
fixtures to set up and tear down test conditions, and scope those fixtures as broadly or narrowly as needed (e.g., from test-session-wide to only function-wide).
The options you can add as you go are many and powerful. Hundreds of pytest
plug-ins let you integrate tests with many third-party services and actions. Finally, the unittest
and nose
test suites can be run as part of the testing process as well, so you can integrate pytest
with existing suites without having to ditch them and start from scratch.
nose and nose2
nose
and its successor nose2
expand on tests written with unittest
and make them easier to run. nose
itself is no longer maintained; nose2
continues the project with Python 3 compatibility and some changes in functionality, but with the same underlying philosophy. We’ll concentrate on nose2
here.
nose2
differs from unittest
in two key ways: It has a plug-in architecture and uses .ini
-style configuration files to control how tests run. Many of the nose2
plug-ins are provided and enabled by default. The .ini
file includes settings like which directories to probe and which plug-ins to use during the test run. Thus a set of nose2
tests with uncommon behaviors can be scripted precisely and re-used without hassle.
When nose2
runs, it looks for modules starting with the name test
, plus any classes derived from unittest.TestCase
, so it can run tests designed by way of the pytest
or unittest
paradigms. nose2
tests can also be configured to run when a module is executed in stand-alone fashion (something unittest
and pytest
also do).
Note that nose2
may not be the best choice for your first foray into testing Python apps, as it assumes some experience with creating tests. You’ll want to get your feet wet with unittest
first.
behave
behave
is one of a number of testing frameworks for Python that use the “behavior-driven development” paradigm, or BDD. You describe the expected behavior for your tests in a plain English “feature file,” write the tests themselves in another file, then execute behave
and watch as the tests are run with annotations provided from the plain-English description.
This may be more pedantic than the usual test-writing methods, which involve writing code first. But the BDD approach is intended to make tests easier to reason about for the developer who wrote them, and easier for others—including end users and fellow developers—to understand what’s being tested for and why. behave
is a good match for projects where meticulousness is a must.
behave
allows you to selectively run tests from the command line by specifying “tags” or metadata that can be added to individual cases in the feature file. You can also define granular stepwise behaviors like setup and teardown procedures. And behave
tests can be integrated with the Django and Flask web frameworks.
There are several other BDD frameworks for Python, such as Lettuce and Radish, but behave
is among the most widely used.