Writing good unit tests, Part 2: Follow your nose

Assertions, exceptions, and refactoring in test code

1 2 3 4 5 Page 5
Page 5 of 5

Refactoring: It starts with broken windows

Test code has to be written in the same manner as real production code. That means that you should include a refactoring step in your test implementation process. Your unit tests should evolve via the following steps:

  1. Use case analysis to understand the requirements.
  2. Test design as a pre-step to implement the expected unit behavior testing.
  3. Test code implementation using the test framework of your choice.
  4. Test automation with support from tools like Ant and Maven 2.
  5. Refactoring your test code to improve the quality of both the tests and your code.

The question that arises for many developers is, when is it better to refactor, versus leaving well enough alone? Well, you may have heard the phrase "Broken window, broken house"; it formed the basis for Rudolph Giuliani's policing strategy when he was mayor of New York in the 1990s. If there is one tip Giuliani can share with Java developers and testers, it's the power and simplicity of the broken windows theory. Every time you find a "broken window" in your code -- like massive nested if-else statements, code duplications across classes, or dead code in your tests -- your warning bells should start ringing. Why? They're all signs of software entropy -- a condition to avoid.

Refactoring Java code

Refactoring is basically the object-oriented variant of restructuring. As described in a doctoral thesis by W. F. Opdyke, refactoring is "the process of changing a[n object-oriented] software system in such a way that it does not alter the external behavior of the code, yet improves its internal structure." The key idea here is to properly rename, shorten, or redistribute classes, variables, and methods across the class hierarchy to facilitate future adaptations and extensions. In the context of software maintenance and evolution, the goal of refactoring is to improve the quality of software.

When to refactor

Refactoring is a powerful technique, but it needs to be done carefully. When done improperly, some quality metrics may become even worse. The main danger of refactoring is that it can inadvertently introduce errors, especially when it's is done by hand. Refactoring test code is also different from refactoring production code. You have no unit tests as a safety net to check your test code, so you have to be a more careful. Refactoring test code is a matter of making small steps (perhaps with peer reviews) and thinking twice before making larger ones.

Static code analyzers, which you learned about in the first half of this article, are among the tools that can help you find and judge possible problems awaiting refactoring. They can detect and measure weak spots by indicating problems using simple length metrics (long methods, long classes, long parameter lists), reporting violations of naming conventions, or indicating needed design modifications.

In addition to testing tools, the most powerful tool for deciding whether your test code needs to be refactored is your nose, which can be trained to quickly detect what has become known as code smell.

Code smell

Code smell was first described by Martin Fowler in his famous book, Refactoring: Improving the Design of Existing Code. Code smells listed in Fowler's book include:

  • Comments. There's a fine line between comments that illuminate and comments that obscure.
  • Uncommunicative or inconsistent names.
  • Long methods.
  • Long parameter lists. If you use a test framework that allows for parameter-driven test methods, then you should watch out for long parameter lists.
  • Duplicated code.
  • Conditional complexity.
  • Large classes.
  • Data clumps.
  • Dead code, including unused imports, variables, and constants.
  • Speculative generality.

Testers in any agile process should recognize these smells as they maintain test code. Code smells typically affect the maintenance cost of tests, but they may also be early warnings signs of behavior smells to follow.

You should look for code smells as you write your test classes using the code inspection capabilities of your IDE. Code inspectors are tools designed to assist in maintaining and cleaning up your Java code. They accomplish this by searching for dead code, bugs, or inconsistencies, and then suggesting solutions for any problem found. Eclipse, NetBeans, and IntelliJ's IDEA, for example, have built-in support for code inspection. Mark code about which you are uncertain with a "TODO" label, and look for such markers later on with your code inspector to clean up ugly code blocks.

Learning from a wise grandma

As co-author of Refactoring: Improving the Design of Existing Code (see Resources), Kent Beck apparently took inspiration from his grandmother's child-rearing motto: "If it stinks, change it!" Likewise, you will know when to change your test code, assuming you are familiar with test code smells.

When your test code package is compiled and cleaned up this way, you can perform further analysis using FindBugs, an open source tool that seeks out bug patterns. It's quite smart and can even discover synchronization problems in multithreaded programs. That's especially useful when you're writing multithreaded unit tests to check concurrency of SUTs.

If you do not want to invest much money in static analyzers, you can continue with PMD and Checkstyle. These tools are open source and easy to customize to satisfy your needs, and they detect a wide range of possible problems.

Test code smells

Once you've sniffed out and fixed the typical code smells in Fowler's book, you'll want to turn your attention to test code-specific smells. In xUnit Test Patterns, Gerard Meszaros describes the following test code smells:

  • Hard-coded test data. Does your test include lots of "magic numbers" or Strings used when creating objects? Such code is likely to result in an unrepeatable test.

  • Test code duplication. Do identical or near-identical code sequences appear many times in many tests? This makes for more code to modify when something changes, which results in fragile tests.

  • Mystery guest. When a test uses external resources, such as a file containing test data, it becomes hard to tell what the test is really verifying. In such cases, either the setup or the verification of the outcome is often external to the test.

  • Complex test code. Does your test include too much test code or conditional test logic? That makes it hard to verify correctness, and means that your tests are more likely to have bugs.

  • Can't see the forest for the trees. Is there so much test code that it obscures what the test is verifying? Such tests do not act as a specification, because they take too long to understand.

  • Conditional test logic. Do your tests contain conditional logic (if statements or loops)? If so, how do you verify that the conditional logic is correct? Does it always test the same thing? Do you have untested test code?

  • Complex undo logic. Do your tests include complex fixture teardown code? Such code is more likely to leave the test environment corrupted by not cleaning up after itself properly; it results in data leaks that may later cause this or other tests to fail for no apparent reason.

There are also a number of behavior smells -- that is, smells that jump out at you while you are running your tests:

  • Fragile tests. Do your tests fail, or fail to compile, or fail every time you change the SUT? With such tests, you need to modify much of them to get things "green" again. This greatly increases the cost of maintaining the system. Contributing code smells include test code duplication and hard-coded test data.

  • Fragile fixture. Do your tests start failing when a shared fixture is modified -- when new records are put into a database, for instance? This could be a result of tests making assumptions about the contents of that shared fixture. A contributing code smell is the mystery guest.

  • Interdependent tests. When one test fails, a number of other tests may also fail for no apparent reason; this may be because they depend on a previously run test's side effects. Tests that cannot be run alone are hard to maintain.

  • Unrepeatable tests. You may find that your tests cannot be run repeatedly without manual intervention. This is often caused by tests not cleaning up after themselves properly and preventing themselves (or other tests) from running again. The root cause of this problem is typically hard-coded test data.

  • Test run war. Do you encounter seemingly random transient test failures? You may note that this occurs when several people are testing simultaneously; if so, it's probably caused by parallel tests interacting with each other through a shared test fixture. This problem can arise with multithreaded tests.

Unfortunately, there are no specific rules that come along with static code analyzers to identify these smells and propose corresponding refactorings (though you are free to write custom rule checkers if you're feeling ambitious). It's up to you to look at your test code very carefully and to check for possible test smells. As the second step, you can try to carry out refactorings as proposed by Gerard Meszaros or Arie van Deursen and his coauthors to clean up your code. Test refactorings are changes to test code that do not add or remove test cases. Because you have no safety net (remember, you are refactoring the test code itself), the production code can be used as a (simple) test case for the refactoring: If a test for a piece of code succeeds before the test's refactoring, it should also succeed after the refactoring. This obviously also means that you should not modify production code while refactoring test code (just as you shouldn't change tests when refactoring production code).

You can also try to implement an even more defensive test coding strategy, by using test patterns, as Meszaros describes in xUnit Test Patterns, or Jon Thomas and others in Java Testing Patterns. With test patterns, you can implement high-quality test code right from the start. Furthermore, you should design your production code to unit test by following the advice given by  Akshay Sharma. He tells us to follow some general design rules, like "program to an interface, not an implementation"; "favor object composition over inheritance"; "avoid unnecessary singletons"; and "leverage the use of dependency frameworks." All these techniques force you to think about unit testing up front.

When are your tests good enough?

To determine when you can stop writing tests, you should ask the following questions (taken from James Bach's "A framework for good enough testing"):

  • Are your tests covering the aspects of the product you need to cover?
  • Are you using a sufficient variety of test techniques or sources of information about quality to eliminate gaps in your test coverage? In other words, do you write good test cases?
  • What is the likelihood that the product could have important problems you do not know about?
  • What problems that your testing should have found are reported through means other than your test process?

Answering the question "When are my tests good enough?" is a lot like knowing when to stop upgrading your personal stereo equipment: you can always buy better amplifiers, better loudspeakers, or better CD players. The problem is to find the right balance between cost and benefit. One last thought I'll leave you with is: look at test code with the eye of an artist or painter -- see if your tests are beautiful in three ways (as described in "Beautiful Tests" by Alberto Savoia):

  • Beautiful in their simplicity. With a few lines of test code, you should be able to document and verify the target code's basic behavior.

  • Beautiful because they reveal ways to make code more elegant, maintainable, and testable. In other words, tests should help make code more beautiful. The process of writing tests often helps you realize not only logical problems, but also structural and design issues with your implementation.

  • Beautiful in their breadth and depth. Very thorough and exhaustive tests boost the developer's confidence that the code functions as expected -- not only in some basic or handpicked cases, but in all cases.

Implementing a good design and architecture for your code (both in testing and in production) has a lot to do with elegance. Quite often, elegance equates to simplicity. To quote Albert Einstein: "Things should be as simple as possible, but not any simpler." This quote could easily refer to test architecture, design, and code. It's this minimalist view that generally yields the best results, and refactoring is the path to this goal. So good test are also beautiful tests, and beautiful tests are simple, insightful, and widely applicable.

Keep the conversation going

If you agree with James Gosling, who once said "I don't think anybody tests enough of anything," it's obvious that we need more tests of better quality. Of course, not everybody can follow Google's testing on the toilet initiative. Google posted flyers about everything from dependency injection to code coverage in the bathrooms of its various headquarters -- almost 500 stalls worldwide -- to educate employees about testing. The rest of us, however, may have to learn about testing in more standard ways.

Even so, I hope the best practices in this article will help you improve your unit tests. If you come across examples of good or bad tests, I'd like to hear about them. That includes my own test code examples -- if you have suggestions for improvement, I have an open mind for constructive criticism. Learning together, we may someday end up with "GUTs all over the place" (to quote Alistair Cockburn). And in the meantime, we should always code as if the person who ends up maintaining our (test) code will be a violent psychopath who knows where we live ...

Klaus P. Berg has a master's equivalent (Diplom) in electrical engineering and applied informatics from the University of Karlsruhe in Germany. He was an architect and implementor for projects at Siemens focused on Java GUI development with Swing and Java Web Start, and he also acted on the server side, creating Java-based intranet applications. Now he works as a senior engineer in the area of software quality, focusing on functional and performance testing, mainly for JEE software.

Learn more about this topic

More from JavaWorld

This story, "Writing good unit tests, Part 2: Follow your nose" was originally published by JavaWorld.

Copyright © 2009 IDG Communications, Inc.

1 2 3 4 5 Page 5
Page 5 of 5