Unit testing -- a form of software testing done by developers using hundreds of small, fast tests -- is a central practice at sites that are committed to software quality. By following a dictum of "unit test, rather than debug," developers and their managers identify problems early and solve them as they go, giving them confidence in the code under development and assurance that a project will not founder once development completes and the quality assurance engineers start testing.

The Bottom Line AgitarOne 4.1.1 Agitar Software, agitar.com Very Good  8.1 criteria score weight Capability 9 30% Ease-of-use 8 20% Performance 8 15% Scalability 8 15% Documentation 6 10% Value 8 10% Cost: Subscription pricing depends on project size and number of developers; starts at $37,500 per year for 8 to 10 developers Platforms: Windows XP, Windows Server 2003, Linux (kernel 2.4.22 or later); requires developers to use Eclipse Bottom Line: AgitarOne is a capable, useful, and well-integrated tool that generates unit tests automatically, runs additional tests in the background, enforces code rules, and reports the results in an elegant dashboard format. However, this first release (despite the version number) is fragile in several ways, and it relies on documentation of insufficient quality. About our Reviews and Scoring Methodology

Agitar is a company whose entire product line is devoted to unit testing. Its flagship Agitator product (see our April 2006 review) reads through Java code, creates tests on the fly, runs them in the background, and then reports the results (successes and failures) as well as the lines of code that were not exercised. The benefit of this approach is that Agitator will often create tests that developers would overlook due to excess familiarity with the code. In addition, Agitator runs phone numbers, Social Security numbers, and other common fields through a gamut of extensive prebuilt tests.

AgitarOne, the focus of this review, is the heir to Agitator. During the next year, AgitarOne will replace Agitator at most customer sites, as Agitator is being retired. AgitarOne is a software stack that expands on Agitator's capabilities in useful and innovative ways. However, it is a heavyweight solution that may amount to more software than smaller teams need or want.

Click for larger view.

Dashboard for developers
AgitarOne follows a trend that has been emerging in software development tools during the past year, which is to provide a dashboard with project metrics. (In the past 12 months, weve seen products such as Enerjy's CQ2 and Borland/CodeGear's JBuilder 2007, among others, adopt this approach.) AgitarOne's dashboard is hosted on CruiseControl, an open source product that serves as a platform of sorts for generating and executing unit-tests, applying code rules, and displaying the metrics dashboard.

The developer triggers these activities using client-side software that must be installed in a Java IDE. At the moment, Agitar supports only Eclipse and Eclipse-based IDEs (such as CodeGear JBuilder 2007 and IBM Rationals family of IDEs). This installation is done via a series of plug-ins that can be downloaded into Eclipse from the URL of the AgitarOne server. Some activities can be run in a command-line batch mode.

Once everything is set up, the developer clicks on a project or a file, and from an Agitar-specific, drop-down menu elects whether to generate unit tests, "agitate" the code, or run the code-rules engine. A queue can then be checked to watch the progress of these tasks.

The unit tests AgitarOne generates have two benefits: They can fill in test suites written by the developer, and they can capture current behavior. To its credit, Agitar is clear that its generated tests are a supplement to developer-created tests, not a replacement for them. The first of these benefits is most important where unit tests are used as a quality-validation tool. AgitarOne can create tests the developer would not have thought of and catch errors by exercising the code in legitimate ways the developer did not foresee, since a frequent blind spot for developers is their over-familiarity with code they've written. AgitarOne's generated tests do this well. In this sense, they are instructive, and regularly examining them will surely help most developers broaden their own test-writing skills. The second benefit of generated tests is capturing existing behavior so that when a developer changes the code, failing tests can highlight unanticipated consequences. This usage, termed "characterization tests," is particularly valuable when a developer is modifying legacy applications.