Agitator stirs up unit-test drudgery

No one doubts the benefits of unit testing. Nor does anyone doubt that unit testing can be as tedious as it is important. It’s like going back to check your carries and borrows in a math problem.

Free IT resource Open Source Business Conference (OSBC) May 22-23, 2007 Sponsored by OSBC Free IT resource Virtualization Insights from Top Experts - Learn how virtualization gets real! Sponsored by Dell Agitator 1.5 Agitar Software, agitar.com Excellent  8.7 criteria score weight Ease-of-use 9 20% Performance 8 20% Reliability 9 20% Setup 9 20% Documentation 8 10% Value 9 10% Cost: $150,000, 50-person deployment; $225,000, 100-person deployment; prices are estimates because deployments are often customized Platforms: Windows 2000, XP Professional, or later; Red Hat Enterprise Linux Version 3 (or Linux kernel v2.4.22 or later) Bottom Line: Agitator is a pricey development adjunct, but its capabilities are extensive and worth the investment. If your organization is doing serious Java work, Agitator makes building unit tests almost enjoyable. And that’s saying a lot. About our Reviews and Scoring Methodology

But none of the Java unit-test tools I’ve seen so far are quite as good as Agitar Software’s Agitator 1.5, which automates unit testing to the proper degree by recognizing the point at which it needs human help. Most unit-test tools require you to craft the unit tests from the get-go. Agitator builds boatloads of unit tests that you never really see. You simply see the results and pick which observations are (or should be) correct.

That’s just the start. Agitator also tracks code coverage and attempts to exercise as much of the target code as possible. If you’ve compiled the target classes with the debug switch set and the source code is available, you can use the Agitar IDE to explore your source and identify which lines escaped testing.

Agitator is more than a black-box testing tool. It actually performs flow analysis of methods’ internals. Information gathered by the analysis guides the creation of tests and improves code coverage. Agitator can “backtrack” a variable’s usage to make educated guesses about how variables influence decision points in the code. This permits the tool to adjust its tests in order to explore code that would otherwise remain untouched.

Agitator’s IDE uses a project paradigm to organize your work. After defining a project and selecting target Java classes, you can literally turn the tool loose on your code. As tests are created and executed, Agitator finds relationships among variables or incidents of variables taking on specific values.

Agitator expresses this data using comparison statements.  You can drill into the results to determine at which lines in the source code the expressions were true, as well as to see which test cases caused expressions to become true.

This is where you, the human being, lend a hand. Expressions that must be true, you set as assertions. Agitator will exercise your assertions and verify them. Some expressions describe behavior that shouldn’t happen -- that’s a bug.

The expressions you write to describe code can be quite elaborate and therefore handle wide ranges of circumstances. If Agitator’s automatically generated tests are not complex enough for your application’s classes, you can draw on the wide set of built-in factory objects.

Agitator has three levels of “agitation.” Normal is fastest, exercising code the minimal amount of time to get useful information. Extended agitation adds more time, a wider variety of input values, and more algorithms for exploring code. Finally, aggressive agitation calls each method as many times as possible to achieve greatest coverage.

I was impressed with the complexity of some of the relationships that Agitator discovered during my tests. In addition, Agitator was smart enough to phrase the relationships with expressions that used “getter” functions, rather than variables themselves. This process insulated the expressions from changes to the method’s plumbing.

Agitator is also capable of identifying an item’s state before and after a method call, and it can express that in the relationships it discovers. This allowed me to form assertions that tracked how a variable’s state changed.

And if that’s not enough, Agitator includes extensive support for mock objects, even a “mock database” for unit testing that depends on a database back end. I never thought I’d enjoy writing unit tests, but with Agitator, I certainly did.