End the holy war over agile development

Out with stale debates about agile vs. waterfall - versatility is the new agility

IT leaders looking to move their organizations forward should check out the YouTube video "I want to run an agile project," this week's winner of the Best Inadvertent Example of How Not to Behave in a Corporate Setting Award.

What you'll hear if you can bear the 10 minutes: a project manager named Luke repeating over and over again, to everyone he has to deal with, in a distinctly whiny tone, "But I want to run an agile project."

[ Also on InfoWorld.com: Get Bob Lewis's continuing IT management wisdom in his Advice Line blog and newsletter. | Find out why running IT as a business is a train wreck waiting to happen. ]

Here's a piece of advice for all the Lukes out there: "I want" followed by a direct object is not a compelling business case. "Here's why you should want me to manage this as an agile project," followed by specific benefits is persuasive instead of fingernails-on-a-screen cringe-worthy. It would keep Luke out of trouble, too, because in tomorrow's IT, development methodologies will be chosen based on the situation, not personal preference or trend-spotting.

Why agile is essential for next-generation IT

Depending on whether you're a proponent or a detractor, agile is either a family of methodologies or a religion composed of one true denomination and a handful of variously misguided sects whose only virtue is being less misguided than the waterfall contingent. The same may be said for waterfall, only backward.

In fact, for all the religious fervor, the question of which methodology is superior can only be answered by the phrase business managers like least from consultants: "It depends."

Without a doubt, next-generation IT organizations will have to master agile. Why they'll have to master it has nothing to do with "I want to run an agile project" and everything to do with "it depends." Specifically, "it depends" boils down to two factors: how fast a company needs a solution, and how long that solution will have to last. In an increasing number of business situations, deadlines and lifespans are getting shorter all the time -- criteria for which agile is well-suited.

The upside of agile is that businesses that implement it are great at adapting to change quickly. That's because the software supporting the business is built in small chunks that are deployed quickly and bring business benefits early. When business circumstances change before the next chunk is supposed to be ready, it doesn't matter, because agile designs are just in time, not all at once.

Agile is great at one more thing: graphical interfaces that fit how users want the software to operate. That's because agile calls for high levels of informal interaction between developers and users, without any need for intermediaries.

Does that make a business more agile? It sure does, because the better the software fits how users think about their work, the less time they'll need to learn it, and the less often they'll misuse it when taking care of oddball situations.

Waterfall's role in tomorrow's IT

Agile has its downsides, too. One is enforcing a consistent application architecture. It's a solvable problem, but as is often the case with compliance issues, most solutions cause delays in the form of external reviews. These make agile, well, less agile.

Or maybe it makes more sense to use waterfall when excellent internal engineering matters greatly -- when durability is more important than quick delivery.

Take accounting, for example, where debits always have to go on the left, credits always belong on the right, and if transactions aren't ACIDic (atomic, consistent, isolated, and durable) the books won't balance. Given that the field hasn't changed all that much since Pacioli invented it 500 years ago, whatever it is you're developing for the accounting department will probably last a while.

Then there are instances that depend on complex business logic. A secret of which we dare not speak is that, when complex business logic is involved, business managers rarely take the time to turn their ad hoc decision making into consistent, teachable rules. Usually, it takes a strong business analyst from IT to untangle the threads.

But agile takes business analysts out of the loop, preferring direct interaction between developers and users. Complex business logic inevitably pushes these types of projects in a waterfall-ish direction.

For that matter, when IT untangles complex business logic and codifies it in an application, it automatically makes the business more integrated and consistent. This, interestingly enough, makes the business less agile by definition, because coding business logic into software is very much like drafting business decisions into the "policies and procedures manual." Anything that's official is harder to change than something that's improvised on the spot.

The code-reuse conundrum

Here's a paradox: When developers can call an existing, well-tested module instead of having to write new code, they'll finish a given assignment faster. That makes them more agile, doesn't it? Well ...

Sure, so long as (1) they aren't the one who has to write the reusable module in the first place; or (2) the module fits their situation well and isn't a rounded hole into which they have to push their oblong peg.

Writing reusable modules takes more time and care than writing something for one-time use. "More time and care" doesn't sound very agile, does it? I didn't think so, either. It's a short-term/long-term issue, only agile really should be about being good at the short-term stuff, shouldn't it?

As for the pegs-and-holes situation, it's similar to what businesses face when implementing ERP suites. Software is simply an opinion of how business should run. However, when a business manager's opinion about how to run their part of the corporation differs from what the ERP system's designers had in mind, implementations become chancy. Adapting the system takes longer; as a result, everyone ends up having to change their procedures to do what's natural to the software.

Other than the price tag, there's no difference between "reusing" code embedded in a commercial ERP suite and reusing internally developed code. Either way, business managers either share the software designer's opinion or have hard decisions to make.

Moving IT beyond agility

Where does this leave someone trying to take IT into the next generation?

Every business of any size and age has some areas subject to rapid and unpredictable change, and others that are, for the most part, stable. IT has to be able to support both, which means using agile and waterfall techniques based on what the situation demands, not personal preference.

In other words, next-generation IT has to be something even more difficult than agile: It has to be versatile.

This story, "End the holy war over agile development," was originally published at InfoWorld.com. Read more of Bob Lewis's Advice Line blog on InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

Copyright © 2011 IDG Communications, Inc.