Chasing bugs away

There’s no such thing as bug-free software. According to most studies, the average number of software bugs per 1,000 lines of code hovers between five and 20. Most are errors in syntax that never surface as problems. But with applications ballooning to millions of lines of code, the chance of a show-stopping mistake affecting any given application rises precipitously.

Examples are easy to find. Take Microsoft’s ongoing server vulnerabilities, Oracle’s first release of its E-Business Suite 11i, or Netgear’s router firmware released last May that continuously pinged the University of Wisconsin’s public Network Time Protocol servers, resulting in an inadvertent DoS (denial of service) attack. The perception among many IT customers is that bugginess has reached crisis proportions.

The effect of bugs on productivity is high. The National Institute of Standards and Technology in 2002 released a report stating that software errors cost the U.S. economy $59.5 billion per year. The study found more than a third of that expense could be eliminated by improved testing that enabled earlier and more effective identification and removal of defects.

To tackle all manner of software quality problems, enterprises are establishing best practices in the development phase, using third-party testing software to catch errors, and hiring third parties to inspect code after the fact.

Stopping Bugs Before They Breed

According to Jeff Payne, president and CEO of code-assessment services provider Cigital, software failure occurs for three reasons. “First, software is probably the most complex [thing] we try to build today,” Payne says. Second, the nature of software is such that no foolproof set of rules can be created that will absolutely eliminate bugs. The third reason, Payne says, is “the fact that developers and people who build software just do a very poor job of testing, validating, and building what they’re doing.”

Most analysts agree that although a separate QA procedure should always be in place, the best way to increase software quality is to have developers test as they go -- and to establish procedures that ensure business-side requirements are well-understood. Bugs often spring from common human error, but of equal culpability are poorly conceived or poorly conveyed business requirements. When something doesn’t work as intended, users don’t care whether the cause was a programmer’s slip of the finger or a misread requirements document.

“Best practices is [to] build quality in. Don’t try to test it in,” Payne advises. For Cigital, proper software engineering means specifying what is to be built, and then architecting and designing before coding and testing. Using test-driven development, code is tested early in the process, rather than waiting to test the entire system when it’s more expensive to fix problems. In addition, software quality reviews and artifact analyses help companies that build software cut costs by eliminating expensive human hours for reworking and late life-cycle testing costs.

“You cannot catch all of the bugs through QA,” says Alberto Savoia, co-founder and CTO of startup Agitar, developer of Agitator bug-testing software (due for release in early 2004). Savoia also advocates getting developers more involved in bug detection rather than leaving this function to QA personnel.