Code catcher in a box

The BugScan appliance could be a developer’s best friend: It catches potential security holes in your application code

A programmer, even a good programmer, can produce insecure code. Not because of a mental slip-up, but simply because the programmer is focusing on solving the problem at hand (and meeting a deadline) rather than safeguarding that code from exploitation by some hypothetical hacker. But, as the Blaster worm's mayhem emphasized, software must do more than simply work well -- it must also run securely.

HBGary's BugScan is a static program analysis tool that locates an application's potential security problems, sections of code that could be attacked to crash the applications and, possibly, the host system.

BugScan analyzes binaries: C/C++ EXEs (Windows or Linux x86) and DLLs. The tool's static analysis traces all of a program's paths of execution, and from that uncovers insecure coding patterns. I tested the 2003 release of BugScan, and found it useful as part of a larger testing strategy, though its interface still needs a little smoothing out.

The Analyzing Appliance

BugScan arrives as a software/hardware combination appliance: a Dell PowerEdge 650 with a 2.4GHz Pentium 4, 256MB RAM, and 80GB of disk storage.

Installation could not be simpler. Plug the appliance in, connect an Ethernet cable, turn the system on, and dust off your hands ... you're done. BugScan appears at a nonroutable (internal) IP address. The system I tested was configured to, so I pointed my browser at the URL, and the log-in page appeared.

BugScan comes configured with two log-ins, "admin" and "guest." The former has full administration privileges; the latter has access to BugScan's analysis capabilities. Only five users can be simultaneously connected to BugScan; that’s the standard configuration, but you can get support for more users if necessary.

Putting BugScan to work on a file amounts to selecting the "Analyze a File" screen, entering the path to the file into a textbox, and clicking the submit button. BugScan downloads the executable to the appliance, and places an entry for the file in an input queue. The analysis loop in BugScan pulls items off the queue, processes them, and puts the results in a list of output reports on the Report page. It works at a nice clip -- somewhere between 100K and 1MB per second, depending on size and complexity of the code being processed.

BugScan's analysis is entirely static. It reads the executable file, partitions the codes into blocks, and plots how execution will flow from one block to another when the program executes. Once BugScan has the code mapped out, it uses pattern recognition to locate sections of code that are potential security risks. The process detects such situations as insecure library calls, potential buffer overflow sites, possibly problematic signed/unsigned conversions, poor randomness (which can create insecure cookies and IDs), and format string problems (which allow an attacker to snoop into memory via poorly-formed printf()-family calls).

This sort of analysis has its pluses and minuses. On the minus side, the code is never executed, so BugScan only identifies potential problem code. In other words, it's possible that BugScan will flag false positives, and the version I used doesn’t even provide a way to filter the tests.

On the plus side, BugScan travels through all possible avenues of code execution. Consequently, it achieves 100 percent coverage, something that is extremely difficult and time-consuming to accomplish with run-time tests. Based on these pros and cons, BugScan is best employed as part of an overall testing protocol that involves both static and dynamic (run-time) exercises.

Just the Facts

BugScan's report format has no distracting graphics, fonts, or icons; just a header and a table. The header includes the number each of red, yellow, and green alerts. Red alerts are the most severe potential violations, yellow less so, and green alerts indicate secure, well-written code sequences.

Beneath is a table of four columns: Name, Severity, Risk, and Description. The first column is a brief name for the error found, such as "no stack protection." The Severity column lists a number from 1 to 3, indicating the alert level of the problem. The Risk field categorizes the error by type, such as "overflow."

The most useful column is Description, which contains a rather lengthy explanation of the issue found. In many cases, the field begins with the hyperlinked text "Code Sample"; select that link, and a new window pops up containing an example of code illustrating the problem. The code is reasonably well-commented, so you don't waste your time searching for the source of the error.

The Name cells are also hyperlinked. Click on a name, and the bottom of the window is populated with a list of all the hexadecimal locations in the executable where the problem was noted. Initially, I bristled at this presentation: A hexadecimal offset is not exactly user-friendly. I wished for a link into the source code, but that wasn't possible; BugScan processes only the executable.

Luckily, you can track the problem to its source using a method briefly described in the BugScan documentation. You need the symbol information for the executable, which is stored in the PDB (program database) file, and a free open source program called CrashFinder (available as a download). CrashFinder takes the hexadecimal offset (which you can cut and paste out of BugScan) and -- with the help of the PDB -- returns a source file name and line number information. It's not glamorous, but it works.

BugScan will also generate an XML document of the report, which you can save on your file system or in your favorite XML database. The version I tested simply refused to produce the XML document, but HBGary gave me Internet access to its newer version, 2003A (scheduled for release in mid-March), that happily handed me XML when I asked.

Fast — For a Price

Although its examination is thorough, BugScan's current user experience is a little rough around the edges. For example, once a file is in the analysis queue but before analysis is complete, BugScan is somewhat at a loss as to the file's status. The tool literally tells you that either there's been a problem, or BugScan is still working on the file. The solution? You just have to wait patiently, and if the results don't show up in, say, a half hour or so, then something must really be wrong. I pointed this problem out to the BugScan engineers, and they acknowledged the problem, assuring me that it will be addressed in future versions.

The final tally: BugScan is a mixed bag. I must admit that installation was about as easy as installing a small kitchen appliance. BugScan is completely self-contained; you don't have to worry about OS or hardware compatibility, or possible negative interactions with other applications. I can see that the arrangement is easier for BugScan's engineers, too: They needn't expend QA time testing their product on countless OS and hardware variants.

On the other hand, you pay for hardware when what you're really buying is software. And, since BugScan is a closed system, it appears that the only way to back up past reports is either to save the HTML screens or download the XML into a file or database. In either case, you have to back up reports one at a time.

BugScan costs $19,500, which includes one year's maintenance; after that, the maintenance fee is $3,900 per year. That's pretty steep. But if you need a software security analysis tool that's a snap to install, and that chugs through code at a swift pace while catching your unintentional errors, BugScan might be worth it.

InfoWorld Scorecard
Setup (20.0%)
Value (10.0%)
Security (10.0%)
Performance (30.0%)
Ease of use (30.0%)
Overall Score (100%)
BugScan 9.0 5.0 7.0 8.0 9.0 8.1