Last week I summarized the difficulties in preventing buffer overflows in complex software and introduced fuzzers. With multiple buffer overflows announced every week on some of the world’s most popular products, I want to continue the discussion on the practical realities of vendors trying to prevent them. If the biggest software companies, with their vast resources and talent, have a hard time preventing buffer overflows, can any complex software product be built without security flaws?
Again, I turned to buffer overflow expert David LeBlanc for a deep discussion of the issue at hand. David was a chief bug finder for Microsoft for six years and is now working at Webroot as the chief software architect. Here’s what David had to say about the difficulty of flawless code-writing:
“Software bugs are measured in total number of defects per 1,000 lines of code [kloc]. Think of an application as having two measurements -- the first is total number of defects per kloc, and the second is the percent of all defects that have security implications. A typical programmer with typical development practices will have around 50 defects/kloc. A really good developer, aided by peer code review, can drive the defect rate down to five defects/kloc. Also, note that if we learn to recognize a new type of defect, the count could go back up.
"That's still 5,000 defects per million lines of code, and a complex application can run up to around 3 million to 5 million lines. Operating systems and associated helper apps and services can run up into around 50 million lines. A typical utility app will run around 250 kloc.
"Code review isn't perfect because we're relying on people. They'll be less effective before the coffee kicks in, less effective right after lunch, and it will vary from person to person. People also have problems maintaining state, and will tend to notice bugs that are local, and won't often find bugs where the mistake was three function calls ago and the consequences are local. So code review is great, but not perfect.
"Next, we have automated tools -- these are getting better all the time. These are great; they're consistent, but there are limits to what they can be taught to find.
"Both of the previous measures will tend to reduce defects/kloc without regard to exploitability. A good development team will eliminate defects without bothering to see if they are exploitable. There were several instances where Office 2003 was not vulnerable to a problem reported in a previous version because someone cleaned up some code without realizing they were actually fixing an exploitable condition. Many other projects, notably OpenBSD, advocate the same approach. Theo de Raadt has long advocated fixing bugs without regard to exploitability.
"Next, we start getting into mitigations. For example, if we have stack protection enabled, there will be some defects that were previously exploitable that are now not exploitable, others where it may be harder or that will force new techniques to be developed, and still yet others where the mitigation has no effect. The rule here is that for any given mitigation, there will be some defects that become completely non-exploitable, and for a sufficiently complex set of attack conditions, any given mitigation can be circumvented. An important thing to note here is that these often overlap, so multiple mitigations increase effectiveness quite a bit.