Coverity and Klocwork code analyzers drill deeper
Industry leaders show remarkable scalability and prowess but differ in design philosophy
A key differentiator is that K7 can analyze C, C++, and Java, whereas Coverity's product works only on C and C++. K7 can perform analysis based on Java source code and bytecodes, the latter being Java's form of executable file. If the bytecodes contain debug information, K7 can trace defects back to specific lines of code. If not, it can simply identify that a certain type of bug has been found. This option enables sites that rely on third-party Java components to screen them for possible defects before use and to identify the type of defect to the vendor.
Overall, this is a comprehensive and very impressive package, made available at a remarkably low price.
Coverity Prevent 2.2.2
The Coverity tool emerged from academia; in many ways, it retains the feel of that environment. Whereas Klocwork K7 provides comprehensive analysis tools and a well-designed set of supporting utilities, Coverity Prevent is a pure analyzer with a simple interface. It has no management console. The only way to see what has changed between runs of the analyzer is to run diff -- a programming utility from Unix that identifies what has changed in a source file. Whatever differences it comes up with, that's what's new. Dashboards or other displays of project status are nonexistent.
Because Coverity is limited to C and C++, it has good representation in embedded contexts. As a result, it works on a very wide variety of platforms and with an enormous number of different compilers -- far more than K7.
Coverity's Unix-like aspect is visible in how it does configuration. For example, to limit the number of false positives, it enables you to provide detailed configuration files that are then compiled with the project, or stub functions that redirect Coverity's checkers, or annotations that are placed as comments directly in the source code. (This last option is of doubtful value. Few sites will change large code bases to accommodate a static-analysis tool.)
The Unix scripting approach is also evident in how the code scanner works. And in this respect, the products are distinctly different. In all tests, K7 found more defects than Coverity. Stripping out false positives still left K7 ahead in total bug counts. However, some defects reported by K7 are close in nature to the items lint reports, whereas Coverity kept far away from reporting these issues. If I removed those items from the bug counts, the products had comparable defect counts.
An important question is, Which approach makes more sense? Personally, I think that if a product finds an undeniable bug, it should be reported -- regardless of whether it seems like a bug for lint or not. This is the path that K7 wisely chose. If for some reason you don't want those results, you can filter them out of the report or display. But always keep the option of seeing them. Coverity does not include such defects at all. If you want them found, you must script your own extensions to the analyzer. Coverity provides samples of such scripts, but it does not build them into the product. This approach reflects the Unix orientation, where anything can be done by writing scripts or using little languages. This view seems valid for Unix, but it's hard to accept in an enterprise-level bug-sniffing tool. And certainly at Coverity's price, you should reasonably expect every bug to be reported without writing, testing, and implementing your own extensions.
Both packages are large and have many features, so installation and configuration take time. These are both true enterprise tools, so evaluations should be done with deliberation and careful consultation with sales engineers from the respective vendors. Fortunately, trial licenses are available along with considerable assistance in performing evaluations.
Both products are admirably effective detecting hard-to-find bugs, especially cross-functional defects. Their results are comparable and this measure should not serve as the primary basis for comparison. I prefer Klocwork K7 because it is a more complete tool and is less expensive. Not only does K7 cover more languages, but it has a superb console/dashboard for managing analytical runs and their numerous generated results. In addition, I believe Klocwork's approach to bug identification is superior. In counterpoint, Coverity's strengths are its great flexibility and its capability of running on numerous platforms.