DevPartner Fault Simulator takes on .Net fault handling

Version 2.0 improves integration, helps developers and testers "ruggedize" .Net code.

In electronics engineering, a fault simulator predicts what will happen to a circuit when a component malfunctions, without having to actually install a bad component in the circuit. DPFS (DevPartner Fault Simulator) applies a similar idea to software: It shows you what will happen to your .Net application when an abnormal situation occurs, without having to actually create the situation.

Fault Simulator 1.0 arrived in 2005, to mixed reviews. InfoWorld’s previous review praised the concept but raised questions about the utility of the implementation. The biggest concerns were DPFS 1.0’s lack of integration with other Compuware tools, and its inability to affect classic C++ exception handling.

DPFS 2.0 still does not simulate faults for classic native C++ exception handling, although it does simulate faults for managed C++. It also now integrates with Compuware’s Coverage Analysis tool and has a number of other improvements that will make .Net developers’ and testers’ lives easier. However, there’s room for growth -- the feature set and documentation still have some rough spots.

Fault-Finding Mission

DPFS 1.0 showed developers where it could simulate .Net faults by displaying curly underlines in Microsoft Visual Studio. The underlines appeared beneath lines of code calling .Net functions that DPFS recognized as ones that throw exceptions.

DPFS 2.0 improves on that display, posting an icon to the right of each line of code with a possible exception, and another icon at the end of each method that can throw exceptions. If you click on the icon at a line end, you can add simulated .Net faults for the methods called as well as add surrounding try/catch blocks for any or all of the possible faults. This advances DPFS from merely simulating faults to also generating some of the code to handle the faults.

Click for larger view.

Click on the icon at the end of a method, and you can add XML documentation comments for the possible exceptions to the method, indicating that you want to allow exceptions to be caught by callers of the method. Once you’ve built the application, the new XML comments can be read by DPFS and used to create simulated faults for the methods they document.

I think the try/catch block generation is a major productivity boost for a developer trying to make an application more robust. DPFS doesn’t actually write the body of the catch clauses, just the block skeleton; you still have to think about what action is appropriate to handle each fault.

I was pleasantly surprised to find that if you have multiple successive lines of code that can generate the same fault, DPFS 2.0 correctly surrounds all of them with a single try/catch block.

For the Code Testers

Using DPFS with Visual Studio is great if you’re the developer and have all the source code. But what if you’re the tester? DPFS 2.0 has three new features for you: a standalone Fault Simulator application that allows testers to simulate faults without Visual Studio; a command line interface (CLI) for setting up automatic regression tests, and a “Watch My Target” function that treats an app as a black box.

“Watch My Target” is the most interesting addition. It monitors an application as it runs and determines what resources it uses. After the observation phase, it lists the activities and offers to create environmental faults to simulate failures of these activities. The program handles five types of environmental fault: COM, Disk I/O, Memory, Network, and Registry.

When DPFS simulates a fault, it injects code into the application at load time to intercept the appropriate action and generate the desired fault condition. It then monitors and logs what happens at run time; if the code throws an exception, DPFS captures the entire call stack, making it easy to find the offending line of code. If code coverage integration is enabled, DPFS also tracks which lines of code have been tested by the fault simulation.

If you have Visual Studio Team System, you can submit a work item of type Bug from DPFS, either from a fault instance during an active session, or from the simulated faults report after a session is completed. DPFS also integrates with Compuware TrackRecord for defect submission.

Code injection does take time, and the application runs slower when you are monitoring it for fault simulation. DPFS 2.0 can suspend a running fault simulation, which mitigates the slowdown somewhat. Suspending the fault simulation makes it easier to test specific parts of an app, and also makes it easier to skip over program initialization and get to the code you want to test.

Making an Effort

I find DPFS 2.0 much improved over its predecessor and a useful tool for serious .Net developers and testers. The standalone Fault Simulator is a good addition, and the integration improvements are very welcome.

Remember, however, that this is a specialized product that is primarily an add-on to another product (Visual Studio) -- so its capabilities are on the specialized side, and its price is on the high end of the scale.

The documentation could also be better organized: I think all the information is there, but some is in PDF form, some in HTML, some in compiled HTML help, and some integrated with Visual Studio. To understand the product and documentation, it helps to be familiar with the concept of software fault injection, as well as with the .Net framework and its structured exception handling. But if you’ve been waiting for the next release, DPFS 2.0 is worth checking out.

InfoWorld Scorecard
Ease of use (15.0%)
Value (10.0%)
Documentation (10.0%)
Performance (15.0%)
Integration (20.0%)
Capability (30.0%)
Overall Score (100%)
Compuware DevPartner Fault Simulator 2.0 8.0 7.0 6.0 8.0 8.0 7.0 7.4