Will the complexity of IT systems be our undoing?
ObjectWatch CTO Roger Sessions gives his take on the problem of increasing complexity and outlines some possible solutions
Roger Sessions, CTO of ObjectWatch and an expert in software architecture, argues that the increasing complexity of our IT systems will be our undoing. In fact, he just recently got a patent for a methodology that helps deal with complex IT systems.
Network World Editor in Chief John Dix recently caught up with Sessions to get his take on the extent of the problem and possible solutions.
[ Get expert networking how-to advice from InfoWorld's Networking Deep Dive PDF special report. ]
NW: Outline the IT complexity problem as you see it.
Roger Sessions: The basic problem is that the larger and more expensive an IT project is, the more likely it is to fail. You can do a lot of analysis as to why that is. You can say maybe we're not using the right methodology, or communications is failing, or any number of things. But ultimately the only variable that appears to correlate closely with failure is complexity.So my basic proposal is that as systems get bigger and more expensive they get more complex and complex things are harder to deal with and therefore more likely to fail. So if the system is under, say $750,000, it has a good chance of succeeding. Once it approaches $2 million it has less than a 50 percent chance of succeeding. And by the time it gets much larger than that, the chances of success drop to near zero.
NW: Clarify what you mean by failure.
Sessions: The way I define failure is that, if at the end of a project the business looks back and concludes it would not have taken on the project knowing what it knows now, then the project is a failure.
NW: How about some examples?
Sessions: It could be a system that was abandoned because it got too complex or people got confused and started going off on tangents. An example would be the National Program for Information Technology in the U.K. It was an attempt to build an IT infrastructure for their healthcare system. It was probably a $20 billion effort, and it was abandoned and now they're starting from scratch.
NW: $20 billion?
Sessions: Yeah. It's a system I wrote about in my last book three years ago. At the time it hadn't yet failed. But the system was huge, highly complex, and there were no policies in place to deal with the complexity. I predicted that it could not succeed. And just in the last couple of weeks they've announced that they're discontinuing the system.
That's an extreme example. Most failures are in the $2 million to $4 million range. If you look at studies of how many systems in this range are successful, the number is less than 50 percent.
So the question is not, are we failing? We know we're failing. The question is, why aren't we doing something about it? Why do we keep doing the same thing over and over again if we can see very clearly how unsuccessful it is?
NW: Before we discuss that, are those numbers based on your own research or other people's research?
Sessions: Other people's research. The Communications of the ACM in '07, for example, found that systems significantly under a million dollars have a better than 75 percent chance of success. $2 million to $3 million it drops down to 50 percent to maybe 40 percent success. $10 million plus it's under 10 percent success.
