True IT confessions

Imagine you are working with a contractor to build a new house. At the final walk-through, the house itself looks great and everything works, but the circuits in the electrical box are unlabeled, there are a few loose wires when you open one of the kitchen cabinets, and there's a pile of sawdust in one of the bedroom closets. Your house is technically complete, but are you really ready to move in with these little bits of disarray? What else are you going to find when you open the bedroom closets, much less crawl underneath the house?

Free IT resource Hear how top CIOs turn change into a competitive advantage. Sponsored by HP Free IT resource Try Sun servers, workstations and storage products free for 60-days. Sponsored by Sun Microsystems

Now think of the contractor above as the IT organization within your company, and the new homeowner as your business. In corporate IT, random piles of sawdust collect every day, only with IT, the piles are usually invisible to the business until something goes wrong (think Y2K). Because your servers are hidden away in a datacenter and software is just plain ephemeral, the dirty little secrets of your IT department generally remain hidden from view -- but, oh, they can be ugly. 

One problem is simple: documentation, which sometimes hides the ugliness from the IT department itself. I’ve written before about the power of Weblogs to keep documentation up-to-date, since documentation for production systems is usually an organic beast. However, most developers and systems administrators finish a project and quickly put it to bed, sometimes without any documentation. Many times, I have scanned a particular system's documentation during a crisis only to find that it had not been updated in several months (or even years). Small and undocumented changes can add up to large, undocumented messes. To battle this tendency, I've set up weekly staff meetings to discuss all the changes that have been made in the previous week. This tends to help us keep all the sawdust swept up. (I still long for the days of IT budgets that included technical writers and documentation specialists.)

The forces of entropy figure into the equation, as well. No matter how much time you devote to IT priorities, something is going to be last on the list and will be ignored in favor of more critical priorities. But how many things at the bottom of the list will go unnoticed if they break? Not many, and as we all know, the item that is last on the priority list is often the first to break. Most of the time in this column, you hear about the cool stuff we are doing, but there are skeletons in my IT closet, and every CTO I know can rattle off a list of five things that could go wrong at any moment.

So, here are a few of my confessions for your entertainment. Until recently, we were still running Windows NT 4.0 to power our Lotus Notes environment. We had just replaced a key server that had been set up three years ago, and no one really knew how it worked, so we simply backed it up well and prayed daily. Right now, we are running our prepress publishing server software on a blueberry iMac -- not exactly the ultimate server by any measure. Finally, with tight facilities and IT budgets, the fire suppression system in our server room is currently a water-based sprinkler. All of these issues are being corrected, of course, but I'm sure many of you have your own nagging ones -- write to me and I'll post the best ones to my Weblog.