Backing into disaster

“What is the best way to fire half my IT staff with minimal disruption?” It’s not often that I receive calls asking me questions as pointed and desperate as this one, but I got just such a call from a friend of a friend who recently had been given the responsibility of running IT at her company. Although she doesn’t come from a purely technical background, she knows enough to understand her situation is dire. Four months of company e-mail has been lost and simply can’t be recovered by the IT staff. They have tried everything, and she’s ready to pull the plug on those involved — and I really can’t blame her. Sometimes bad things happen in IT, but such a massive failure is beyond the pale.

Free IT resource Virtualization Insights from Top Experts - Learn how virtualization gets real! Sponsored by Dell Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft

I gave her more emotional support than advice and wished her luck. The discussion gave me chills — who hasn’t lost key data at least once? — to the degree that I made a mental note to review our backup and restore capabilities here at InfoWorld. I never want to learn that lesson the hard way.

All too often, IT departments obsess over backup policies, but forget the most important part: the restore. Every day, all over the world, backups are picked up by data management companies and ferried to mysterious off-site facilities that can withstand the worst natural and human disasters. Be it war, fire, hurricane, or locusts, your tapes are safe. But what about recovering the data you dumped on them? In my friend’s unfortunate case, the IT staff had dutifully performed regular backups. There were plenty of backups available when disaster struck — just not any good ones. Those tapes could have been in Dick Cheney’s briefcase in a safe, secure, “undisclosed location” and it wouldn’t have mattered because no one had taken the time to see whether they could actually restore from them.

Such failures happen for a number of reasons. If you ever have to explain one of these disasters to company management, make sure you polish your résumé and clean out your desk beforehand. One of the most prevalent causes is simple but just plain stupid: When backups run slowly, to save some time, some administrators turn off the essential but time-consuming “verify” feature built into most enterprise backup software. This sort of backup and restore strategy is the IT version of Russian roulette. Not verifying backups because your data set is getting too large to back up quickly is a double whammy because you put larger and larger data sets at greater risk.

Another consistently overlooked area is backing up client desktops. Desktop backups are seen as too painful by many IT departments, which frequently instruct end-users to manually back up important documents to a central server. Yet users generally don’t work that way, and even if they did, backing up critical preferences and installed applications goes by the wayside. As a result, when a hard drive in a laptop goes bad, all hell breaks loose. It doesn’t have to be that way. With reasonably priced, outsourced, over-the-Internet backup solutions such as Connected DataProtector available by subscription, full desktop backups are simply no longer a pain. Which makes neglecting desktop backups shortsighted and, ultimately, inexcusable.

If you don’t verify your server backup tapes or you disregard desktop backups simply to save a little time, you probably will have some extra time soon enough — to watch The Oprah Winfrey Show and look for another job.