Why you should stage a data-loss fire drill

FREE

Become An Insider

Sign up now and get free access to hundreds of Insider articles, guides, reviews, interviews, blogs, and other premium content from the best tech brands on the Internet: CIO, CITEworld, CSO, Computerworld, InfoWorld, ITworld and Network World. Learn more.

Today's businesses breathe data like air. Have you really thought about what would happen if the lights went out and your data went away?

Our enormous appetite for data has bred an equally huge existential dependence on that data being available. It's not just the big names on the Fortune 1000 who can't live without info, either. Businesses as small as florist shops and veterinary clinics can't get by without their delivery schedules and patient records -- all of which are stored digitally.

That's progress: Digital data allows us to work more productively with less overall waste. In the early part of the last century, you could've said the same thing about electric power or the telephone. They solved all kinds of problems for humanity -- yet look at the chaos that ensues when they suddenly become unavailable in ways no one considered.

Stage your own data outage

Any time you develop a heavy dependence on anything, it's important to take a step back and imagine what you'd do without it. Unless "quit immediately and move to Mexico" is your answer, staging a tabletop data loss fire drill is a good idea.

In today's do-more-with-less culture, it might seem like an astounding waste of time to spend a few hours simply talking with your team and stakeholders about unlikely worst-case scenarios. After all, you make backups and test them (right?). What's the point of imagining what you'd do in the improbable event that they fail at the precise moment they're needed?

In every single data loss emergency I've witnessed, someone has said something akin to "Man, if we had only done X." In those cases, X could be as simple as having an automated report of the next few days' orders or deliveries be pushed out to a flat text file on a workstation. If the ERP system bites the dust and it takes days to reconstruct the data, that one step could allow production and shipping to continue while you put the pieces back together. Without it, you might as well close the factory floor and send everyone home.

Just as often, I've seen a completely unplanned recovery method save the day. In one case, a mission-critical database had been corrupted and its backups had silently failed for long enough that the most recently available backup was nearly useless. By chance, a developer had grabbed a copy of the production database only a few days before and stored it on his workstation.

That happenstance saved the company's bacon, but there's no reason it should've been a mistake. That company now automatically dumps its databases to the dev environments daily -- it not only allows the devs to work with recent data sets, but is an informal data protection layer that augments their backups.

Getting started

It's important to start the discussion by assuming that any data protection scheme you have in place has already failed or can't be exercised. If your data protection plans are really solid (if so, good for you), that might require a bit of imagination, but that's the point of this run-through.

To continue reading, please begin the free registration process or sign in to your Insider account by entering your email address:
Join the discussion
Be the first to comment on this article. Our Commenting Policies