Of course, the network team started investigating -- and there were a lot of screams when they saw that the server wasn't physically in the data center.
When they finally tracked the server down at Robert's desk, they discovered that Robert hadn't seen the label on the server and had somehow failed to notice that it was powered on and in use. He'd decided -- without investigation -- that it was no longer needed.
The network team rushed to get everything restored, then ran into a bigger problem: There wasn't a backup of this server. It turned out that those managing it didn't realize that the database was saved there. They'd thought that if there was a problem they could just reinstall from the firewalls without losing anything. They had backups of the firewalls, but not of the server.
It took about five months to repair this damage, rebuilding mostly everything from scratch.
As you may imagine, the managers were not happy with Robert, and he was transferred to another position in the IT department. The data center best practices got an overhaul and all the staff a refresher course. And that server got backed up regularly from then on.
Do you have a tech story to share? Send it to firstname.lastname@example.org, and if we publish it, you'll receive a $50 American Express gift cheque.
This story, "Data center don't: Unplug first, ask questions later," was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.