It's one of the unwritten laws of physics: At some time or another, everybody screws up.
But when IT pros make mistakes, they don't mess around. Entire buildings go dark. Web sites disappear. Companies grind to a halt. Because if you're going to mess up, you might as well make it count.
"I always tell my guys, hey, you're gonna do stupid stuff," says Rich Casselberry, director of IT operations at Enterasys, a networking systems vendor. "It's OK to do something stupid if you have the wrong information. But if you do something stupid because you're stupid, that's a problem. The trick is to not flip out, which only makes it worse, or try to hide it. You need to figure out how to keep it from happening again."
[ For more adventures in IT mishaps, check out Stupid user tricks 3: IT admin follies and Stupid QA tricks: Colossal testing oversights ]
We've gathered up some of the more egregious examples from IT pros brave enough to share their screwups with us. Backups gone bad, people with admin privileges who probably shouldn't, what can go south when you unplug the wrong equipment -- in some cases, we've obscured their identities to spare them embarrassment; other geeks, however, are perfectly willing to own up to their youthful mistakes.
Sure, some of these mishaps are amusing in retrospect. But don't laugh too hard. We know you've probably done worse.
True IT confession No. 1: The case of the mysterious invisible backup
Our first tale of misadventure involves a longtime IT pro who doesn't want his real name used, so we'll just call him Hard Luck Harry.
Harry had his share of mishaps when he started out a decade ago at a major networking equipment maker in the Northeast. There was the time he changed an environmental variable that broke everything on his company's financial apps, earning an e-mail from his boss ordering him to "never hack on this system again." Or the time he crashed the company's core ERP system by overwriting /dev/tty. Harry says after he accidentally ripped the company's T1 lines out of the wall with his pager, he was banned from ever reentering the telecom closet.
But the worst one happened after Harry installed an Emerald tape backup system. Did he bother to read the manual? Please. This was child's play. Just load install.exe and let the software do its thing.
It seemed to work perfectly. Four hours later, the first backup completed and everything looked fine.
Fast-forward six months. Harry gets a call late one night at home from one of his work pals. That night's backup tape is completely blank, the friend tells him. Worse, the last four weeks of backups are also blank.
As Harry soon discovered, that particular backup program installs in demo mode by default. Demo mode looked exactly like real mode and even took the same amount of time as an actual backup, but nothing ever got written to tape -- a fact that was noted in the manual, which Harry might have seen had he read it.
This whitepaper explains the terminology and concepts behind Data Replication technologies and establishes some sizing rules through worked examples. Learn the new paradigm in disaster tolerance—protect data anywhere.
Download now »
Server virtualization is a popular option for dealing with mounting datacenter costs. Another equally promising approach is the use of an Application Delivery Controller. Citrix NetScaler provides a low-cost way for organizations to reduce their server count and accrue cost savings from a reduction in space, cooling, power and personnel.
Download now »
The emergence of WLANs has created a new breed of security threats to enterprise networks.
Included in HP ProCurve WLAN solutions is security technology that alleviates threats from WLANs through:
* Monitoring wireless activity inside and out of the enterprise
* Classifying WLAN transmissions into harmful and harmless
* Preventing transmissions that pose a security threat to the enterprise network
* Locating participating devices for physical remediation
Effectively address data protection challenges, implementing solutions that help store and protect business–critical data while cutting costs and improving efficiency and reliability.
Download now »
Sign up to receive InfoWorld Resource Alerts
54 Recommendations
