Admin tip: Leave well enough alone, dummy

If you weren't a little OCD, you wouldn't be in IT. Here's how a nagging desire to put everything in its place can wreak havoc

There are times when you just can't resist arranging things in perfect order, even though it doesn't really matter -- times when the minor or nearly nonexistent problem you decide to address turns into an avalanche of work out of nowhere, times when you wish that there was an Undo button outside of the computer, times when it wasn't broke, but you decided to fix it anyway.

Sadly, I have examples aplenty, like the time a power cord was threaded incorrectly through the back of a data rack that prevented the rear enclosure doors from closing properly. The power cord ran to a switch connected to only back-end management links, so a brief power outage to the switch while I rerouted the cable would be completely unnoticed. That part was very true.

[ Also on InfoWorld.com: Read Paul Venezia's instant classic, "Nine traits of the veteran Unix admin." | Then, if you dare, join the debate about rebooting Unix-based systems. ]

However, when the switch was plugged back in, the power supply promptly failed and rendered the switch DOA. Luckily, it was a "cheap" 24-port 10/100 switch that could be replaced by a trip to Staples, but it was after 10 p.m. during a late-night maintenance window -- it would have to wait until the next day.

Naturally, this was a problem because some of those management links were needed to complete the maintenance work. I wound up borrowing a switch from a conference room to maintain order for the remainder of the evening.

There are two ways to look at that tale. The first is that I should have left well enough alone, and all would have been fine. The second is that I hastened the failure of that power supply -- and caused it to fail while we were there rather than in the future, at what would have probably been a less fortunate time. Nonetheless, I'd probably leave the power cord alone if I had to do it all over again.

Another example of unintended consequences: the firmware and/or software updates you make on a whim. In this situation, you're doing something completely different with a switch, router, or server and note there are one or two updates available for some software or the switch firmware. Thus, you decide to pop that new version since you're right there anyway. About 80 percent of the time, this is a nonissue and things go as planned (though it wasn't actually planned). The last 20 percent usually involves someone's hair catching fire or a meteor strike or something.

But this phenomenon isn't limited to production systems. In the buildout phase of an important project, application, or whatever, all the necessary pieces are in place, the proof-of-concept is running happily, and everything looks great for a rollout into production. Until you decide to go through it with a fine-tooth comb to make sure that even though everything seems perfect, it really is perfect.

During that somewhat OCD search, you see a few parameters that don't appear to be set right -- as innocuous as a typo in the secondary NTP server field or the lack of a description entry on some configuration parameter. None of it's really a problem per se, but still isn't quite correct and dammit, it should be right.

So you happily fix the NTP address or add a brief description in the text field and save the configuration. Then you realize you hadn't backed up the previous configuration, and another admin had been toying with some other settings for some reason. By saving the config, you've toasted the known-good configuration for the whole setup. But that secondary NTP server field just had to be fixed, didn't it?

Most of this is just boneheadedness -- the IT equivalent of someone seeing a piece of paper stuck to the wall, pulling on it, and ripping a huge hole in the wallpaper that takes some plaster with it. Or it's the thread hanging out of the sweater that causes the whole piece to return to the ball of yarn from whence it came.

Sometimes, many times, the big items are easy. But the little details done with no malice -- yet also no forethought -- can be just killer.

This story, "Admin tip: Leave well enough alone, dummy," was originally published at InfoWorld.com. Read more of Paul Venezia's The Deep End blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

Mobile Security Insider: iOS vs. Android vs. BlackBerry vs. Windows Phone
Recommended
Join the discussion
Be the first to comment on this article. Our Commenting Policies