From 2001 to 2003, I worked as the local IT admin and jack-of-all-trades for a branch of a collection services company in California. It was fun in that I got to do a lot of stuff that a larger organization would have compartmentalized, but it was also difficult in that I had no one to back me up when I needed to make an argument. The boss happened to not only behave like the "pointy-haired boss" in Dilbert, but he actually had pointy hair -- a coincidence that often manifested itself when he showed both his lack of IT knowledge and his poor management decisions.
When I first took the job, the network consisted of a hodgepodge of equipment run on cat3 and coax. By "hodgepodge," I mean we had ancient reel-to-reel tape drives and an old mainframe running data "platters" that I had never heard of until then, and the user environment was old orange-screen terminals.
[ Want to cash in on your IT experiences? Send your story to firstname.lastname@example.org. If we publish it, we'll send you a $50 American Express gift card. | Show off your geek quotient: Get your InfoWorld shirts, cups, and more schwag. ]
When the day came to upgrade to a new network, I got tasked with the project and spent a couple of weekends pulling Cat5 through suspended ceilings and conduits, and I got to install a textbook server to replace the old mainframe. I did much of the grunt work myself and planned very carefully, and I was thrilled to see I had quite a bit of money left over in my budget. I went through the appropriate channels and bought an extra server to use as a mirrored server for fault tolerance.
In the course of a few months, the mirrored server setup prevented costly interruptions to service several times -- and I was pleased with myself for my foresight.
Then I went on vacation.
While I was on vacation, the IT guys from the main office flew out to take my "extra" server to their office to reimage and use as an additional Exchange server. My manager accepted their explanation that I was wasting company resources with this "extra server" and let them abscond with a very much needed piece of equipment.
That Saturday my pager went off. I called in to find out that the network was down and 400 collection agents were idling away their time because of the outage. I apologized to my wife and drive the 450 miles back to the office to find out that my backup server was gone. After an hour, the lone server was back up and running from a backup tape that was several days old. My boss demanded to know why the server failed -- and I demanded to know where my backup server was.
He admitted that he let the HQ guys take it, then I reminded him that he gave away a $30,000 server that our office had been charged for and that it was not an "extra" server but part of our fault-tolerance system.
To his point of view, my mirrored server was an "extra" because he never saw the system failing. In retrospect, I suppose I should have put just one server online at the start to illustrate to him why a backup was important. Seeing the reasons on paper sometimes aren't as effective as seeing the reason in action.
We never got the server back, so what ended up happening was that an additional server was bought for around $10,000. Then the company went bankrupt later that year, ending my worries about the whole scenario when I helped lock the doors for the last time on Oct. 30, 2003.
Stupid user tricks 3: IT admin follies
IT heroes toil away unsung in miserable conditions -- unsung, that is, until they make a colossally stupid mistake
More stupider user tricks: IT horror stories redux
Idiot-proof your enterprise with these 10 hard-luck lessons of boneheaded IT miscues
Stupid QA tricks: Colossal testing oversights
The trick to nipping IT miscues is testing, testing, testing, as these hard-luck lessons in boneheaded quality assurance attest
Programming IQ test
Are you a script kiddie or a hacker hero? Take our quiz to find out
Even dirtier IT jobs: The muck stops here
More dirty tech deeds, done dirt cheap
The 7 dirtiest jobs in IT
Somebody's got to do them -- and hopefully that somebody isn't