July 07, 2009

Oh, were you using that server?

In this IT tale, the boss gives away an "extra" server that's part of the fault-tolerance system

He admitted that he let the HQ guys take it, then I reminded him that he gave away a $30,000 server that our office had been charged for and that it was not an "extra" server but part of our fault-tolerance system.

To his point of view, my mirrored server was an "extra" because he never saw the system failing. In retrospect, I suppose I should have put just one server online at the start to illustrate to him why a backup was important. Seeing the reasons on paper sometimes aren't as effective as seeing the reason in action.

We never got the server back, so what ended up happening was that an additional server was bought for around $10,000. Then the company went bankrupt later that year, ending my worries about the whole scenario when I helped lock the doors for the last time on Oct. 30, 2003.

Related articles
Stupid user tricks 3: IT admin follies

IT heroes toil away unsung in miserable conditions -- unsung, that is, until they make a colossally stupid mistake
More stupider user tricks: IT horror stories redux

Idiot-proof your enterprise with these 10 hard-luck lessons of boneheaded IT miscues
Stupid QA tricks: Colossal testing oversights

The trick to nipping IT miscues is testing, testing, testing, as these hard-luck lessons in boneheaded quality assurance attest
Programming IQ test

Are you a script kiddie or a hacker hero? Take our quiz to find out
Even dirtier IT jobs: The muck stops here

More dirty tech deeds, done dirt cheap
The 7 dirtiest jobs in IT

Somebody's got to do them -- and hopefully that somebody isn't

White Paper

D2D Virtual Tape Library Replication Primer

This whitepaper explains the terminology and concepts behind Data Replication technologies and establishes some sizing rules through worked examples. Learn the new paradigm in disaster tolerance—protect data anywhere.

Download now »

White Paper

An Alternative to Virtualization for Datacenter Cost Savings

Server virtualization is a popular option for dealing with mounting datacenter costs. Another equally promising approach is the use of an Application Delivery Controller. Citrix NetScaler provides a low-cost way for organizations to reduce their server count and accrue cost savings from a reduction in space, cooling, power and personnel.

Download now »

White Paper

Why Your Firewall, VPN, and IEEE 802.11i Aren't Enough to Protect Your Network

The emergence of WLANs has created a new breed of security threats to enterprise networks.

Included in HP ProCurve WLAN solutions is security technology that alleviates threats from WLANs through:
* Monitoring wireless activity inside and out of the enterprise
* Classifying WLAN transmissions into harmful and harmless
* Preventing transmissions that pose a security threat to the enterprise network
* Locating participating devices for physical remediation

Download now »

White Paper

Bringing the Edge to the Data Center

Effectively address data protection challenges, implementing solutions that help store and protect business–critical data while cutting costs and improving efficiency and reliability.

Download now »
zanzan42 7-Jul-09 9:12am
1 reply
It's the classic issue of management's view of IT: When things don't work, you get yelled at for not doing your job, and when things are working, they wonder what they're paying you for. On the subject of going out of business, I was lucky enough that my very first real job was for a company that eventually went out of business a year or two later. In hindsight, it was valuable experience which allowed me to subsequently recognize management stupidity early and often.
JTB2468 9-Jul-09 7:35am
7-Jul-09 10:12am It's the classic issue of management's view of IT: When things don't work, you get yelled at for not doing your job, and when things are working, they wonder what they're paying you for.

+1

The main reason I want to switch to a job working for a strictly IT company (setting up servers, networks and whatnot for other companies) is because of dealing with non-IT managers. It drives me insane! I'm the only IT person here, so I'll come up with an idea and the bureaucrats will use some odd form of logic to conclude some off the wall ridiculous reason that I'm doing it wrong, even though they have a hard time performing simple end user tasks.
Accounting IT Guy 7-Jul-09 9:50am
1 reply
Who the heck just goes and takes a production server offline and reimages it without even consulting the local admin who maintains the system??
JTB2468 9-Jul-09 7:29am
Sounds exactly like something my boss would do.
BigRonG 7-Jul-09 5:58pm
1 reply
It's amazing the number of incompetant goofs that are considered managers in this society. Since about 10-12 years of my 30 plus year career were spent as a consultant, I got to see it quite often. Several companies that I worked for went belly-up. It was hard to tell at times whether it was the previous screw-ups or the current ones that killed the company. Some highlights - throwing away a bleeding edge software (homegrown) suite because the new manager had never worked in that language; trying to re-write the application serving 200 plus retail stores in a new language (Visual Basic) that was not robust enough at that time (lacked critical functions) to handle the job because the new CIO had an in with a firm in the town he came from; trying to rewrite the base application serving a car financing company when the commercial app that had extensive mods worked fine spending hundreds of thousands of dollars that the company desperately needed to stay alive (it died). Only a few of many - we would be better served to create a certification program for managers and require the letters after the name.
gunner@gulftel.com 8-Jul-09 6:49am
I would argue that certifications are certainly no true indication of ability - or knowledge. I actually had customer come in once who was looking for a new video card. When I asked her why, she said her printer driver told her she had to run at a higher resolution and her card couldn't do higher than 8-bit color. I eventually discovered from her that she'd recently reloaded XP. Obviously she hadn't loaded any of the drivers yet. I told her to download and install her drivers and then was forced to explain where to get them, how to install and verify them, etc. Her next sentence was "are you hiring? I have my Microsoft Certification and everything." I deserve an Oscar for not laughing out loud in her face. All I could think of was "certified for WHAT?!" We had hired an A+ tech at a prior company and he's the first person I've EVER seen get fired on the same day they're hired. After spending all day trying to put parts together to build a system he determined the motherboard was bad because it wouldn't boot. We told him to RMA it to the vendor and get another. Luckily another tech decided to check it out, because in the box was still the P4 CPU mounted to the board (a $300 processor at the time). This same tech successfully booted the system and we discovered the "new hiree" had been using a bad boot floppy - but never tested it. Oh, and for more icing on the cake: I spent 12 years in the military and I can confirm that college education does NOT equate to better management... The best leaders I ever saw never had any college...
willicueva 8-Jul-09 3:16am
What is most important in this story seems to be neglected, namely that the IT Guys from the Head Office were the ones that took the server, without checking to see what it was there for. What does that say about the HQ "IT Guys"???
Accounting IT Guy 8-Jul-09 9:02am
It says that they were arrogant and ignorant. They think their needs outweigh the needs of a smaller office and that they dont have to follow procedure. Which will always = FAIL.
jgaskell 8-Jul-09 11:24pm
While the HQ IT guys and the PHB obviously did the wrong thing, I'd be interested to know why the main server fell over so often that a second redundant server was required. Maybe some of that money left over in the budget would have been better spent buying a more reliable main server.
glennho 9-Jul-09 11:56am
1 reply
An IT engineer once told me, "I have no server level control over stupid." There is a huge difference between knowledge and good judgment, and even an overabundance of the former will not compensate for a deficit in the latter. If tests and certificates were guarantees of good judgment, auto accidents would involve only unlicensed drivers and "acts of God". You can't get around somebody else's poor judgment - that's why cars have airbags; however, if we carefully share information up-stream, we Can improve the knowledgebase of mgmt, and hopefully gain some respect and job security (if such a thing exists) as well. Smooth-running IT cannot be invisible and mysterious. Especially when you're THE IT Guy, doing it all, you gotta advertise and self-promote. When you need to spend, don't just say, "We need this," but say, "This is the situation; A - we can do nothing, and risk X; B - we can spend this, and risk Y; C - we can spend This, and be covered. I recommend C." Now mgmt knows not only the need, but the purpose, the risks, the fact that You Know Things that they never think about, AND that you think things through instead of shooting from the hip. Even if they don't let you have your first choice (after you made it So Obvious), your cred and your visibility have improved, and you can righteously say, "I told you so, neener neener neener," when it's time to put your stuff in a box and collect your final paycheck. But before that happens, get Visible and STAY Visible -- let them know along the way about bullets dodged ("Server A crashed today, but B picked right up, with no down time - I'm sure glad we were prepared for that!") Then, down the road, when they're looking for ways to cut costs, they're more likely to acknowledge your value -- they may keep you in your seat instead of hiring the boss's teen-aged kid to come in on weekends to do the server stuff. It takes a careful balance of brevity and clarity, with just enough technobabble to give it weight, and must always demonstrate your Deep, Deep Committment To Doing What's Best For The Organization. This practice may not have averted the situation that started this discussion - it sounds like Bad Judgment was a criteria for promotion at that place. I am convinced, however, that if IT tries to be invisible and mysterious, sooner or later, the only mystery will be "Hey, what happened to that guy that used to do the server stuff?"
zanzan42 11-Jul-09 8:59pm
While you are completely correct, your story didn't include the ending, which goes something like this: Even after telling management about options A, B, and C, and the costs and consequences of each, they chose A because it was cheapest. Then when downtime inevitably happened, you still got held responsible for the failure. You never get to say "I told you so" because management doesn't care. If they did, they would have chosen C. You are screwed no matter what you do, even if you document everything. Do you really think that the management level is going to tell their management that *they* screwed up by not following your recommendations? Never happens.

Sign up to receive InfoWorld Resource Alerts

Subscribe to the Off the Record Newsletter

The one-stop resource center for IT professionals.

©1994-2009 Infoworld, Inc.