At the time of this story, I worked at a small IT service shop that supported a variety of customers. There were only a few techs and business was good, so we stayed fairly busy.
Our boss had never worked IT before but had decided to turn his hobby of building PC clones on his dining room table into a small business. His focus was always on cost cutting. He assigned jobs based on our primary skill sets, but didn't see any need for documentation, best practices, or ongoing training. We implemented such measures in the shop when and where we could, by mutual agreement among the technicians.
[ The tech inferno is not buried deep within the earth -- it's just down the hall. Do the nine circles of IT hell sound familiar? | Follow Off the Record on Twitter for tech's war stories, career takes, and off-the-wall news. ]
The boss's attitude changed all at once, during one of my vacations.
Our company had been asked to handle an emergency installation of software on a network server over a weekend, late at night. This was one of our contract service clients -- essentially someone who kept us on retainer for such things -- so we were obligated to say yes. The problem was that I was the only one familiar with the customer's network and wasn't available. To cover all bases, the boss scheduled all the other techs to take care of it.
The clients were computer illiterate for the most part and had tried to install some third-party software on their Novell server by themselves. However, the software wouldn't run properly after being installed (most likely a permission issue), and the client called our shop to ask for help.
Our guys arrived on-site, checked in with the suits who had tried to do the initial install, set up at a terminal, and got started. Six hours later, they had tried without success to install and get the software working and had tried calling me several times, but my chosen vacation spot had no cell service (which is true even to this day).
They kept at it, installing the offending software only to watch it mysteriously uninstall itself while they were still copying files to the server. Some time after the 12-hour mark, they went to see if the client's company executives were still in the office and to let them know they were still having problems. They found the suits sitting at a terminal, cursing.
What had happened was that the client's suits were trying to uninstall their failed attempt at earlier installations, which kept magically coming back while our IT guys tried to install it from another terminal. So the suits kept deleting while our IT guys kept re-installing -- over and over again.
For over 12 hours, the two teams had fought each other from different locations, one installing software and the other deleting it, because the corporate guys thought they were helping and none of the IT guys had thought to disable logins.
The solution we decided on in-house was to do more to train one another in the basics of each of our specialties and to offer a summary document with information about each of our clients. That's when our boss decided that adding a classroom and offering training classes would be a great idea.
Do you have a tech story about supporting users? Send it to offtherecord@infoworld.com, and if we publish it, you'll receive a $50 American Express gift cheque.
This story, "The software install that would not end," was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.