At the time of this story, I worked as a sys admin at a large company. My team managed a suite of critical tools on our customers' machines. One day, our company signed a multi-million-dollar contract with a new customer, a very large corporation. Our team was understaffed already, so our company's management created and trained another team -- aka the B Team -- to maintain this suite of critical tools for the new customer.
At first, the arrangement seemed great. But the B Team proved to be so inept that management was soon calling on us for emergency interventions and serious mess-cleaning.
[ Want to find out when the latest tale of IT terror hits? Get a new tech tale in your inbox every week in InfoWorld's Off the Record newsletter | Follow InfoWorld Off the Record on Twitter. ]
One Sunday around 7 p.m., I was the primary staffer on call, and my cell phone started ringing. It was my manager, asking me to please do a favor for the B Team, which had a problem it couldn't solve.
A few minutes later the B Team called and explained the problem: The team had restarted a Unix server and couldn't bring the tool back up. The B Team techs had been trying to bring the tool back up for nine hours, and the maintenance window would be over in a few hours.
I didn't even have access to the servers, so I tried to diagnose the problem over the phone, dictating commands and having them read the output to me. After a few minutes it became clear that they didn't know anything about the tool -- it turned out they were trying to start the wrong server component.
The conversation continued, and they kept bringing new people in to the conference call, "just in case." They spent almost an hour trying to find a DB2 admin, although I told them that the tool didn't work with DB2. It took me three hours to get them to write the proper commands and get the thing restarted. It was all downhill from there.
One day at about 8:20 a.m., I was on my way to work. Again, I was primary on-call, and my cell phone started ringing. It was somebody from the B Team. She told me that the upgrade had already started and that I was due to begin my task in 40 minutes.
This was outrageous -- I hadn't been informed of this upgrade. Company regulations stated that I must approve, with several days' notice, any upgrade or operation that I will carry out or take part in. I guessed the B Team wanted me to do their job because they suddenly realized that they didn't know how to handle it.