Working in the IT department of a big company, all too often corporate bureaucracy interferes with day-to-day operations. Offering solutions or asking for clarification are dead ends -- we are given our instructions and expected to carry them out. Our role is made clear: We are the hired help that appears when management's summoning bell rings.
Case in point: Our company had recently acquired a new division located on another continent. The division's former company had outsourced IT services to a global TLA (three-letter acronym) outsourcer as a cost-cutting measure, and part of the deal when we acquired the division was that TLA would continue providing services for 24 months while our IT department prepared to take over.
[ Want to cash in on your IT experiences? InfoWorld is looking for an amazing or amusing IT adventure, lesson learned, or war story from the trenches. Send your story to offtherecord@infoworld.com. If we publish it, we'll send you a $50 American Express gift cheque. ]
My company had never worked with TLA -- or any outsourcer -- before and The Head Honcho was determined that we'd get our money's worth.
One day a high-priority problem arose at the new division: A VPN data link was very slow and unusable, stopping productivity, and TLA wasn't having any luck solving the problem.
Because TLA was involved, The Head Honcho insisted on calling the shots. I was the unlucky one who was told to "look in" on the problem and dial in to a conference call or two. I was not allowed to change anything -- strictly "look but don't touch."
Many conference calls and e-mails later, TLA's proposed solution was to buy a new router with four times the processing power required and with every bell and whistle imaginable, at substantial cost. They claimed it was the "TLA standard" for all new router deployments. The Head Honcho thought it was too expensive, so gave me the green light to offer alternatives.
After remotely looking over the equipment and the configurations, it was apparent that the gear was out of date and underpowered. The router ran 100 percent CPU at slightly less than half the bandwidth on a VPN tunnel. This pretty much proved that TLA didn't do such a great job of monitoring the network -- there was nothing wrong with the router, it was just too slow for the job it was asked to perform and had been that way before we acquired the division.
I suggested that we ship a less expensive, proven firewall VPN solution. It would do the same thing, as demonstrated by our numerous installations around the globe, and it would shift the VPN duties to a dedicated VPN appliance and allow the router to just route. The Head Honcho agreed.
Three days later, the new VPN gear had arrived. But try as they might, the TLA techs couldn't make it connect. Again, I sat in on the endless conference calls where the facts were repeated and the solutions were non-existent.
Five minutes into one such call, I asked for explicit details of what they were doing. I had assumed that the senior router personnel who managed the VPN equipment knew how to set up a point-to-point tunnel. Was I ever wrong! I made a few suggestions, very basic stuff, and it was like I'd illuminated their world.
A couple of days later, TLA finally managed to achieve a VPN tunnel -- but nothing flowed across it. CrackBerrys buzzed for hours into each evening. Finally, we scheduled yet another conference call, but this one was flagged as High Importance so that we could get to the bottom of the issue. The Head Honcho was even scheduled to attend.
Everyone from my side made the call. Only one guy bothered to dial in from TLA -- and he could barely spell VPN. Everyone else from TLA blew it off.
The Head Honcho was irritated, but inexplicably determined to get our money's worth from TLA and refused to fly any of our IT staff to the site or let us do more than "look in" on what TLA was doing.
By this time, with my exasperated manager's blessing, I had prototyped the whole solution in the lab with the equipment the division was known to have on hand. It was only six clicks in a GUI to configure everything for a VPN tunnel. I sent TLA the tested configurations, and they still were not able to make it work. Both my manager and I again asked The Head Honcho if we could take control of the site. Again, denied. We were paying TLA to manage it.
E-mails kept flying back and forth about the VPN issue. TLA was still stumped and fell back to their same old plea for a new router.
This time, The Head Honcho allowed me to ship a router, not decked out like TLA requested, but more than capable of doing the job for years to come. Somehow TLA managed to implement that without any help, and with impressive speed. It worked well enough to get the division through until the contract with TLA closed.
Finally, that day arrived, and our IT department had control of the remote routers. As part of the migration, I attempted to install our preferred standard arrangement with a local firewall and VPN -- the same solution we had proposed early on and TLA had not been able to implement. It worked for me on the first try.
In the end, it was confirmed that none of the TLA techs really knew what they were doing. The talent that had built the division's network left for more lucrative contracts as soon as our company acquired the division. Since we were known to be working toward assuming ownership of all hardware and support, TLA gave us the B-team.
Maybe next time The Head Honcho will not be as stubborn to prove a point or at least let us know the reasoning. Or maybe not. There are things in corporate culture that won't change, but the point is to keep trying. And in the meantime, keep cashing the checks.
This story, "I could do my IT job if management got out of the way," was originally published at InfoWorld.com.