By this time my supervisor had arrived. He explained that he'd negotiated the software contract to be paid on a monthly basis that also included part of the original purchase amount along with the yearly maintenance fee. He verified the bill had been paid each month and began to make phone calls to his contacts at the vendor. Of course, as luck would have it, the vendor was in another time zone, so another half hour passed before we got a return call.
The vendor said the software wasn't working because we were using old keys and the company had emailed new keys to us three months before. The rep even said the team had carbon-copied me as well since they had my contact information from ongoing communications we'd had about setting up and using the software.
Not wanting to outright deny ever having received the email, I went to my PC and checked my saved messages. I specifically create a directory for each vendor's email and save all communications from them in that directory. I had no emails from this vendor the entire week that the rep claimed to have sent the keys.
I reported this to my supervisor, who was frantically searching his emails, also to no avail. In the meantime, the vendor reissued the keys, we applied them, and payroll was back up and running. But the missing emails were still a problem.
My supervisor and I looked through our emails again -- nothing. Studying the calendar more closely, we discovered that another change had been implemented the week prior to when the software vendor said we'd been sent the new keys. We'd switched to a new email scanning and filtering service in the cloud to relieve some of the burden on the mail server. The service had required close monitoring at first, training it on what to permit to let pass to the mail server.
We traced what had happened. Initially, this software vendor had been flagged and blocked, which was discovered about a week into using the new email service. The settings were changed to allow future emails from the vendor to go through, and we verified that at that point normal weekly updates and communications came in without a hitch. Since the mail was flowing, no one had dug any further back; otherwise, they would have discovered the email with the keys was still blocked.
It's hard to cover all the bases even though we try. We can make lists of details to address and check them twice. We can plead with users to report errors and messages to IT immediately. But even though it's maddening, in the end it doesn't hurt to get a reminder now and then to be more diligent on checking and following through with tech changes.
Do you have a tech story to share? Send it to firstname.lastname@example.org. If we publish it, you'll receive a $50 American Express gift cheque.
This story, "Follow the money: IT unravels payroll puzzle," was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.