To make a very long story short, an Exchange server experienced catastrophic hardware failure, scrambled the mailstore, and was rendered completely inoperable. An hour on the phone with HP resulted in a promise that the warrantee replacement parts would arrive within 24-48 hours, no guarantee. However, an "uplift" service was available for the low, low price of $2,500 to expedite parts for next day. The whole server cost less than $2,500. Instead, I found a refurbished 1U server locally for around $1k with twice the resources of the original. We rebuilt the Exchange server, only to find the mailstore corruption. 11 hours later, the mailstore repair tools had finished and the mailstore was finally remounted... with errors.
Some (not all) users could access their mailbox, but were not receiving new mail. The errors in the event log elicited only two hits on Google, and none on Microsoft's site. The two hits were related to the same forum post with no resolution -- useless. Microsoft's own tools for locating resources pertaining to this error went nowhere. The actual error is from source MSExchangeIS Mailbox, ID 1025, error data "An error occurred on database "First Storage Group/Mailbox Store (MAIL). Function name or description of the problem: SLINK::EcUpdate Error: 0xfffffa84". Since I could find references to part of this error, but nothing about it in it's entirety, I had to call Microsoft. That's where the fun started.
After calling in and navigating half a dozen voice prompts, I spoke with a customer service rep who kindly took my credit card information for a $245 charge, gave me a case number, and told me that the tech queue wait time would be 5 minutes or so. I waited on hold for 62 minutes before I got a tech with a very heavy accent. He asked for the case number. I was surprised that he didn't have it already. How could he not have it? I was on the same call. I looked around and couldn't find the paper I'd jotted it down on over an hour earlier. I asked if he could look it up, and he brusquely transferred me back to the CSR pool. While he did this, I found the number. I checked it with the new CSR representative, and discovered that I'd been given the wrong number to begin with. Then, I was told that I'd be put back in the queue, with a hold time of only 45 minutes. I spoke with a supervisor who was very sorry that this was the case, but there was nothing he could do. Back I went into the land of Nod.
I was two hours and $245 into a service call to Microsoft to find more information on their own error code that isn't referenced anywhere on their site that I could find. I finally got a tech for the second time, who asked for the case ID, interrupted me twice while I was trying to give it to her, then abruptly hung up on me. Livid doesn't begin to describe my state of mind. I've worked with Microsoft enterprise products for a decade, and I've never had to call them before. That's truly a blessing.
What a racket. I'd love to get $245 to abuse my customers like this. I'd be a billionaire.