Censorship test

Stahl was one of the BEA officials Merrill corresponded with, and he deserves credit for not stonewalling these requests the way we’ve seen certain other companies do in this situation. He didn’t recall Merrill’s specific case, however, because he gets many similar requests. “Probably not a week goes by that we don’t have to deal with a request like this,” Stahl says. “I can understand the frustration that those making these requests feel, but we invest many millions of dollars into the product. So we don’t want anyone publishing our benchmarks unless we agree on how it’s being tested, because if it’s done wrong, it could have a devastating impact on our business.”

Stahl thinks the only answer is for companies to get together and settle on industry-standard benchmarks similar to the Transaction Processing Performance Council tests that database vendors use. “That approach lets each vendor put forth their best representation and alleviate all this problem of letting people publish anything they want,” he says. But Stahl admits that such tests don’t always reflect real-world results and are also subject to abuse by competitors who find ways to sidestep the censorship clauses. Specifically, he points to Microsoft-sponsored J2EE vs. .Net benchmarks published in November (about the same time Merrill’s BEA-less results were published) as a “sham of the highest order” and “fraudulent misrepresentation of our product” that are still to be found on Microsoft’s site in spite of having been debunked.

If this sounds vaguely familiar, it’s because Microsoft officials have had similar things to say on how J2EE competitors have misrepresented .Net performance. In spite of all these vendors having censorship clauses that are supposed to protect them from publication of biased or badly conducted performance tests, they all claim to be victimized anyway. So what good are these benchmark restrictions, other than keeping the small-fry like Merrill from collectively providing a more balanced and objective view? And, by the way, there’s also the little detail that “people publishing what they want” isn’t supposed to be a problem in this country but a right.

So the answer to our quiz is …. Sorry, it was a trick question. Not even E) really adequately covers who suffers from the fact that open discussion about a crucial technology is being thwarted by a handful of companies on an almost daily basis. Our whole society, and our future, is suffering incalculable damage. It’s time we wake up to that fact.