InfoWorld Home / Tech's Bottom Line / Can you trust Gartner's 'Magic Quadrant' or other...
October 22, 2009

Can you trust Gartner's 'Magic Quadrant' or other analysts' reports?

A new lawsuit alleges their influential conclusions are biased to those who pay. It may be time for financial-industry-style disclosure rules

| Print | 9 comments|
144 Recommendations

Talk about David versus Goliath. A tiny Silicon Valley software vendor is taking on mighty Gartner, one of the technology industry's largest and most influential market research and consulting companies. The battle is playing out in a San Jose federal courtroom, where ZL Technologies is asking for $132 million in damages (plus even more in a punitive judgment), saying the research outfit damaged its prospects by ranking it in the bottom segment of its closely watched Magic Quadrant report. The MQ divides technology providers into different classes, with the bottom segment essentially forming a "do not buy" recommendation.

Sure, vendors complain all the time that Gartner or IDC or Forrester and their cousins do a crummy job and unfairly diss their products. Sometimes they have a point; often they don't. There are, after all, a lot of crummy products out there. But the ZL Tech lawsuit raises issues that have bubbled under the surface of the technology industry for years. Most significant, I think, is the common practice of selling research and consulting services to the very companies whose products it evaluates. (Disclosure: IDG, which owns InfoWorld, also owns IDC, a major technology research company that competes with Gartner.)

[ IT faces another information disconnect: Its big vendors' agendas often don't match IT's, as InfoWorld's Eric Knorr explains. | Keep up on the day's tech news headlines with InfoWorld's Today's Headlines: Wrap Up newsletter. ]

"It's naïve to think that it doesn't matter [to a research firm] if a vendor delivers a huge chunk of revenue," says ZL Tech CEO Kon Leong. What's more, he says, vendors with big marketing budgets are unfairly favored by Gartner, and that stifles innovation in the industry. "Small innovative guys are running uphill against the Gartner wind," he told me.

Update: A judge later rejected the lawsuit against Gartner, ending ZL's efforts. But the issues raised remain real.

Related Content...

White Paper

D2D Virtual Tape Library Replication Primer

This whitepaper explains the terminology and concepts behind Data Replication technologies and establishes some sizing rules through worked examples. Learn the new paradigm in disaster tolerance—protect data anywhere.

Download now »

White Paper

An Alternative to Virtualization for Datacenter Cost Savings

Server virtualization is a popular option for dealing with mounting datacenter costs. Another equally promising approach is the use of an Application Delivery Controller. Citrix NetScaler provides a low-cost way for organizations to reduce their server count and accrue cost savings from a reduction in space, cooling, power and personnel.

Download now »

White Paper

Why Your Firewall, VPN, and IEEE 802.11i Aren't Enough to Protect Your Network

The emergence of WLANs has created a new breed of security threats to enterprise networks.

Included in HP ProCurve WLAN solutions is security technology that alleviates threats from WLANs through:
* Monitoring wireless activity inside and out of the enterprise
* Classifying WLAN transmissions into harmful and harmless
* Preventing transmissions that pose a security threat to the enterprise network
* Locating participating devices for physical remediation

Download now »

White Paper

Bringing the Edge to the Data Center

Effectively address data protection challenges, implementing solutions that help store and protect business–critical data while cutting costs and improving efficiency and reliability.

Download now »
9 of 9 comments
Sign In or Register to comment
johnradams 22-Oct-09 9:43am
I ran into Gartner's game 15 years ago. We were an up-and-coming leader in a market niche, and were generally considered as one the most significant players (if not the outright leader). Gartner wouldn't talk to us unless we first subscribed to their "services" for several tens of thousands. We refused because they seemed like such scumbags. Through hard work, better products, better pricing, and better services we eventually won the market and our company was acquired by one of the large OEMs. By the way, Gartner ripped-off the whole "quadrant" model from the Harvard Business School.
Gray_Hair 22-Oct-09 9:52am
"Industry Analyst" firms in Information Technology are all to often among the worst of any industry, particularly if you are looking for meaningful data on vendors. Their bias for their customers is long and storied, and their "reports" depend so heavily on undisclosed "methods" as to be practically useless. They keep labor rates down by allowing a turnover rate among analysts that prevents their data grinders from ever getting really good at what they do. Those that do develop some tenure are usually those best at securing revenue rather than those savvy enough technically to ask meaningful or revealing questions. I have yet to read a report from this group that was worth what it cost, and way too many of them strike me as little better than Demand Media could have produced.
joncz 22-Oct-09 9:58am
For those with any experience in the IT industry, Gartner's MQ designation can be taken as a warning. I have never seen it as validation of anything other than a robust marketing department. Sadly, I ran across a vendor the other days whose homepage led with the MQ designation. That's their best product feature?!?
Regaug 22-Oct-09 2:14pm
Is it still "free speech" if you're getting paid to say it?
cag4 22-Oct-09 5:14pm
I remember having *exactly* the same experience of johnradams back in about 2000-2001, when I was running an internet publishing company in NYC. They wouldn't consider including us in their reports unless we signed up for their many thousand dollar "services" first. We ran from the obvious scam... glad to hear someone is calling them on the racket.
Bill Snyder 22-Oct-09 6:00pm
Wow. Some of these comments seem a bit harsh. But I certainly understand the anger of someone who invests blood,sweat and cash into an idea and have it trashed for no apparent reason.
msmith20001 23-Oct-09 8:04am
1 reply
Go to ZL website and read their "Press and News" section. It is obvious that ZL's sales and marketing has been sleeping for the past several years. So just because they subscribed to Gartner they should be now placed in a better quadrant? That funny looking CEO of ZL has no idea how business is run. Focus on YOUR sales and marketing you idiot and stop being a "sissy" and "cry-baby-bunty".
johnradams 23-Oct-09 9:31am
I can't speak with any knowledge about ZL's marketing, or lack thereof. But I can speak with certainty that Gartner only includes a vendor if they pay to be included. And, by the way, the more you pay (oops -- the more services you decide to benefit from!), then the more coverage you get. From your comments, I get the impression that ZL wasn't listed by Gartner until they "subscribed" -- is that the case? If so, isn't the the crux of the problem: you have to pay Gartner a toll to be listed? It would be nice if something like Consumer Reports existed in this space. Takes no advertising, takes no money. Anyone care to start "Business Reports"?
sfox618 23-Oct-09 11:24am
I have personally been involved in 5 Silicon Valley start-ups. 2 IPO's and 3 sell-offs (First IPO in 1994 - last in 2001). One of the big events in both of the IPO companies was moving into the upper right corner of the MQ. In my experience in all cases moving into the upper right quadrant happened right after we signed a consulting agreement with them. This is not unique; all of these firms work this way. I'm always surprised when people don't understand this and are surprised that the analysis they are reading is biased.

Sign up to receive InfoWorld Resource Alerts

Subscribe to the Today's Headlines: First Look Newsletter

Find out what will be news for the day, with our first-thing-in-the-morning briefing.

©1994-2009 Infoworld, Inc.