Don't expect big-time results from big data

Big data is here, but barriers suggest success will be modest; for many companies, it may prove unsettling and disappointing

1 2 Page 2
Page 2 of 2

The new staff and retraining, and (if you go this way) cloud storage, server capacity, and network pipes or flash drives have significant collective budget impact. That means CFO approval will be required.

Capital investment (if you go on-premises) means the dreaded capex committee will have to be involved. Running a proposal through the capex committee is, in most companies, a lot like running across hot coals -- it takes confidence and a lot of calluses. Even then, you're likely to get burned.

With big data, the CFO and very likely the capex committee are wallets -- the money comes out of operating and capital budgets. This will be a much bigger hurdle than big data's relatively minor customer/consumer disconnect.

Disruption: In most companies, big data will be useful only if it's disruptive.

Not technically -- while it will take a significant investment in technology, big data systems will mostly be walled off, with data flowing in but not out. But culturally, big data will either be disruptive or worse than worthless.

Big data won't be disruptive in those relatively scarce, healthy businesses that have a culture of honest inquiry in which the evidence leads and the company follows it. In these sadly rare organizations, executives and managers are hungry for information because it will help them make better decisions. For them, there's no cultural disruption and big data can be a great fit.

Most businesses are far away from this -- their executives and managers either make decisions by "trusting their guts," or they make decisions by trusting their guts, then solve for the answer they've already decided is right. For these decision makers, in other words, evidence is used as ammunition, not for illumination.

The ones who trust their guts will have no use for big data -- because what part of "they trust their guts" wasn't clear?

The ammunition gatherers will be worse; instead of just arguing like they do right now, they'll present convincing charts and graphs that "prove" they're right about whatever it is. But big data's contribution has been negative, because their actual accuracy hasn't changed -- only their persuasiveness.

Assessing big data's long-term viability

From the perspective of immediate industry acceptance, the big barrier to big data is that the wallets -- the CFO and capex committee -- are likely to see it as big cost, without any clear value in return.

Should big data clear this hurdle, it's good to go. Because while it will (or at least should) be highly disruptive to the average company, the need for this disruption will mostly go unnoticed: Few executives will change how they make decisions, just because they can.

My prediction: Big data will be here for the long term, but its success will be modest rather than spectacular. That's the technology, as measured by sales and implementations.

Value is a different matter. For most companies that head down the big data path, the value will never be entirely clear, and the whole outcome will be vaguely but unsettlingly disappointing.

Those will be the lucky ones. For some it will be much worse, but that story will have to wait until next week.

This story, "Don't expect big-time results from big data," was originally published at Read more of Bob Lewis' Advice Line blog on For the latest business technology news, follow on Twitter.

Copyright © 2013 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
How to choose a low-code development platform