You want big data? Here's your big data

Finding meaningful sources for big data is no slam dunk, nor is dealing with executives' dreams of banking on IT's latest trend

1 2 Page 2
Page 2 of 2

That doesn't matter, because if they don't get the answers they want, they'll keep trying different analyses until they do. Along the way, they might get interested in the details of where the data came from and how you processed it to get it into Hadoop.

But if they get the answers they want, they'll be happy. And they'll stop. This is a well-understood aspect of human behavior: When people read an analysis that challenges their biases, they'll nitpick it to death, but when they read one they find agreeable, they'll ignore even the most egregious logical fallacies and errors of fact.

If you'd like help developing a similar solution, I'd love to have you as a client -- seriously. Adding in the other services I could sell you, I could retire in comfort.

Big data solution No. 2: Having big data without doing big data

There's no shortage of voices advising you to have a "big data strategy," whatever that means. If it means embarking on solution No. 1, call me. For real.

I strongly suspect that most companies don't have the data. But even among those that do, most lack the other prerequisites for success. To succeed with big data, companies must:

  • Be awash in data with mining potential
  • Have sophisticated statisticians and analysts on staff
  • Prefer data-driven decision-making to executives "trusting their guts"
  • Have an overall culture of honest inquiry

If your company isn't like this, solution No. 1 would probably save you a bunch of time, money, and headaches, because no matter how good your technology, the most anyone will do with big data is pretend.

But if your company has all four prerequisites, it is well worth your while to get on board with big data technology without further delay.

Never mind the solutions; here's an opportunity

"IT had better get on board" is advice for IT management. If you aren't in the management hierarchy -- if you're part of the IT staff in a company that ought to be investing in big data technology but isn't, due to an IT "leadership" team that isn't paying attention -- consider this a major career-building opportunity.

In what you laughingly call your spare time, learn everything you can about big data, down to the last painful detail, especially everything that can go wrong if a company isn't careful. Eventually, the folks you report to will find themselves in the spotlight, having to answer the question, "Why aren't we doing this already?" Even worse is the next inquiry: "How long will it take to catch up to where we should be?"

When that happens, there are worse roles to play than a white knight, ready to ride in and save the day.

This story, "You want big data? Here's your big data," was originally published at Read more of Bob Lewis' Advice Line blog on For the latest business technology news, follow on Twitter.

Copyright © 2012 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2