Here comes the big data bubble

First there was the dot-com bubble; then the housing bubble. But nothing will compare to the big data bubble

Now that it turns out our smartphones have been spying on us, it's easy to imagine the possibilities. Think big! And by big, I mean big data.

Just imagine dumping all that near real-time smartphone log data on app usage, buying habits, and physical location -- daily commutes, lunch spots, hotel room trysts -- into a gargantuan Hadoop hopper. Add a generous helping of Facebook, Twitter, Google+, and Amazon clickstreams. Process vigorously. What do you get?

[ See "Is a privacy backlash brewing?" for a more serious treatment of this topic. | "The big promise of big data" provides an overview of the big data trend. ]

Whole new businesses! There's more than enough here to finally lift the economy by its bootstraps. Lackluster consumer demand, not debt, has been the primary culprit keeping the global economy in the doldrums. Crunch on the behavior patterns of hundreds of millions of individuals long enough and you'll get the ultimate big data outcome: knowing what any consumer wants to buy at any given second.

Resistance will be futile. Big data will know what you want before you do.

Sure, they say the petabytes of data being collected about consumers are anonymized. They say they're just looking for "patterns." But do you really believe that? By definition, any service or website where you have an account can attach your behavioral data to you, personally. After all, social networks already track your surfing habits by default.

But that's a good thing. With your identity and a complete record of all the wonderful things you do with it every day, marketers will present you with the most enticing sales pitches you've ever had the pleasure of experiencing. As they say in the Internet marketing business, "It's not spam if you want it."

Who will make money from this surfeit of data? If you're an aggressive self-promoter, maybe you! It's not just about buying; it's about selling, too. Social influence scoring services such as Klout and PeerIndex identify those who, through their social media circles and activities, appear to have the power to convince others. Advertisers are willing to pay a commission for that.

Just imagine a world in which everyone is selling to everyone else. And with mobile devices as a kind of walking RFID tag, you'll be able to detect a sucker coming a mile away.

The possibilities are endless. Considering buying a big-ticket item, like a car? LED billboards will sense your smartphone's approach and, knowing your tree-hugging tendencies, flash you an ad for a Chevy Volt. (No, I didn't steal that one from Philip K. Dick; the ASICS shoe company did something similar with its Support Your Marathoner program last year.)

Sure, some wet blankets say all that data may be misused for, say, "Minority Report"-style crime prediction software (already being tested) or an excuse to jack up your car insurance premium due to your perfectly legal but rather erratic driving habits.

But come on. It's worth it. Big data is all about getting to know you, what you do, and what you want really, really well -- and serving it to you on a platter at just the right instant. Once we were told the most patriotic thing we could do was go to the mall. Thanks to big data, the mall is coming to us.

This article, "Here comes the big data bubble," originally appeared at Read more of Eric Knorr's Modernizing IT blog, and for the latest business technology news, follow InfoWorld on Twitter.