The big data trend has met with serious pushback in many IT departments. The testy version goes something like this: "Enough already. I have plenty of SQL data stores with lots of data, and I have all the business intelligence tools my business needs. So exactly what are you taking about?"
Sure, by now everyone has heard of Hadoop and the insight businesses have obtained from mining semi-structured data, but in many cases that value proposition is lost on conventional IT folks. I believe this intransigence is part of the reason an increasing portion of the IT spend is slipping from the hands of the CIO to the CMO and line-of-business managers.
Last week I interviewed Rod Smith, vice president of emerging technologies for IBM, who has been deeply involved in IBM's big data practice and product development for four years. I asked him why the resistance seems so entrenched in some cases.
As an example, Smith cited a big data Twitter engagement, where a "very large customer in the financial area" with 150,000 business partners wanted to monitor Twitter for information about potential new partners. If, for instance, a new restaurant tweeted that it got an award, it might be targeted with a congratulatory message as a way to begin a new customer relationship involving a loan or credit card.
The reaction of IT was less than enthusiastic, according to Smith, who observed resistance to seeing the Twitter app as a real application with real value:
It's hard to convince the IT guys they're going to do something where, outside, you can get to the Twitter data or the Facebook data. And it's fleeting. It might be useful for a particular campaign you're going to do or the launch of a product, but then it might be disposable at that point. So that makes it really tough for the IT guys to say, yeah, we're going to spend with you here, because we know that we'll build it and you'll use it will have that value over and over again. They can't touch it or feel it the same way anymore.
No surprise, then, that many of IBM's big data engagements begin with the business side. "A line of business or the CMO doesn't want to wait 18 months to have their application on some backlog, while the IT guy tries to figure out whether there's real business value there."