For all the promise of big data, the fundamental challenge with collecting massive volumes of data from different sources is finding new business uses for it, according to several IT managers at Computerworld's BI & Analytics Perspectives event held in Phoenix this week.
Technology vendors and industry analysts tout the enormous business benefits that enterprises can gain from mashing up traditional structured data with unstructured data from the cloud, mobile devices, social media channels and other sources. But business executives have little idea of how to take advantage of big data or how to articulate their requirements to IT, according to several executives at the show.
[ Also on InfoWorld: 4 questions to ask before starting a big data initiative. | Harness the power of Hadoop with InfoWorld's 7 top tools for taming big data. | Learn how to stay on top of the ever-growing amount of enterprise data with InfoWorld's Information Overload blog and Data Explosion newsletter. ]
Business leaders often "don't know what they don't know," said one frustrated IT manager, and therefore they are incapable of explaining to IT shops what to do with all this data that's being accumulated.
Over the past couple of years, private investors and venture capital firms have poured hundreds of millions of dollars into startups developing new technologies for collecting, storing, organizing and analyzing petabyte-scale volumes of structured and unstructured data. The tools have made it easier than ever for companies to pull in data from web logs, clickstreams, social media, video and audio files, machine sensors and micro-blogging sites such as Twitter.
The real challenge is not the technology, but finding business value out of all the data that can be collected, said Reid Nuttall, CIO of OGE Energy, an Oklahoma City-based energy company. OGE owns nine power plants and delivers power to more than 758,000 customers in a 30,000-square-mile area. The company recently installed smart meters across its customer base that provide meter readings in two-hour increments, compared to the once-a-month readings it received previously.
Nuttall is optimistic that the large volume of data generated by the smart meters can help OGE analyze and influence customer behavior and reduce peak demand over the next few years. He is looking for people within his organization who will start extracting this kind of business value from the data. "We have lots of data, and we are figuring out what to do with it," he said.
Nuttall set up an information "factory" and a business analyst competency center inside the organization to help spur creative uses of the data at OGE's disposal. OGE is investing in business intelligence tools and new data visualization and presentation capabilities to get analysts to think about and use new data, in different ways. "Big data is forcing IT and business intelligence [teams] together" to find ways of exploring new data together, he said.
Payroll processor ADP is taking the same approach. The company has set up an Innovation Lab to manage how it stores, processes and analyzes extremely large data sets.
The idea is to create an environment where subject matter experts from different industries and backgrounds can work together to tackle big data analytics, Roberto Masiero, vice president of ADP's Innovation Labs, said in a keynote address.
In a sense what is happening is reminiscent of the situation when enterprises first started using online analytical processing tools, said William Herridge, managing director of emerging solutions at the Tribune Company.
"When we made the transition to OLAP (online analytical processing), it was hard to get business users to get over their [existing] mindset," of using tabular data, he said. "They didn't have any idea of the value of OLAP till you started showing them," he said. IT organizations face the same challenge with big data, he said.
"We see the value in this, but getting users to understand that value and seeing it is there," is a huge challenge, especially when dealing with concepts such as unstructured data, he said. "Until business users can see some benefits, they are not going to sign on to big data projects," Herridge said.
The hardest part of using big data is trying to get business leaders and executives to sit down and define what they want out of the huge amount of unstructured and semi-structured data that is available to enterprises, said Vivek Ratna, a partner with Digital Learning Solutions in Irving, Texas.
"The fault is ours because IT has not articulated as well as we should have what value business can derive," from big data, Ratna said. Many IT organizations are still not collecting or using unstructured data because they are unsure of the business value and not because of technology reasons, he said.
"Unless we can define what value can be derived [from big data] or the business leaders can tell us what value they want to get out of it, we are just playing in the dark," he said.
The sentiments are consistent with those expressed by respondents in a recent survey by market research firm TheInfoPro. The survey of 255 IT professionals showed that a majority of companies had no big data plans because they didn't have a specific business case for one.
Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan or subscribe to Jaikumar's RSS feed. His email address is email@example.com.
Read more about big data in Computerworld's Big Data Topic Center.
This story, "Finding the business value in big data is a big problem" was originally published by Computerworld.