What we call "big data" today will one day simply be called "data." Big data is not a fad, but rather a new phase in the evolution of data management. It's a way to describe the exponential data growth we all struggle to accommodate -- and a way to underscore new opportunities to extract meaning from that data using emerging technologies.
This blog will be your front-row seat on big data as it evolves. I'll be highlighting case studies on a range of big data topics, including NoSQL databases, the Hadoop platform, predictive analytics, and more. In particular, data visualization -- which opens a new window on analytics -- will be a recurring theme.
[ Cut straight to the key news for technology development and IT management with our once-a-day summary of the top tech news. Subscribe to the InfoWorld Daily newsletter. ]
Big data has a host of different definitions. In this instance, I think Gartner got it right, describing big data as "high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization." Primarily through case studies, we'll explore how organizations are using new technology to handle tremendous data volumes, process huge data sets rapidly, and/or manage virtually any kind of data type.
Big data's big break
Why has big data "arrived" at this particular moment? The root cause is the Web itself. With the Web, we are no longer dealing with thousands of users of a client-server application powered by an RDBMS. Web applications are open to worldwide consumers. Millions of users instantly start generating data by interacting with those apps.
That level of activity requires highly scalable systems to keep users engaged -- and has helped spawn the NoSQL movement. And it requires analysis of huge, semi-structured data sets in order to figure out how to optimize the user experience and monetize behavior. That's given Hadoop its start and pushed the evolution of the MPP (massively parallel processing) analytic databases and new sets of business intelligence tools.
The innovators and early adopters of big data have found conventional technologies insufficient. They've embraced NoSQL and Hadoop in particular because they were not satisfied with traditional technologies in terms of cost and effort.
