But do smaller businesses really need access to big data? Simply put, yes. All companies have big data whether they realize it or not. For example, most online businesses collect large volumes of data from their log files and clickstream data. For companies that don't have such data streams, storing gigabytes not terabytes of information, big data tools let them tap into the vast trove of publicly available data sources.
Witness: The World Bank makes its statistical data about the entire world available online, and the Library of Congress is archiving all Twitter data since March 2006. What's more, there are plenty of news and investment data services that offer low-cost access to their information. Big data techniques can be used to analyze these data sources, in addition to your own data -- or both together.
Take, for instance, FlightCaster, a company that offers improved accuracy in predicting flight delays and, in the process, outperforms estimates by the major airlines. FlightCaster mines large amounts of historical data on domestic flights and factors in real-time conditions, as well as other proprietary elements using much of the same (public) data available to the airlines. FlightCaster's secret sauce is its practical understanding of big data analytics and the application of the proper tools to calculate the outcome in real time.
As costs fall and companies think of new ways to correlate data, big data analytics will become more commonplace, perhaps providing the growth mechanism for a small company to become a large one. Consider that Google, Yahoo, and Facebook were all once small companies that leveraged their data and understanding of the relationships in that data to grow significantly. It's no accident that many of the underpinnings of big data came from the methods these very businesses developed. But today, these methods are widely available through Hadoop and other tools for enterprises such as yours.
This article, "The big promise of big data: What you need to know today," originally was published at InfoWorld.com. Follow the latest developments in enterprise data management at InfoWorld.com.