Google was one of the first companies to illustrate the power of the map/reduce design pattern that powers the processing of big data. As the outside world took notice, some clever open source programmers created Hadoop so that everyone could scale their compute jobs. Over the years, the tool has grown more and more sophisticated, and now there are rumors leaking out of Google that the programmers inside look on jealously at the fertile open source project. The follower is now the leader, at least in the eyes of some.
Hadoop subprojects are turning the framework into more than a way to coordinate a rack of machines working on the same problem. The Hadoop Distributed File System will retrieve data from any node. If that's not enough, there are distributed database layers like Cassandra and HBase that work with the network. If basic queries aren't enough, a subproject called Hive will create summaries of the data throughout the network. In all, you have all of the tools for chewing through huge piles of data spanning tens, hundreds, or thousands of nodes.
The Bossies 2011 index:
Bossie Awards 2011: The best open source software of the year
Bossie Awards 2011: The best open source applications
Bossie Awards 2011: The best open source desktop and mobile software
Bossie Awards 2011: The best open source application development software
Bossie Awards 2011: The best open source data center and cloud software
This slideshow, "Bossie Awards 2011: The best open source application development software," originally appeared at InfoWorld.com. Follow the latest developments in application development and open source at InfoWorld.com.
