Hortonworks and Pivotal vie for Hadoop dominance

The companies' new releases offer radically different ways of making Hadoop a useful enterprise tool

If you want a case study in the radically different ways Hadoop can be packaged and distributed as an enterprise data tool, look no further than this week's product releases by Hortonworks and Pivotal. On the one hand, you have a pure open source play; on the other, you'll find an avowedly commercial product outfitted with a new licensing model.

Hortonworks' pure open source play

The pure open source product is of course by Hortonworks: the 2.1 release of its Data Platform product, which bundles Hadoop along with a slew of other Hadoop-related open source products (all under Apache governance) to create a package with more utility to enterprises than might be had by deploying Hadoop manually.

This includes more than, say, SQL-style querying and reporting functions, although those are included, by way of the Stinger project. Hortonworks also adds data lifecycle management and security functionality, plus engines for stream processing and search.

One of Hortonworks' motivations for delivering Hadoop with this broad a range of additional components is for it to serve as evidence for how, as Hortonworks' Jim Walker (director of product marketing) put it, "the open source community can outpace the innovation of any proprietary solution, much faster and in a much more complete manner."

While there may be other proprietary products with Hadoop at the core that offer a comparable feature set, Hortonworks believes the contributions to Hadoop -- which come from folks like Microsoft, Red Hat, and Yahoo -- will allow the whole product to iterate faster than a commercially developed solution.

Pivotal's licensing magic

Pivotal, on the other hand, doesn't seem to have any qualms about taking the commercial approach -- not when it believes its analysis and reporting tools are built first and foremost as business tools to extract value from data. But its latest Hadoop-powered item, the Pivotal Big Data Suite, distinguishes itself from Hortonworks and the rest of the Hadoop pack mainly in how it's being licensed.

This isn't to say Pivotal's product, which bundles all its Hadoop-related offerings under a single unified licensing scheme, isn't intriguing as a big data product. Pivotal HD 2.0, provided with the bundle and released earlier in March, adds in-memory database processing, SQL-style querying, and various programmatic analytics on top of Hadoop. One of the vaunted pluses of using Pivotal's toolset is cutting IT out of the equation when it comes to provisioning data in a warehouse or other environment for analysis; the data can be crunched as-is.

But the real attention-getting feature is, again, the licensing. Pivotal's new suite is licensed per-core, with each component of the suite available at the same licensing rate and with the purchased licenses available in a pool that can be freely shuffled between products in the suite as demand indicates.

Pricing wasn't released beforehand, but the company vows to make it an "aggressively priced" offering. But the appeal here seems less to those already running Hadoop and more to those looking to break away from other, more costly analytics suites with more onerous and restrictive licensing (such as Oracle).

More than one way forward

A big part of why Hadoop continues to generate such intrigue is the way it's gradually eclipsing not just data warehousing systems but also processes. Instead of, for instance, backing data up and moving it into an offline format where it can't be processed, said data can simply be left in Hadoop -- an approach taken by Cloudera, too -- and kept live.

Both Hortonworks and Pivotal are eyeing such developments, and the newest versions of both of their products nod in that direction, where big data is an always-on, always-available proposition. Pivotal's guaranteed to continue making a business-grade audience for itself. But Hortonworks' pure open source approach, which feeds right back into Hadoop and all the other Apache projects that drive it, stands a better chance of becoming the more influential of the two.

This story, "Hortonworks and Pivotal vie for Hadoop dominance," was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest developments in business technology news, follow InfoWorld.com on Twitter.

Copyright © 2014 IDG Communications, Inc.