Cloud and analytics galore: IBM finally reveals its master plan

At its big data event, IBM lays out an end-to-end cloud and analytics strategy, with a vision for the future that makes sense for developers and big enterprise customers

virtual data wave pattern

“IBM is like an armada,” grinned Sean Poulley, vice president of databases and data warehousing for IBM. “When IBM as a company, as an armada, goes in one direction, we’re a pretty strong force.”

The impression I get from attending the IBM Insight 2014 event in Las Vegas this week is that IBM’s fleet of business units is indeed steaming ahead, with an array of recent announcements lining up in a surprisingly coherent formation.

Until fairly recently, I found it hard to make heads or tails of IBM’s cloud strategy, at least until the 2013 purchase of SoftLayer gave me a clue. As for analytics, I knew the company was an early leader in big data with InfoSphere BigInsights, but I didn’t see how that Hadoop-centric play fit with the rest of the company -- nor with, say, Watson or the Cloudant acquisition.

But Monday, in an indoor arena packed with 12,000 IBMers and customers, Bob Picciano, senior vice president of the Information & Analytics Group, emceed a 90-minute group presentation of IBM’s master plan. Watson, the Apple partnership, the Cloudant and SoftLayer acquisitions, and even the selloff of IBM’s server and chips businesses all started to make sense.

Analytics everywhere

“Data is the what, cloud is the how, and insight is the why,” intoned Picciano. Then Inhi Suh, vice president and general manager of Big Data, Integration & Governance, announced three new analytics offerings that illuminated the insight part of that formulation.

The first is DataWorks, essentially a cloud version of IBM’s InfoSphere Information Server offering, which provides data integration, governance, and cleansing. Next comes another cloud entrant, IBM DashDB, a data warehouse as a service. The third falls from cloud to earth: Cloudant, a NoSQL document database as a service based on CouchDB, will now be offered in an on-premises version.

Wrapped around these new offerings is IBM Watson Analytics, a “natural-language based cognitive service” announced last month that enables ordinary users to access advanced and predictive analytics capabilities. To that end, IBM this week added Watson Explorer for data exploration and content analytics, as well as Watson Curator to give experts the assistance they need to cull and classify documents and other source material to fuel useful analytics outcomes.

What do all these pieces add up to? Well, DataWorks addresses one of the biggest problems in cloud computing -- integration between the cloud and on-premises data stores -- and adds a generous helping of data cleansing and governance. These have traditionally been strong areas for IBM and provide substantial differentiation from the cloud platform competition.

IBM DashDB and Cloudant together provide a data-rich cloud environment for application developers, particularly users of IBM’s Bluemix PaaS, based on Cloud Foundry. As Poulley explained to me, “Cloudant is the data layer inside of the Apple partnership and DashDB is a warehouse service for our Web and mobile applications.” If you’ll recall, IBM emphasized the iOS apps that grew out of the Apple partnership would go beyond mobile facsimiles of enterprise applications and would be powered by analytics.

Apps of this type are part of a new category IBM has branded as “systems of insight” (wedged between Geoffrey Moore’s "systems of record" and "systems of engagement"). The idea is to create business value by keeping analytics turned on all the time, so businesses can better target customers, predict system failures, or divine new patterns in everything from patients’ vital signs to supply chain seasonality.

Normally, ambitious plans for spreading analytics around run afoul of human bottlenecks: There simply aren’t enough analytics experts or data scientists to go around. That’s where Watson comes in. If you hadn’t noticed, machine learning is the darling of Silicon Valley's emerging technologies -- and is considered absolutely necessary for making sense of big data. IBM was early to this game and is repurposing its champion "Jeopardy" player to recognize data patterns and interact with nonexpert analytics users.

Attracting a new generation of developers

Watson’s capabilities can be accessed through a new set of cloud APIs. IBM's Watson Developer Cloud, announced earlier this month, makes an array of services available through IBM’s Bluemix PaaS for building what IBM describes as “cognitive apps.” Add Cloudant, DashDB, DataWorks, and more to the Bluemix stack, and you have a rich, unique stack in which developers can immerse themselves.

IBM has a long history of serving developers, but for more than a decade that legacy has centered on Java. With the acquisition of Cloudant in February, IBM took aim at a new generation of Web and mobile developers who were under the spell of of MongoDB and JSON -- and added that attraction to its Bluemix cloud platform.

To hit that target market, however, Cloudant CEO Adam Kocoloski admitted to me that he had to do substantial work. “I’ll be one of the first to tell you that we’ve learned from [MongoDB]. We understood that we’d built something that was bulletproof to operate, but we didn’t invest as much as we needed to in the developer experience.”

Kocoloski maintains that this experience has been vastly improved in both the cloud and new Cloudant Local version. He also says his group has worked hard to port Cloudant management features to the Local version, and he is clearly excited that he has created what he considers to be a viable competitor to MongoDB.

Whether in the cloud or on premises, Kocoloski sees MongoDB and Cloudant as playing similar roles in the enterprise -- a sort of middle-tier, operational data store that draws on multiple data sources in an organization. With the data plumbing maintained beneath that tier via DataWorks, Web and mobile developers need only concern themselves with Cloudant -- plus, if they so choose, the analytics resources offered by Watson APIs.

Cloudant Local, says Kocoloski, underscores that this is very much a hybrid approach. "What we’re trying to do with this notion of a fluid data layer is actually support rich applications that span both environments and are able to connect between them," he says.

IBM knows that few large enterprises will go all-in with the public cloud and will remain hybrid for the foreseeable future. Up and down the line, IBM can now tout consistency between its cloud and on-premises offerings.

A Big Blue turnaround?

As part of its cloud push, IBM has invested $1.2 billion in building out SoftLayer data centers -- and divested in its server and chip fab businesses. Perhaps no other reallocation of resources highlights IBM’s change in direction so clearly.

SoftLayer’s buildout across Europe and other locations answers two common cloud objections: latency and regulatory constraints. Delays in round trips to the cloud are addressed by a combination of local points of presence and in-memory technology -- and data centers housed in each country can now help customers observe the complexities of local regulations.

In an interview at the Insight event, I asked Steve Mills, IBM senior vice president of Software & Systems, about the enterprise’s cloud future. Surely IBM’s cloud won't be the only one customers use, so what is IBM doing to foster portability and interoperability? He replied:

The question is, does one cloud provider end up giving me the coordination services over the top, if you will, of the other services that sit underneath? That’s a big part of what we’re investing in, are the over-the-top control services for management, for orchestration, for data control.

This proposed role as gatekeeper for the public cloud smacks a little of the old IBM. The company clearly wants to own the entire hybrid cloud proposition for its big enterprise customers. But smaller businesses may well be a different story, says Mills:

The smaller the company, the less likely they’re going to run IT. If you own the local lumber yard, basically, your business is lumber and building supplies. IT is probably not high on your list of what you want to do. I’m going to have workstation devices in my company, but connected to someone else’s servers. I’m dealing with inventory, order management, billing, and payroll. Why do I need to do any of that myself?

IBM's recent announcements leave little doubt that it fully understands the cloud value proposition -- not to mention the attractions for Web and mobile developers. Of course, the company's cloud and analytics initiative has so many moving parts, I’m in no position to determine how much is truly integrated and how much remains to be knitted. But from what I can observe, the vision has finally come into focus, and a broad swath of IBM technologies, marketing pitches, and business unit leaders appear to be cruising in the right direction.

Copyright © 2014 IDG Communications, Inc.