Knorr: Can you talk about about firm plans for IBM applications?
Smith: We try to build applications around what customers have asked us for. It's fascinating how these domain experts want to use very large data almost in a personal computing manner. It's not on their machine at the desktop, it's someplace over there, but they want to manipulate that data like it was right there at their fingertips. I'll mention one specific application, which is what we're going to do with Watson on "Jeopardy."
Knorr: Oh, yes. But how does Big Data play into that?
Smith: Well, they use a number of technologies as the foundation, one of them being Hadoop. They read in -- what is it? -- a million books, 2 hundred million pages. And then they have specialized analytics that go through and do the analysis and categorize things, very much on the text and natural language part of it, machine learning. And then come out with an application.
Now, in two weeks, when Watson is on "Jeopardy," you're not going to hear about the technology underneath it. Nobody except us are going to care about that. But that's the cool thing, in my mind. What's the technology that makes this work differently than I've seen before? I'm just going to take for granted that these new technologies like Hadoop and others are enabling that application you would not have thought about before.
Knorr: Give me some other examples.
Smith: Well, medical. Maybe you want to come back with a half a dozen answers that look feasible. Just like with Watson, you'd like to trace back exactly how it came to those answers so you could understand how it arrived there. We've done some things with doctors and BigInsights. One of their points is, "As you create data and process it, I'd like to see every step, backwards and forwards, because another researcher might look and say -- oh, you took this assumption and went there. I might want to take that and go this way with it."
And I think that's the value of this, really, is that it puts it in the hands of more of these professionals. Because otherwise it goes to IT, IT writes it, goes back and says, "What do you think?"
"Not quite what I had in mind." It adds development costs, it adds all the other standardized types of things, without any new value added to the business.
Knorr: Well, that poses an interesting challenge, doesn't it? Because right now, in most cases, there's some sort of BI specialist who actually stands between the technology and the end-user. And usually that's a good thing, or they're going to end up asking the wrong questions and drawing the wrong conclusions. The domain areas that you're exploring are incredibly diverse. It's not simply a narrow business context; it can be all sorts of contexts. So to find these patterns, the interaction of the end-user or the consumer of the data should be much more direct than in more normal business intelligence.
Smith: I think it should be, and it's also richer for everybody, because otherwise it's just like compartmentalizing someone and saying, "Please look at this from a BI perspective." As opposed to, "You know, here's my hunch on what might be happening. Now let's go do that. And now, did my hunch pay off or not?" And so now a BA or BI expert is like, "Ah! Maybe I know where to go look for part of these things, too."
I think some of these cases will show you that using Hadoop and using BigInsights, I can get you answers in a couple of minutes. So it feels like my cost is much smaller. That's very interesting. I like that. As opposed to more traditional IT- or BI-type things, which take some time. So I think this opens the aperture for better collaborations, richer collaborations, with business analysts and business professionals. And I think the term "analytics" is too narrow. It just doesn't add up to the business problems we're seeing out there.
Knorr: But this kind of circles back again to the idea of new types of applications that can allow this sort of correct interaction.
Smith: Yep. And this is why Watson is an interesting one. It's a new type of application of what we're trying to do.
Knorr: When can we expect to see new products coming out, and what part of IBM will they be coming out of?
Smith: Well, I can't talk about the other ones yet. But I think you'll see that Big Data is going to be kind of a horizontal component to a lot of what we do. You mentioned log files, and Tivoli does log files. There are different domains in here where Big Data touches us, and we'd like to be in a position of offering those kinds of capabilities for a broad range of our solutions.
Knorr: Great. Thank you very much.
Smith: Thank you.
This article, "A conversation with IBM's Mr. Big Data," originally appeared at InfoWorld.com. Read more of Eric Knorr's Modernizing IT blog, and for the latest business technology news, follow InfoWorld on Twitter.