One customer, interclick, uses ParAccel to analyze demographic and click-through data to let online advertising firms know which ads to display to end users, he said. It has to work in near real-time, so interclick runs an in-memory database of about 2TB on a 32-node cluster, Zane said. Other customers with larger data sets use a disk-based architecture.
ParAccel also lets developers write SQL queries, but with extensions so they can use the MapReduce framework for big-data analytics.
"SQL is a really powerful language, it's very easy to use for amazingly sophisticated stuff, but there's a class of things SQL can't do," he said. "So what you've seen occurring at ParAccel, and frankly at our competitors, is the extensibility to do MapReduce-type functions directly in the database, rather than try to move terabytes of data in and out to server clusters."
Cloudant, which makes software for use on-premise or in a public cloud, was the only company on the panel that has developed a "noSQL" database. It was designed to manage both structured and unstructured data, and to shorten the "application lifecycle," said co-founder and chief scientist Mike Miller.
"Applications don't have to go through a complex data modelling phase," he said. The programming interface is HTTP, Miller said. "That means you can sign up and just start talking to the database from a browser if you wanted to, and build apps that way. So, we're really trying to lower the bar and make it easier to deploy."
"We also have integrated search and real-time analytics, so we're trying to bring concepts from the warehouse into the database itself," he said.
The company's software is hosting "tens of thousands of applications" on public clouds run by Amazon EC2 and SoftLayer Technologies, according to Miller.
Cloudant databases vary from a gigabyte all the way to 100TB, he said. Customers are running applications for advertising analytics, "datamart-type applications," and "understanding the connections in a social graph -- not in an [extract, transform and load] workflow kind of way using Hadoop, but in real time," he said.
While cloud databases can solve scaling problems, they also present new challenges, the panelists acknowledged. The quality of server hardware in the public cloud is "often a notch down," said Zane, so companies for whom high-speed analytics are critical may still want to buy and manage their own hardware, he said.
And while many service providers claim to be "cloud agnostic," the reality is often different, Miller said. Cloud software vendors need to do "a lot of reverse engineering" to figure out what the architectures at services like Amazon EC2 look like "behind the curtain," in order to get maximum performance from their database software.
Still, Sharir and Zane were both optimistic that "big data analytics" would be the"killer application" for their products. For Starkey it is simply "the Web."
"Everyone on the Web has the same problem, this very thin pipe trying to get into database systems," he said. "Databases don't scale, and it shows up in a thousand places."