Truviso: New tricks with old SQL

If Werner Heisenberg had pursued business analytics in the early 21st century instead of quantum physics in the early 20th, he might have struck on a different uncertainty principle: If your analysis and decision-making capabilities don't keep pace with the explosion of critical event data across enterprise networks today, soon you won't know the state of your business, the direction it's heading, or how fast it's getting there. 

A host of technology companies are working on the science behind taking these measurements. Perhaps the newest of these event stream processing or complex event processing vendors is Truviso (formerly Amalgamated Insight), a Silicon Valley startup founded on work led by co-founder and CTO Michael Franklin at U.C. Berkeley, where Franklin is a professor of computer science.

The seed of Truviso was planted in Franklin's frustration at what he considered wrong-headed approaches to the analysis problem. Many seemed to think that analyzing event streams -- be they RFID events from sensor networks, service events from SOA networks, system events from traditional networks, events from financial markets, or other kinds of electronic transactions -- was merely a matter of plumbing. If you could just get this device talking with that device, you would somehow enable pervasive computing.

"And my view," Franklin recalls, "was that once you solved the plumbing problems, what you've got is a data management problem. Because you've got huge volumes of data being created all over the place. Lots of applications need that data, and lots of users need insight into what's going on in these dispersed environments."

Beyond the plumbing, others were trying to reinvent the wheel. Attending an RFID standardization meeting as a guest, Franklin listened as group after group, each working at a different level of the system, described the data at their level and how to access it. And he realized, each one was defining a data model and a query language, and each one was different. That's when the seed sprouted.

"Sitting there as a database guy, I was just getting irritated," he says. "Jeez, why wouldn't you just use SQL for all of this stuff across all the levels? And on the way home from that meeting, I realized, well, you know, why don't I just do that."

Truviso's three-part solution combines a stream processing engine, a Java-based integration framework, and a visualization front end based on Adobe Flex. Naturally, the processing engine, which is embedded into the open-source PostgreSQL database, uses SQL to continuously query events as they flow past.

"It lets you write SQL queries that are basically always running," Franklin explains. "They're always producing the answer so far based on the data they've seen so far."

To enable SQL to query infinite data streams, Truviso extended it in two ways, adding a windowing capability to restrict the amount of data it would be exposed to in any instant and adding some "syntactic sugar" to define event patterns you might want to inspect. The "continuous analysis" engine is adaptive, Franklin says, meaning you can add or remove queries on the fly.