I've always found it odd that some capabilities in the datacenter are handled by specialized appliances and others are handled by specialized software running on top of standard servers. What is it that makes, say, a router or a firewall more popular as an appliance and a database more popular in software? Is it inherent in the task being handled or is it just a case that system admins and buyers are used to consuming technology in a certain way? And more importantly, does this gap provide an opportunity for new companies to disrupt the market by making the technology easier to consume or more powerful?
Last week at the MySQL Conference and Expo, there were several announcements by new appliance vendors who appear to be doing things in silicon that have not been easy (though in some cases, perhaps they are still possible) in software alone. Kickfire has been at it the longest with a data warehouse appliance that has a dedicated SQL chip, much like a graphics processor that speeds up some queries by two orders of magnitude, making them up to 500 times faster.
[ Keep up with the latest open source developments with InfoWorld's open source topic center. ]
The data warehousing market is one that seems ripe for disruption given the high prices of "last gen" appliances like Terradata and Netezza. Data warehousing is one market where there has been acceptance of dedicated DBMS appliances, so it makes sense that at least some buyers are used to working that way. (On the other hand, there are plenty of non-appliance-based solutions, like Infobright that runs on standard hardware; I believe the market is large enough to have both approaches.)
The example of new general-purpose appliances comes from Schooner, which has two on offer and more expected in the future. There's a MySQL appliance and a memcached appliance, and both are built on Intel's multicore Nehalem chips and targeting Web 2.0 workloads. (The appliances are built and supported by IBM -- which is ironic given that CEO and founder John Busch worked for Greg Papadopoulos at Sun for several years.)
The key premise behind Schooner is that there's an opportunity to do a greater degree of systems integration through open source. Busch described to me how in the past there was always great cooperation between the OS team, the compiler team, the chip team, and so on, to optimize overall system performance. But with the emergence of commodity components, CPUs, motherboards, and the like, each subsystem is locally optimized, but overall system performance is not. Busch realized there is an opportunity with open source technology to change the game by getting in and tuning the heck out of every software component to maximize throughput and efficiency. The net result is that with Schooner you can consolidate multiple servers and achieve higher performance at a lower operating cost.
No doubt there are going to be many ways to slice and dice this market. But it's great to see that open source is powering a new level of innovation in the industry.