High-performance computing: Supercharging the enterprise
Thanks to lower barriers to entry, compute clusters and grids are moving out of the labs and into the mainstream
Merlin Securities, a new prime brokerage providing trading, financing, portfolio analysis, and reporting for multibillion-dollar hedge funds, needed a competitive edge. Its larger rivals, such as Bank of America, Morgan Stanley, and UBS, had the advantage of expensive mainframes that could consolidate and analyze millions of trades each day and return reports via batch processing the next morning that measured performance on a monthly basis. So Merlin outclassed its competitors by returning trade performance information in near real time with performance measured on a daily basis and performance attribution on multiple levels, including in comparison to other securities in a market sector, numerous benchmarks, and other traders in the firm. What’s more, it did it using an inexpensive compute cluster made up of four dual-processor Dell PowerEdge 2850 servers.
Merlin’s story is a perfect example of where the HPC (high-performance computing) market stands today. Multimillion-dollar systems from Cray, Fujitsu, IBM, and NEC are rapidly giving way to clusters or grids of inexpensive x86 servers from mainstream vendors such as Dell, Hewlett-Packard, and IBM. Meanwhile, the specialized operating systems of yesterday have largely been replaced by Linux.
“The HPC market has been turned on its ear,” says Earl Joseph, research vice president for high-performance systems at IDC. “Cray, NEC, and Fujitsu now make up less than one percent, while HP and IBM are at about 31 percent each, with Sun at 15 percent and Dell at 8.5 percent.”
Up-and-coming Linux hardware vendors such as , Rackable Systems , and Verari Systems have begun selling scads of standards-based server clusters into the traditional HPC/technical computing markets, such as higher education, the life sciences, oil and gas, and industrial design. More importantly, however, inexpensive HPC is finding its way into much smaller environments than before, as well as into financial, search, and logistics applications previously outside its province.
 Click for larger view. |
“The future server architecture for the entire enterprise is a cluster,” says Songian Zhou, CEO and co-founder of grid/clustering software vendor Platform Computing . “The server is just a component; the server operating system, a device driver. The real OS will be a layer of resource scheduling and allocation software.”
But Frank Gillett, principal analyst at Forrester Research, is not quite as bullish. “High-performance clustering may be getting cheaper, but taking advantage of it is not really getting easier,” he says. “Clustering will become accessible at the rate at which software is written for the architecture, and it will be quite a while [before] that’s all sorted out.”
