HP adds Lustre to cluster offerings

Hewlett-Packard Co. plans to become the first major vendor to offer a product based on the open source Lustre file system when it unveils a turnkey product based on Lustre at the International Supercomputing Conference in Heidelberg, Germany, on June 23, the company said.

Free IT resource Open Source Business Conference (OSBC) May 22-23, 2007 Sponsored by OSBC Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft

The product, called the StorageWorks Scalable File System, will be the second product based on HP's Storage Grid networked storage architecture, said Bob Schultz, executive vice president and general manager of HP's storage business. "It will be integrated with hardware... much like the RISS (Reference Information Storage System)," he said, referring to an e-mail archive and retrieval product HP announced last month.

With the announcement, HP will be the first major vendor to integrate Lustre into a product, said Phil Schwan, chief executive officer of Cluster File Systems Inc., the Boston company that manages the Lustre software development project. "They're capitalizing on being the first to market with an easy-to-use Lustre solution," Schwan said.

First developed in 2001, the Lustre file system is used by a growing number of supercomputers to let thousands of machines quickly share files. It is similar to proprietary software such as IBM Corp.'s General Parallel File System and Silicon Graphics Inc.'s CXFS.

"It's a good thing to have a scalable version of Lustre that is supported," said Scott Studham, manager of computer operations with the Pacific Northwest National Lab's Molecular Science Computing Facility, which has been beta-testing the Scalable File System since August 2003.

HP's Scalable File System will include software that simplifies the installation and management of Lustre, freeing up engineering time that would otherwise be spent learning the software, he said.

Because file system configuration can have a great effect on a supercomputing cluster's performance, assigning the right ratio of disk drives to file servers and clients using the Lustre file system is very important, Studham said. "Having someone like HP come in and provide you with a solution means you'll get those ratios right," he said.

"It's an art to configure Lustre. In the state it's in now, you really need a lot of expertise to set it up," said Robin Goldstone, group leader with the Production Linux Group at Lawrence Livermore National Laboratory, which has been using the Lustre file system on its Multiprogrammatic Capability Cluster since 2002. "If they're focusing their efforts on making it something in a box that's easier to set up and administer, then I think that's going to be good for Lustre," she said.

StorageWorks Scalable File System will be generally available in 2004 and will be followed by a third product called Scalable Tiered Storage, which will use "grid concepts" in order to simplify the management of multiple storage arrays, Schultz said.

Last year, HP began certifying a number of turnkey cluster offerings based on software from a variety of third parties including Scyld Computing Corp., Scali AS and MSC Software Corp., as well as the free NPACI (National Partnership for Advanced Computational Infrastructure) Rocks cluster distribution. Called the HP XC clusters, these products are available in configurations of 16 to 256 Intel Corp. Xeon or Itanium 2 processor nodes.