Cycle harvesting, in particular, is largely limited to universities, government research agencies, and altruistic grids such as Parabon Computation’s Compute Against Cancer project. “Most Fortune 1000 companies need to get the job done with a guaranteed quality of service, and you’re not going to get that with cycle harvesting,” says Verari CEO David Driggers.
And then there are some scenarios for which the entire loosely coupled clustering paradigm is unsuited. Applications such as weather forecasting, seismic analysis, and fluid analysis have to run interdependent calculations that require message-passing among cluster nodes during job execution, according to Forrester Research, which means they need a more tightly coupled architecture.
A Small Matter of Software
In contrast to the simple, Gigabit Ethernet-connected designs of loosely coupled clusters, tightly coupled systems typically use some incarnation of the MPI (Message Passing Interface) standard to communicate between processes, and the clustered servers are linked with a high-speed interconnect such as InfiniBand, Myricom Myrinet, or Quadrics QsNet. Applications usually have to be heavily modified or written from the ground up for tightly coupled HPC, although some vendors, such as Virtual Iron, provide virtualization software that claims to allow server applications to run unmodified across the cluster.
“Writing for MPI is not easy,” says Ed Turkel, product marketing manager for the HPC division at HP. “Most of our industrial accounts go to an ISV with a commercial application.”
Vendors of these applications, including Abaqus, Accelrys, Fluent, Landmark, MSC.Software, and Schlumberger live in a different world from typical enterprise application providers. “Many of these applications tend to look more homegrown than off-the-shelf,” Verari’s Driggers says.
Driggers adds that programmers are finding new, better ways to unleash applications from being tightly coupled, but even loosely coupled applications are usually either written that way from the start or else have to be modified to support clustering. Most ISVs don’t see sufficient demand to go through the trouble of modifying their applications for HPC, so customers are left to do it on their own -- something they are seldom equipped to do.
A Full-Service Approach
“Most enterprise users are afraid to modify an application without ISV approval,” Gillet says. “Some will approach the ISV, who will tell them to work with a vendor such as United Devices, Platform Computing, or Data Synapse,” which provide grid or clustering software and services for modifying applications to run on them. In addition, software licensing schemes are also typically not geared to clustering and grid scenarios, where per-server or per-CPU pricing is prohibitive.
For this reason, hardware vendors are moving toward providing customized, turnkey HPC solutions, either mixing open source HPC components such as the MPICH, LAM, and the Globus Toolkit with their own components or partnering with grid and clustering software vendors or HPC application ISVs.