Inside story: The soul of a failed machine

How a start-up tried -- and failed -- to take supercomputing world by storm

In a data center at Purdue University, a rare supercomputer is crunching numbers for researchers studying a broad range of scientific problems. The 5,832-processor machine is capable of performing 8 trillion calculations per second, yet it consumes just a fraction of the electricity needed by Purdue's other supercomputers. The machine is one of a kind at Purdue -- because the company that built it doesn't exist anymore.

It comes from the recently folded SiCortex, a start-up founded with the idea of building the world's most efficient computers. SiCortex's story illustrates the difficulty of trying to build a new systems company in a maturing industry. Even if the idea is innovative, the product solves real-world problems and the company attracts an experienced management team and venture backing, success is far from guaranteed.

[ IT snake oil: Six tech cure-alls that went bunk. | InfoWorld reveals tech's 25 biggest flops of all time. | Keep up on the day's tech news headlines with InfoWorld's Today's Headlines: Wrap Up newsletter. ]

"It's always difficult to build this type of company," says Jud Leonard, SiCortex co-founder and chief architect. "You're up against a very well established, strongly entrenched business and you know, Intel is a fierce competitor. We never imagined it was going to be an easy job."

IT industry graveyard 2009

Purdue CIO Gerry McCartney says SiCortex helped him address one of his nagging challenges: providing compute cycles to researchers without overwhelming his power and cooling systems. McCartney was hoping to buy one or two more SiCortex clusters and put them together to build one giant scientific research machine, but he'll never get the chance.

In a better time, SiCortex might have carved out a profitable niche in the high-performance computing market. Instead, the vendor shut its doors in May when venture capitalists yanked its funding. SiCortex officials believe the recession played a major role in the company's demise, but even in a good economy the company would have been fighting an uphill battle in a market dominated by Intel- and AMD-based supercomputers.

No other vendor has purchased SiCortex's core intellectual property. But its technology will live on for a time in data centers such as Purdue's, where the SiCortex cluster performs millions of CPU hours of research per month for researchers in aeronautics, computer science, nanoelectronic devices, mechanical engineering and other fields.

"We're not going to unplug it just because the company's gone away," McCartney says of his SC5832, the highest-end machine sold by SiCortex. "The promise of this was a very low power consumption device with a very friendly carbon output. The limiting factor on most of our purchases is the power and cooling requirement. SiCortex was really a very good machine in that regard. It's terribly disappointing that they've gone out of business."

Purdue was one of about 80 customers of SiCortex, based in Maynard, Mass., in the building that once served as headquarters for the once high-flying Digital Equipment Corp. The company was founded in 2002 by chief engineer Matt Reilly and Leonard, who knew each other from their days at DEC. Reilly and Leonard were interested in building a new type of high-performance computing cluster and "thought there must be a better way to build clusters than to pull together a bunch of desktop computers," Leonard says.

SiCortex designed its architecture from the ground up to handle multithreaded applications in scientific research settings while using unusually low amounts of electricity. The company used relatively slow, inexpensive processors but stitched them together with a very fast, distributed communications network in which every processor contains some network ability, in a so-called Kautz graph topology.

As Leonard explains, "Each component of the system includes a small portion of the switch fabric and you wire them together so you don't need a separate component to do the fabric switching." Low power draw made it possible to achieve high density -- up to 5,832 processors in a single system.

The result is a machine ideal for many types of applications that require parallel processing, according to McCartney. Workloads that need fast individual processors are not well suited for the SiCortex machine, he says. But when researchers need lots of processors to perform a task, and don't care about the speed of each one, the SiCortex computers are often the right choice, he says.

SiCortex -- which brought former Novell executive Chris Stone on as its CEO -- received its first funding in 2004, shipped its first beta machine in July 2007 and went into production early in 2008. SiCortex received about $68 million in financing and venture debt over the years from investors including Chevron Technology Partners, Flagship Ventures, JK&B Capital, Prism VentureWorks, and Polaris Venture Partners.

After the economy went south, one investor pulled out and the rest soon followed. "What happened was one of the investors had overcommitted themselves and backed away," Leonard says. "The others just didn't feel they had enough disposable cash to replace that, and they simply had no choice but to let it go."

SiCortex, with about 80 employees, had been building momentum. Its largest machines sold for upwards of $1 million and the company announced revenue growth of more than 100% in the first quarter of 2009. SiCortex wasn't profitable yet, but "we were running ahead of plan," Leonard says.

But building a new systems company requires large amounts of capital over many years, and SiCortex needed more in order to build the next version of its HPC cluster. "The plan called for cash flow to break even in about a year and a half, which was really the problem," Leonard says. "During that time, we needed to bring out a new system and that meant plunking down millions of dollars for the new chip design and getting it through fab. There really wasn't a way to defer those expenses. We knew we were going to have to be competing against Intel's Nehalem chips in the very near future. There just wasn't much of a way to cut the costs and bring the break-even closer."

After shutting its doors SiCortex kept on a skeleton crew staff to support existing customers. "One of the things I found particularly amazing is after the company shut down … we got calls from people who had heard about us and wanted to buy a system before they were out of stock," Leonard says.

Louisiana State University researchers Erik Schnetter and Steven Brandt are among those who wanted to buy a SiCortex machine but weren't able to before the company went out of business. The researchers, who work at the university's Center for Computation & Technology, were able to remotely access a SiCortex cluster to study binary black holes and gravitational waves.

Schnetter said the SiCortex system was comparable to IBM's Blue Gene/P, with similar power draw per core, but the SiCortex machine was easier to use. "It had the best flop per watt of any system out there," Brandt says. "It also had a fairly fast, interesting proprietary network on it. We were hoping to figure out interesting ways of exploiting that."

Despite customer interest, SiCortex faced numerous challenges that made success an uphill battle. Each individual processor in a SiCortex system was not that powerful, notes IDC analyst Steve Conway. But the analyst says the biggest problem was SiCortex was "trying to do something daring and against the grain."

The HPC market has swung heavily toward the use of commodity processors, namely x86 chips from Intel and AMD. But SiCortex went against conventional wisdom by building its own processors and this decision limited the company's market to early adopters, Conway says. In building its chips, SiCortex obtained intellectual property from several vendors, including MIPS Technologies, and tweaked the design to meet its own needs.

"For a company like that entering an established market, it takes time to get a good footprint and the right kind of profitability," Conway says. In this case, "investors were not willing to wait the normal amount of time that in a healthier economy a company like SiCortex would have had."During its years of operation SiCortex had purchased intellectual property from several companies, including QLogic's PathScale compiler business. Supercomputer company Cray decided to purchase the PathScale compiler suite from SiCortex after it went out of business, because the PathScale technology is used by many Cray customers.

But the rest of SiCortex's intellectual property has gone unclaimed. Leonard says he "can't understand why somebody hasn't bought it." But the technology is aimed at a marketplace that is difficult to sell into, says Cray vice president of scalable systems Barry Bolding. The HPC world may not object to energy efficiency, but it's not often the top priority, he says.

"The idea of high-performance computing that's highly energy efficient is a difficult space, because you're trading off performance," Bolding says. "What we've found at Cray is performance is king. It still really is difficult to make a full value proposition on only the energy efficiency."

SiCortex typically sold smaller systems than Cray, Bolding says. In a better economy SiCortex could have survived "but I think they would have had trouble moving up in to the high-performance space," he says.

Conway fears that SiCortex's demise will stifle innovation in the high-performance computing industry. "I think, unfortunately, one of the lessons is that this is not going to encourage other vendors to take risks," he says.

But the men who founded SiCortex, while hoping for a better outcome, don't regret the chance they took. In a blog post after SiCortex shut down, Reilly says he believes there is still room for non-x86 machines in the HPC market.

"It is possible for a small company to compete in the computer systems business," Reilly wrote. "There are some who will say that nobody can compete against 'commodity manufacturers. Ignore them. … There are only two true commodities in the computer business: DRAMs and wafer area. Everybody pretty much pays the same price for DRAMs. Wafer area is what you make of it. If you insist on building giant 100W chips, life will be tough. But if you use the silicon wafer area for something new, different and efficient, a market will open up to you."

Leonard, who is working on some software ideas as he figures out what his next venture will be, says the SiCortex team made mistakes but that he'd "do it again in a minute."

"Of course I'd rather it hadn't ended the way it did, but it was a wonderful time," Leonard says. "There were lots of mistakes of various kinds. I can wish that we had been more aggressive technically. I can wish that we had pulled the schedule in tighter. There are lots of things that in hindsight I would prefer to do differently. I've never been on a project in which that wasn't true."

Follow Jon Brodkin on Twitter:

This story, "Inside story: The soul of a failed machine" was originally published by Network World.