Xserve review Part 3: Power, cooling, heat and noise

(Apple requested that I drop the "Xeon" from Xserve's product name. I don't mind. Since the Xserve G5 I was using has gone back, the distinction is moot for me. Xserve it is.)

This is a project that called for my removal from my office. I'm at the VMworld conference in Los Angeles, connected to Xserve via Remote Desktop, and finally able to write instead of test.

The consensus among commenters is that I should skip the rest of these notes and go straight to the review. I promised this bit on power, cooling and noise next, so I'll fulfill my promise and post the review after this.

Compared to a two socket, dual-core Netburst Xeon, Xserve eats like a bird, but there's no mistaking its power draw for a desktop's. Xserve has a relatively smooth soft start: It doesn't spike its power consumption excessively at power-up. Some servers pull in gallons of extra juice when first powered on to wind up all of the fan motors and disk spindles, and to initialize peripheral devices. A rack of Xserves won't overwhelm your circuit breakers when you fire them up simultaneously.

I measured the range of Xserve's operating power utilization, with varying levels of compute and peripheral load, at 295 to 400 watts. It doesn't have a wide intermediate range; it tended to hover at the top or the bottom. Watching power levels in real-time with Apple's Server Monitor, I saw only three distinct stages of power adjustment reflected in CPU wattage and core voltage. Apple's interface doesn't report clock speed. As it is, I'm not surprised that total power consumption, measured at the AC outlet, doesn't step up and down in smaller increments. Power utilization was nearly identical whether one or two power supplies were used.

As Apple explained it to me, Xserve's power management (which is not user-adjustable) favors performance over efficiency. When it appears to be in a quiescent state, Xserve is actually hard at work waiting for work to do. The advantage is that the system reacts with minimal latency to sudden changes in demand, whereas a server in a power-reduced state might have to wake from that state before it can meet that demand. The disadvantage is that the nominal floor for power draw and heat dissipation is set high, and OS X provides no easy means by which the administrator can lower it.

Xserve apparently has the option of being put to sleep, but the evaluation system wouldn't do it. A possibly related anomaly was that the processor preferences pane would not allow me to switch off CPU cores that had been subjected to\ heavy (100%) load. It permitted this when the system was less loaded, allowing me to knock the machine down to a single core. However, disabling cores, or disabling the second CPU entirely, had only a marginal effect on temperature and power.

Xserve contains a total of nine fans, seven side by side in an array and one each for the (optional) dual power supplies. Each of the fans in the main bank of seven has two independent rotors (and so, motors). The double rotors create a push/pull airflow that takes up relatively little real estate. All fans are monitored individually by Apple's System Management Controller, but at present, all of the fans change speed at the same time. All of intake rotors are assigned one RPM value, and the output rotors are assigned another.

The fans are monitored individually. This brought to my mind the possibility that Apple might update Xserve to reduce noise by adjusting fan speeds according to need. For example, Xserve's front-side bus controller runs markedly hotter than any other system board component, so giving it special attention might allow the other fans spin more slowly. On the one hand, it would make Xserve quieter, but on the other, it would add complexity to a cooling system that is currently simple and predictable at the cost of noise.

It appears that fourteen fans keep this server right on the brink of thermal balance for the configuration I'm running. The Xserve review unit arrived loaded with 3 GHz CPUs, two power supplies, a PCI Express dual gigabit Ethernet card, a PCI Express Fibre Channel host bus adapter, an 80 GB SATA boot drive, two 750 GB secondary SATA drives and 8 GB of FBDIMM memory. That's not worst-case for heat, but it's close enough. When I drove all four CPU cores at 100 percent, the seven fastest fans ramped up smoothly to about 9000 RPM. At that speed, Xserve seemed to take any opportunity to slow the fans. There are short delays in SPECjbb, the burn-in benchmark I chose, and as soon as Xserve hit the leading edge of that delay, the fan speed dropped quickly, but still smoothly.

OS X Server Tiger includes Server Monitor, a detailed dashboard of Xserve's sensors. The system is exceedingly self-aware, with temperature sensors planted everywhere there's a temperature worth measuring. Those numbers are fun to watch, but I was only concerned with measuring external temperature. In a 78 degree (F) space with unimpeded airflow fore and aft, the front of the chassis stayed cool. At maximum load, the rear panel hit a high temp of 118 degrees. The bottom and side surfaces were barely warm, but the top of the chassis averaged about 95 degrees except for a well-delineated hotspot that registered 108. I pity the system above an Xserve in a rack that isn't set up properly.

Finally, the noise numbers were about what my ear led me to expect: Xserve is loud. With seven fans turning at 9000 RPM and seven more turning at 5000 RPM, the sound pressure one foot from the front of the server was 60 dBA. One foot from the rear panel, the sound pressure was 65 dBA, placing Xserve among the loudest 1U rack servers I've used when under heavy load.