Exclusive: Sun's new big iron bends the rules

Sun Fire x4600 M2 octo-Opteron server proves a triumph of capability and design

Looking at Sun’s brand-new Sun Fire x4600 M2, most would figure it for a quad-socket system. After all, at 4U it matches the profile of the four-way HP ProLiant DL585 and Dell PowerEdge 6850. A quick peek under the hood tells a different tale, however: The Sun Fire x4600 M2 holds eight (eight!) easily swappable sockets.

Armed with current dual-core Opteron processors, this equals 16 cores per server, so a rackful would bring the total to 160 cores. The Sun Fire x4600 M2 will also be able to run with the next-generation quad-core AMD Opteron chips, bringing the total core count per rack to an amazing 320. Packed with 128GB of RAM per server, that’s a full terabyte of RAM in the same rack. At $51,995 for a single x4600 with eight dual-core Opteron 8128 CPUs, 32GB of RAM, and two 73GB SAS drives, that power doesn’t come cheap, but the x4600 offers quite a bit of bang for the buck.

While unpacking the x4600, the first thing I noticed was the enormous fan arrays. Two sets of four large fans sit right in the front of the case. The only cooling fans in the entire chassis, they push air directly over the vertically mounted CPU modules, and they are surprisingly quiet during normal operation. Behind these modules are six half-height PCI-E and two PCI-X expansion slots with plenty of elbow room. The almost complete absence of ribbon cables was also surprising. 

Hardware error conditions in the x4600 are handled with aplomb. Should a DIMM fail on any of the processor modules, the release handles light up. Further, pulling a CPU module out of the server and tapping a button will cause the socket containing the bad DIMM to light up using power from an on-board capacitor. All these features lead me to the conclusion that this is one of the best-designed server chassis I’ve ever seen.

Powering the x4600 are four 850-watt power supplies operating in a 2+2 redundant configuration. Although the system wouldn’t power up with only two functional power supplies present, it would continue to run following the loss of the same two power supplies. This adds up to a lot of power consumption, but not as much as I expected given the server’s overall performance and abundance of processors.

49TCsunfire_ph.gif
Click for larger view.

Whither RAID 5?

The x4600 sports four Gigabit Ethernet NICs, two USB2 ports in the rear and two up front, four hot-swap 2.5-inch SAS drive bays, and a vertically mounted DVD-ROM drive. The eight I/O slots in the rear enjoy bandwidth of more than 20GBps to the processor racks. Local disk is handled by an LSI Logic SAS RAID controller, which proved to be the only poor choice in an otherwise pristine piece of server engineering.

While performing initial OS installs on the x4600, I noticed that the system seemed slightly sluggish when the local disk I/O was taxed. Considering the 16 cores ticking over inside the box, the problem clearly wasn’t the CPUs, but the local disk controller.

The LSI Logic controller embedded on the mainboard is limited to RAID 0/1 only, despite the presence of four hot-swap drive bays, and this controller shares a 100MHz bus with the two PCI-X slots. This all adds up to roughly 55MBps sequential reads and 42MBps sequential writes to local disk, as measured by Bonnie++ running on Red Hat Enterprise Linux 4 with a 4GB file size.

That’s about 80 percent of the performance of an older HP DL585 with one Ultra320 36GB SCSI drive. Enabling write caching would provide a boost, but the controller isn’t battery-backed, so that isn’t really an option for a production server.

Sun’s point of view is that nearly all applications for the x4600 will involve external disk in the form of direct-attached SCSI or a Fibre Channel or iSCSI SAN, and the local disk will be used only for booting and swapfile. Sun is absolutely correct, but like it or not, local disk should perform well enough to keep the server’s head above water.

This issue aside, the x4600 has a wide range of I/O options available directly from Sun, which is handy because the I/O slots are half-height. From 2Gbps and 4Gbps Fibre Channel HBAs from QLogic and Emulex to multiport Gigabit Ethernet and InfiniBand cards, there’s plenty to choose from. No 10Gbps card is available yet, but Sun plans to release one before the end of 2006. The use of half-height cards in the x4600 doesn’t seem to make sense at first, because there’s plenty of room in the case, but Sun stuck with half-height cards to remain consistent among the x4100, x4200, and x4600 servers.

Virtuoso of virtualization

The x4600’s plentiful processing and I/O resources make it a tremendous platform for virtualization, and sure enough, the server is on the VMware HCL (Hardware Compatibility List). I’ve been running VMware Infrastructure 3 on it for a week in the lab not only problem-free, but the server barely breaks a sweat while running eight two-processor Windows and Linux VMs under normal load.

It’s always hard to gauge actual VM host loads, because there’s so much fluctuation in the loads on the VMs themselves, but I would estimate that a maxed-out x4600 would handle 32 or more medium- to large-workload virtual servers with no complaints. For many shops, two of these servers would represent a datacenter’s worth of computing power -- and be fully redundant. This makes the power consumption not even worth mentioning.

The ILOM (Integrated Lights-Out Manager) card present in the x4600 conforms to Sun’s new Web UI standards, and it’s quick and easy to use. The Java-based remote console application is extremely responsive and among the best in the industry. I did most of the OS builds on this server via remote console sessions with little trouble, although some OSes won’t install properly from a remote-mounted ISO image. This is because the image mounts as a secondary CD-ROM device, not the primary, preventing some installers from locating the proper device.

Another note is that the onboard LSI Logic controller is not recognized by the Windows installer, requiring a driver disk in floppy format. As there is no floppy drive on the x4600, this floppy could be remotely mounted from a floppy image, or you could use a USB floppy drive; either way, it’s a bit of a pain. Once installed, however, Windows Server 2003 runs like a champ. Red Hat Enterprise Linux 4 Update 4 installed via Preboot Execution Environment with no problems.

Overall, the Sun Fire x4600 is a superb server with an obvious focus on virtualization, HPC, and database applications. If the local RAID controller supported RAID5, and the local disk I/O had more headroom, this server would be nearly perfect.

InfoWorld Scorecard
Management (15.0%)
Scalability (20.0%)
Value (10.0%)
Serviceability (10.0%)
Availability (25.0%)
Performance (20.0%)
Overall Score (100%)
Sun Fire x4600 M2 9.0 10.0 8.0 9.0 9.0 8.0 8.9
Recommended
Join the discussion
Be the first to comment on this article. Our Commenting Policies