As with any SAN, building one with Xsan is a project measured in days, not hours. The easy part is installing FC (Fibre Channel) cards in the devices that will directly connect to the SAN; connecting them to an appropriate switch; setting up the back-end Ethernet network that will manage the SAN (leaving the FC network free for data); and, finally, installing Xsan on the direct-connect machines. At least one Xsan machine will serve as an MDC (Metadata Controller). This is the heart of the SAN, managing access to LUNs (logical unit numbers), and controlling the flow of data across the network. Of course, two MDCs is a minimum for any production environment, and from my experience, I’d want a third MDC. Although the documentation wasn’t initially clear on this, backup MDCs should be set to “low” priority for fail-over, leaving the “medium” setting unused.
The hard part is reformatting all the storage that one will connect to the SAN. One simply can’t take an Xserve RAID out of the box, connect it to the SAN, and expect to use it right away. This will take a day or more, depending on how much block storage is involved and how it’s sliced up into LUNs -- Apple’s factory default of two RAID0 arrays for a fully populated chassis isn’t everyone’s preference.
For my tests, I used an Xserve G5 with 2GB of RAM as my primary MDC, and a late-model Xserve G4 as a backup. My clients used Mac OS X, Red Hat Linux Enterprise AS 3.0, and Windows Server 2003. The Linux and Windows machines were HP ProLiant DL360 G3s, with dual processors and 4GB of RAM in each, running StorNext FS 2.5. The SAN was based on a Qlogic SANbox 5200, while the back-end network was set up on a Cisco Catalyst 4003.
Although some reviews have suggested using Gigabit Ethernet, the controllers on Apple’s Xserve RAID boxes only support Fast Ethernet connections and my switch was never stressed under the slower topology. Apple’s engineers now recommend -- in a document published after my testing was completed -- maintaining separate SAN back-end and RAID management networks.
I’m not surprised that the best practices for Xsan are still emerging; that’s what 1.x versions are all about. In fact, it’s encouraging that the body of knowledge is maturing as rapidly as it has so far.