Last week I wrote about some of the basics of designing a network for use with IP storage. While building in an appropriate level of redundancy and properly configuring VLANs and Spanning Tree are critical, implementing those design fundamentals barely begins to scratch the surface of the work necessary to build an exemplary IP storage infrastructure. After you've set the foundation, the next step is to configure your servers and storage to make use of it.
That typically involves determining how your servers and storage will leverage the redundant network you've deployed -- both in terms of offering redundancy and additional throughput. While you certainly can use a single NIC off each server and a single gigabit interface on the storage device, doing so will dramatically limit throughput potential and won't leverage the redundancy offered by a dual-switch architecture.
It's clear you want to use at least two dedicated storage network interfaces on each server that needs to attach to the storage and at least two -- if not more -- interfaces on the storage itself. That seems simple enough, but there are a lot of details to consider when configuring storage path redundancy and multipath throughput. To make matters worse, the best approach will vary wildly depending on the storage protocol, the specific storage hardware, and even the virtualization stack or server OS you're using.