Either way, it's cleaner to deploy server aggregation switches in each rack and run only a few fiber links back to the core than try to shoehorn everything into a few huge switches. In addition, using server aggregation switches will allow redundant connections to redundant cores, which will eliminate the possibility of losing server communications in the event of a core switch failure. If you can afford it and your layout permits it, use server aggregation switches.
Regardless of the physical layout method, the core switches need to be redundant in every possible way: redundant power, redundant interconnections, and redundant routing protocols. Ideally, they should have redundant control modules as well, but you can make do without them if you can't afford them.
Core switches will be responsible for switching nearly every packet in the infrastructure, so they need to be balanced accordingly. It's a good idea to make ample use of HSRP (Hot Standby Routing Protocol) or VRRP (Virtual Routing Redundancy Protocol). These allow two discrete switches to effectively share a single IP and MAC address, which is used as the default route for a VLAN. In the event that one core fails, those VLANs will still be accessible.
Finally, proper use of STP (Spanning-Tree Protocol) is essential to proper network operation. A full discussion of these two technologies is beyond the scope of this guide, but correct configuration of these two elements will have a significant effect on the resiliency and proper operation of any Layer-3 switched network.
Minding the storage
Once the core has been built, you can take on storage networking. Although other technologies are available, when you link servers to storage arrays, your practical choice will probably boil down to a familiar one: Fibre Channel or iSCSI?
Fibre Channel is generally faster and delivers lower latency than iSCSI, but it's not truly necessary for most applications. Fibre Channel requires specific FC switches and costly FC HBAs in each server -- ideally two for redundancy -- while iSCSI can perform quite well with standard gigabit copper ports. If you have transaction-oriented applications such as large databases with thousands of users, you can probably choose iSCSI without affecting performance and save a bundle.
Fibre Channel networks are unrelated to the rest of the network. They exist all on their own, linked only to the main network via management links that do not carry any transactional traffic. iSCSI networks can be built using the same Ethernet switches that handle normal network traffic -- although iSCSI networks should be confined into their own VLAN at the least, and possibly built on a specific set of Ethernet switches that separate this traffic for performance reasons.
Make sure to choose the switches used for an iSCSI storage network carefully. Some vendors sell switches that perform well with a normal network load but bog down with iSCSI traffic due to the internal structure of the switch itself. Generally, if a switch claims to be "enhanced for iSCSI," it will perform well with an iSCSI load.
Either way, your storage network should mirror the main network and be as redundant as possible: redundant switches and redundant links from the servers (whether FC HBAs, standard Ethernet ports, or iSCSI accelerators). Servers do not appreciate having their storage suddenly disappear, so redundancy here is at least as important as it is for the network at large.