InfoWorld review: Data deduplication appliances

Ever wonder why hard drive capacities continue to get bigger? Do you think IT has ever told management that they will need less storage capacity over the next three years? In fact, three years from now, your company will likely have four times as much data to store as it's storing today. The gigabytes will continue to turn into terabytes, and the terabytes will soon give way to petabytes.

Fortunately, there is a way to slow the inevitable data sprawl: Use data deduplication on your storage system. Data deduplication is the process of analyzing blocks or segments of data on a storage medium and finding duplicate patterns. By removing the duplicate patterns and replacing them with much smaller placeholders, overall storage needs can be greatly reduced. This becomes very important when IT has to plan for backup and disaster recovery needs or when simply determining online storage requirements for the coming year. If admins can increase storage usage 20, 40, or 60 percent by removing duplicate data, that allows current storage investments to go that much further.

[ How does data deduplication work? What's the difference between block-level and file-level approaches? How to choose among source, target, and inline methods? Find out by downloading InfoWorld's Data Deduplication Deep Dive Report. ]

To see what data deduplication can do, I reviewed four storage appliances that use the technology: the FalconStor FDS 304, the NetApp FAS2040, and the Spectra Logic nTier v80 and nTier vX. All four appliances provided excellent scalability, performance, and data deduplication functionality. Each solution has a bit of its own personality -- one looks like a rack of tape drives, another a large network-attached storage system, and a third a direct-connect Fibre Channel appliance.

FalconStor's FDS 304 is a 2U NAS (network attached storage) appliance utilizing SATA hard drives and gigabit and 10-gigabit Ethernet network interfaces. IT will typically deploy the FDS 304 as a disk-to-disk backup partner or as a target for disk-based backups, but it can also serve as main-line storage. NetApp's FAS2040, also in a 2U form factor, can be deployed as a Gigabit NAS, Fibre Channel, or IP SAN, as well as a Fibre Channel over Ethernet device. It too is going to see action as a target for disk-based backups and data replication; it can also be used as a general-purpose storage medium. For enterprises that have a large investment in physical tape libraries or that will be virtualizing their tape farms, Spectra Logic's nTier family is a great choice. A "drop in" virtual tape library (VTL) appliance that uses FalconStor's data deduplication engine, the nTier can replace physical tape systems or run in parallel with a physical tape library while deduplicating stored data.

All of these appliances offer an easy-to-implement, easy-to-manage, and effective data deduplication system that any enterprise network could take advantage of. Based on my tests with a highly duplicative set of Windows and Office files and their backups, you can expect similar levels of data deduplication from all of them. Note: If you plan to deduplicate system backup sets as well as raw files, you'll want to make sure that the deduplication engine works with your backup software.