Put your storage backups on a diet

What are the odds that two rival data-protection companies would release new products just days apart? Very slim, in my view, but that's just what Avamar and Data Domain, two successful promoters of data-reduction technology, did earlier this month.

First let me clarify what I mean by data-reduction technology -- also known as COS (capacity optimized storage) or, when applied to data transfers across a WAN link, COT (capacity optimized transport).

In essence, COS consolidates fragments having the same content, making it possible to shrink the space needed to store a file to a bare minimum. The size of the fragments and the algorithms used for the optimization vary, but the common process for all COS solutions is to divide data into similar-size chunks and look for repetitive patterns.

Imagine applying this technique to data backups, which is what Avamar and Data Domain do intuitively. Using COS creates backup files somewhat smaller than the original because of the space saved by eliminating equal chunks. That saved space can be significant; but there's more, as COS has another rule: Identical chunks are backed up only once. Therefore, backup cycles following the first one will have to copy only a small fraction of the original content, because most files change very little of their content over time.

Just think of how many similar patterns you may have in your datacenter (such as OS files from multiple servers), and it quickly becomes clear that COS-empowered backup applications can achieve tremendous compression rates -- of 10:1 or larger.

The compression rate translates into leaner, faster backup cycles, but COS also benefits data exchanges between remote offices, with faster transfers and significant telecom cost savings. Not surprisingly, both vendors offer a replication application to complement their main products.

Perhaps the most important effect of COS' unprecedented compression ratio is to make disk-based backups cost competitive with tapes, essentially shrinking the cost/capacity ratio differences between these two media to nearly nothing.

The two vendors developed their COS products using a slightly different approach. Data Domain proposes appliances as target devices for backups, whereas Avamar offers its Axion application stand-alone or bundled with an appliance. Perhaps the most significant difference between the two is that Data Domain integrates with major backup applications, whereas Axion replaces them.

"The DD460 is a disk full system packaged as an NAS target for backup software," says Frank Slootman, president and CEO of Data Domain. "Inside that box is our secret sauce that gives different ways to find redundant patterns and store them uniquely."

As for Avamar, "you can buy our hardware or we give you a bill of material saying 'that's what we tested for good performance,'" explains Ed Walsh, CEO of Avamar. "But in general, [Axion] works on any Red Hat Linux box."

Avamar's new Version 3.5 of Axion has some interesting new features, such as a central management console for multiple systems and expanded support for databases, OSes, and Exchange servers. Axion 3.5 also encrypts data, but the new feature I find most interesting is the file system view that makes all the files in the Axion pool of backups easily accessible to just about any application.