As an IT consultant, I see it happening all around me: the heroic struggle to keep up with the exponential growth of data. IT finds itself confronted with larger and more diverse data sets than ever before, thanks to high-definition input devices, low-cost storage hardware, and the blazing processing power of today's servers and workstations. Even small enterprises can now afford to store mountains of data undreamed of only a few years ago.
At InfoWorld we've come to call this phenomenon the enterprise data explosion, and chances are, you're experiencing it right now.
[ For an entertaining look at a key storage management technology, fire up this InfoClip on storage virtualization. ]
Just five years ago, your average departmental file server had maybe 75GB to 100GB of primary storage, usually in the form of direct-attached disk. Today, that same server is much more likely to have -- and need -- five to ten times that amount. There's also a good chance this chunk of storage will come in the form of a NAS or SAN instead of a meat-and-potatoes server. Prices per gigabyte may have dropped precipitously, but the demand for capacity has escalated faster, raising storage hardware investments to new heights.
The less obvious cost of managing and protecting all that additional data is even higher. As more and more disk spreads throughout the enterprise, it becomes more and more difficult to monitor how that storage is being used and to ensure appropriate protection against data loss. Few IT shops I've seen these days are increasing their head counts significantly, so something has to give.
Yet there are no magic technology bullets when it comes to rampant data growth. Instead, you'll find a grab bag of useful of technologies for specific purposes -- plus a set of best practices from which to construct a storage strategy that anticipates constant expansion. Sure, hot technologies such as data de-duplication should play a role in your strategy, but they don't qualify as strategies in and of themselves. And failure to think through a plan at a high level is where many go wrong.
A good storage strategy almost always involves a combination of hardware, software, tools, and perhaps most importantly, policies. Together, these ensure that the correct data is being retained, stored on the most appropriate type of storage hardware, adequately backed up, and able to be restored in an acceptable amount of time. Assembling and fine-tuning that collection of resources is not always easy, but it's far better than being drowned alive in corporate data run amok.