3 challenges of high-performance storage in the age of unstructured data explosion

High-performance flash storage used for unstructured file data necessitates modern solutions

data explosion
Thinkstock

The need for high-performance storage solutions tailored to meet the demands of unstructured file and object data is greater than ever. Because more modern applications use unstructured file data, the growth of unstructured data (29.8 percent) outpaces that of structured data (19.6 percent), according to IDC. Enterprises need high-performance solutions to keep up with this digital transformation.

With new innovations in flash storage, there now lies an opportunity to make storage of unstructured file data faster and better. All-flash array solutions for file and object storage, such as Pure Storage’s FlashBlade, offer enterprises a high-performance solution that works with their unstructured file data.

Using flash for large, unstructured file data means higher performance without the latency that comes with scale-out solutions. In addition, the dense storage of flash means a much smaller footprint than disk or tape, which in turn translates into savings on power, rack, and cooling. While the cost of flash is still much higher than disk, flash is becoming more affordable.

However, significant challenges lie in using flash storage for unstructured file data, necessitating a modern secondary storage solution that works with high-performance flash storage.

Challenge No. 1: File services and data management

Flash vendors have not yet built in the file services that enterprises need to manage their data. This includes data management services such as data protection and backup, data movement, search and discovery, and learning.

This means that high-performance flash storage needs a complementary secondary storage solution with the file services enterprises need to manage large amounts of data. For example, my company, Igneous Systems, offers a secondary storage solution with file services that enable enterprises to easily protect and back up petabytes of data, move data from where it lives to where it’s needed, quickly search through their data, and derive insights from their data.

Challenge No. 2: Legacy solutions can’t keep up with flash

Using flash for unstructured file data storage requires a compatible secondary storage solution, but unfortunately, legacy backup and archive solutions cannot keep up with the speed of flash storage.

A modern secondary storage solution that complements high-performance storage needs to be able to handle parallel streams of data. Igneous uses parallelism to move data as quickly as it can be read off Pure FlashBlade.

Challenge No. 3: Higher price point

Although flash is becoming more affordable, high performance does come at a steeper price. With large amounts of data, it’s difficult to justify the cost of high-performance flash.

But enterprises typically only need to store a fraction of their data in high-performance primary storage. It makes sense economically to tier data that is infrequently accessed in a lower-cost tier optimized for long-term retention and capacity rather than performance.

When you put the performance benefits of flash into the context of a larger strategy to optimize workloads, high performance plus huge capacity is both economically feasible and valuable to your overall data management system.

Conclusion

Using a modern capacity tier to complement high-performance flash storage enables enterprises to harness the power of high-performance storage where they really need it, while equipping them with the file services necessary to manage all of their data more effectively.

High performance and huge capacity for unstructured file data is necessary in this age of unstructured data explosion. With recent advances in both primary and secondary storage solutions, enterprises now have the solutions they need to upgrade their infrastructure as part of a forward-thinking data management strategy.

This article is published as part of the IDG Contributor Network. Want to Join?