Big data needs software-defined storage

With demands for agility and capacity, storage systems can't be islands. IBM's Ronald Riffe explains how software-defined storage provides a broad, hardware-independent solution

The movement to smarten up every aspect of IT infrastructure rolls ever onward, from servers to the network to storage. We've deployed server virtualization and built automation frameworks to adapt elastically to changing workloads; we've also begun rebuilding our networks with SDN. Software-defined storage (SDS) is the next big trend.

SDS enables agile and elastic storage through automated processes that can adapt to changing I/O demands. In this week's New Tech Forum, Ronald Riffe, IBM director of software-defined storage, walks us through what SDS means now and in the not-too-distant future. -- Paul Venezia

Software-defined storage: A must for big data

The proliferation of mobile devices and instrumented enterprise assets is igniting a new data explosion, from which can be gleaned new analytic insights and, in turn, open new business opportunities. At the same time, big data is placing greater demand on existing infrastructures, driving a need for instant access to resources -- compute, storage, and network -- and creating a new imperative to adopt cloud technologies. The flexibility required simply can't be obtained with a traditional hardware-centric approach.

Storage is a constant pain point for cloud deployments, but it has been largely ignored by IT organizations, which have focused their attention primarily on server and network virtualization. With capacity growth, application performance, and cloud-related issues challenging organizations, IT managers must improve storage efficiency with not just virtualization of their server infrastructure but also their storage environments.

According to a 2013 study conducted by EMEA research, storage provisioning and management is a significant bottleneck for 58 percent of enterprise cloud deployments. As a result, storage automation was identified as the top integration requirement for the initial release of cloud projects, as cited by 32 percent of organizations.

For those respondents who had already attempted to deploy a private cloud without an SDS (software-defined storage) infrastructure, an overwhelming 84 percent of them were planning some sort of hardware-independent storage virtualization system to support their cloud.

Abstracting storage services from underlying proprietary hardware through SDS can improve operational efficiency, provide transparent data mobility, and enable common-denominator management capabilities, regardless of the hardware used. SDS makes applications in the cloud more efficient by reducing management complexity and allowing them to scale inexpensively. A review of IDC's Worldwide Software-Based (Software-Defined) Storage Taxonomy, 2013 report along with a consensus of storage vendors indicates SDS has the following key attributes:

  1. Software is at the heart of SDS. It's designed to run on heterogeneous, commodity hardware and can even leverage an organization's existing storage infrastructure.
  2. SDS provides a full suite of storage services.
  3. SDS federates physical storage capacity from multiple locations like internal disks, flash systems, other external storage systems, and soon from the cloud and cloud object platforms.
  4. SDS is easily programmable through a single, unified API that is available through a variety of portals

With reduced complexity, SDS reduces burden for administration. It simplifies, virtualizes, and automates storage services. It improves existing storage utilization and eases data migration between storage systems and storage tiers -- even among vendors -- thus reducing service delivery times.

Here are a few characteristics that are important to have in any SDS infrastructure:

SDS should be open. SDS should support broad client choice in the physical storage infrastructure and integrate with other virtual compute and cloud management software. This openness helps organizations migrate to an agile, cloud-based storage environment and manage it effectively without having to replace existing storage systems, generating dramatically increased value from existing investments. It offers capacity-based storage virtualization and automation, allowing customers to deploy the capabilities needed without licensing complexity.

SDS should be self-optimizing, intelligent, and policy-driven. SDS should adapt automatically to workload changes to optimize application performance, eliminating most manual tuning efforts. Automated tiering across different storage systems and virtual machine vendors and brands can optimize storage by automatically moving the most active data to the fastest storage tier.

SDS needs to be application aware. SDS should automate provisioning and improve productivity so that IT operations administrators can focus on overall storage deployment and utilization, as well as on longer-term strategic requirements -- without being distracted by routine storage-provisioning requests. For example, the right storage solution would perform application-aware snapshot backups frequently throughout the day to reduce the risk of data loss.

1 2 Page 1
Page 1 of 2
How to choose a low-code development platform