disruptive technology. Large vendors made false promises about their products that have generated a lot of fear, uncertainty and doubt, but today, there is significant progress in major areas in the software-defined storage products area.
The benefits of SDS
Software-defined storage (SDS) is a way to separate the greater part of storage software from the underlying storage appliances. Within a public cloud, automation software provisions virtual instances from physical servers. With SDS, these data services run in VMs or containers, allowing automated scaling on demand, robust failure recovery and bandwidth that can service large workloads.
Eventually, standardized API call structures that can chain services together will create much more powerful storage functionalities. The quid pro quo for this marked evolution stems from the commoditization of underlying hardware, while storage overall will do more for the IT department and do it automatically.
SDS covers a broad spectrum of both hardware and software products. The large system vendors each offer preconfigured appliances. This is a way to limit complexity and boost stability, but the market is beyond the early rollout phase. Independent software vendors, such as Nutanix and Scality, are beginning to unbundle their code from the hardware.
SDS aims to automatically mount different hardware platforms, including appliances and drives, to any cluster. This is essential future-proofing in an industry where storage form factors change rapidly. With tight industry standards, such as nonvolatile memory express (NVMe) and Serial Advance Technology Attachment, drives are essentially interchangeable, provided the SDS code knows what to do when a drive is full.
Software-defined storage products on the market
Now, the question is: build or buy? Numerous vendors offer high-quality hardware for organizations that want to build their own SDS appliances. Super Micro delivers vast quantities of gear to the three giant cloud service providers. This hardware is surprisingly inexpensive, especially when it comes to drive purchases. All but the largest IT shops can obtain this gear from distributors or even directly online.
There is a spectrum of alternatives for hardware purchases. You can get bare-bones units or fully configured nodes with drives. At the top end of the price and performance spectrum is NVMe over Ethernet (NoE) clusters. With these products, the nodes at remote direct memory access speeds can reach all of the drives in the cluster. These will undoubtedly be the fastest and the most flexible options.
But software-defined storage products aren’t constrained to rocket-speed configurations. Secondary storage, also called cold storage, is an option, but pay attention to the cost per terabyte. At first glance, this favors hard drive storage, but solid-state drives (SSDs) have so much bandwidth that you can do on-the-fly compression and deduplication, which often multiplies the effective storage capacity by a factor of five times or more. When it comes to cost per terabyte, SSD always wins.
You can also build configurations with fewer nodes. For example, a 14 TB hard disk drive is 3.5 inches wide, while a 32 TB SSD is 2.5 inches. This makes multi-petabyte 1U or 2U nodes very feasible.
SDS covers a huge spectrum of software offerings, from hyper-converged infrastructure (HCI) or virtual storage area network stacks to new file system approaches and object stores. Soon, we’ll see specific services for point application to data flows, too.
Don’t ignore the option to buy prebuilt appliances. If you are thin in bare-metal integration skills, it makes sense to use third-party integrators that may be the original equipment suppliers. Ultimately, buying from the large established vendors is a low-risk approach, but it will come at a price.
With any purchasing decision, avoid vendor lock-in — especially for drives — since there can be significant markup when compared to distribution pricing. Ask the vendor upfront whether they allow you to add third-party commercial off-the-shelf products to the appliance, especially for drives and memory. However, don’t be surprised if the story varies even within a single company. To get a sense of the issues, read internet blogs on the topic.
How to choose software-defined storage products
Some of these products are more mature than others. An early adopter may wonder: Should I go with the best promise, the best price or the most mature solution? The answer comes down to confidence in the vendor and their design and qualification processes. For example, Nutanix and Scality are mature companies with a track record, while Red Hat backs Ceph, which has just moved to a containerized SDS model.
If you are just getting started but have a production cluster to deliver, these tried-and-true vendors are a good place to begin. If, on the other hand, the objective is to determine the strategic benefit of an all-out SDS approach, try the new, but smaller vendors entering the game. There are quite a few of these, especially in the HCI area. The NoE vendors are breaking new ground and make sense for prototyping.
You can apply the SDS approach in the cloud and treat networked storage as the hardware platform. This opens up a whole spectrum of hybrid cloud environments, but it’s very early to show good direction here.