The Future of Hyper-converged Infrastructure
The Art of “Storage-as-a-Service”
Most enterprise datacenters are today considering the hybrid cloud model for their future deployments. Agile and flexible, the model is expected to yield higher efficiencies than traditional setups, while allowing a datacenter to be sized to average, as opposed to peak, workloads.
In reality, achieving portability of apps between clouds and reacting rapidly to workload increases both run up against a data placement problem. The agility idea fails when data is in the wrong cloud when a burst is needed. This is exacerbated by the new containers approach, which can start up a new instance in a few milliseconds.
Data placement is in fact the most critical issue in hybrid cloud deployment. Pre-emptively providing data in the right cloud prior to firing up the instances that use it is the only way to assure adequate those expected efficiency gains.
A number of approaches have been tried, with varying success, but none are truly easy to implement and all require heavy manual intervention. Let’s look at some of these approaches:
- Sharding the dataset – By identifying the hottest segment of the dataset (e.g. Names beginning with S), this approach places a snapshot of those files in the public cloud and periodically updates it. When a cloudburst is needed, locks for any files being changed are passed over to the public cloud and the in-house versions of the files are blocked from updating. The public cloud files are then updated and the locks cleared.
Storage continues to be a volatile segment of IT. Hot areas trending in the news this month include NVMe over Fibre Channel, which is being hyped heavily now that the Broadcom acquisition of Brocade is a done deal. Another hot segment is the hyper-converged space, complimented by activity in software-defined storage from several vendors.
Flash is now running ahead of enterprise hard drives in the market, contributing to foundry changeovers to 3D NAND to temporarily put upward pressure on SSD pricing. High-performance storage solutions built on COTS platforms have been announced, too, which will create more pressure to reduce appliance prices.
Let’s cover these topics and more in detail:
- NVMe over Fibre-Channel is in full hype mode right now. This solution is a major step away from traditional FC insofar as it no longer encapsulates the SCSI block-IO protocol. Instead, it uses a now-standard direct-memory access approach to reduce overhead and speed up performance significantly.
The storage industry continues to evolve rapidly, which is both exciting and challenging. I intend this blog to look at the hot news in the industry, as well as taking a view of trends and occasionally long-term directions.
This promises to be an interesting effort. There are plenty of innovations to describe, while retakes on older ideas crop up quite often. I hope you will find the subject as fascinating as I do.
1.It’s clear that the high performance enterprise hard drive is a dying breed. SSDs and all-flash arrays have undercut demand. With improved wear life, flash-based products meet the stringent needs of the datacenter plus, they are cooler, quieter and smaller and of course they are much faster.
Relevant news on this includes:
Storage in IT comes in multiple flavors. We have super-fast NVDIMMs, fast and slow SSDs and snail-paced hard drives. Add in the complexities of networking versus local connection and price, and capacity, and figuring the optimum configuration is no fun. Economics and performance goals guarantee that any enterprise configuration will be a hybrid of several storage types.
Enter auto-tiering. This is a deceptively simple concept. Auto-tiering moves data back and forth between the layers of storage, running in the background. This should keep the hottest data on the most accessible tier of storage, while relegating old, cold data to the most distant layer of storage.
A simplistic approach isn’t quite good enough, unfortunately. Computers think in microseconds, while job queues often have a daily or weekly cycle. Data that the computer thinks is cold may suddenly get hotter than Hades when that job hits the system. Similarly, admins know that certain files are created, stored and never seen again.
This layer of user knowledge is handled by incorporating a policy engine into auto-tiering, allowing an admin to anticipate data needs and promote data through the tiers in advance of need.
Webster defines the word Virtual as “Very close to being something without actually being it.” By that definition, a Virtual SSD is very close to being an SSD without actually being an SSD. From a more practical viewpoint, a Virtual SSD provides the benefits of an SSD, performance, without the downside of an SSD, price and limited capacity.
A virtual SSD blends 2 or more classes of storage media into a single volume with the performance characteristics of the fastest storage media and the capacity of the combined media. The flexibility of a virtual SSD can be tailored to meet a wide variety of needs. Virtual SSDs are:
- Cost effective because you can blend SSDs with cost effective hard drives
- Uncompromising because you can blend high performing NVMe drives with capacity SAS/SATA SSDs for an all flash virtual SSD
- High value because they offer both best in class cost/IO metrics as well as cost/GB
If you could have all flash performance at 80% of the cost of an all-flash solution, would you be interested? From a financial point of view, it is hard to argue.
It is pretty well documented that for most applications, less than 5-10% of the data across a volume is active at any point in time. Given this, it does not make sense to pay all flash prices to store your inactive data. Granted, there are applications that are active across the entire LBA range of a volume, or that cannot tolerate any unpredictable latency. For these applications, by all means, an all-flash solution is the right choice.
For the majority of us, a hybrid solution is the optimal choice. Keep in mind that not all hybrid solutions are created equal. For optimal performance, your hybrid solution should have the following characteristics:
- File pinning because there are applications for which you want to guarantee full flash performance for all IO
- Real time data promotion to performance storage because the solution should react to dynamic requirements, not yesterday’s activity
- Full automation because your time is too valuable to constantly manage storage
- Unrestricted fast tier capacities because you need the ability to size your storage to your unique needs
- Visibility into your storage behavior because you need information to guarantee your storage is optimized
ATTEND THE WEBINAR June 8th at 10:00 AM PDT
Register here: http://goo.gl/LzsgyI
Exponential growth of data from machines and users demands we scale storage infrastructure in real-time, but are you scaling intelligently? Are you optimizing effectively? Have you defined your strategy, your approach?
Great technology is linked to strong partnerships. Each partner brings his own unique expertise and value to the relationship, making the collective stronger than the singular. In that spirit, Enmotus is proud to partner with Intel and join them in the Intel Storage Builders program. Intel Storage Builders provides an open, collaborative environment for true storage innovation and advancement. The program brings together forward-looking companies, a broad Intel ecosystem, key foundational technologies, and strategic resources to empower our members to move storage forward.
When most people think of hybrid storage, they think about flash mixed with HDD, but you can also have an all flash hybrid solution as well. Hybrid All Flash makes sound business sense. It makes sense for the same reasons traditional hybrid solutions make sense. Hybrid all flash solutions provide the performance of high performance flash combined with the economics and capacity points of more economical flash storage.
IDC Corporation recently published a survey stating that 70% of big enterprises across Asia Pacific have or plan to deploy All Flash Arrays (AFA). This shouldn’t be surprising as enterprises have the budgets to deploy AFAs, which allows them to utilize these arrays to achieve operational efficiency, speed application performance and expand their businesses with sophisticated real time analytics and decision support systems. For those organizations that require all flash arrays, a hybrid all flash array will still provide the application performance but it could also improve operational efficiency even more.