Over the past 25 years, mobile phones have been completely rethought, thanks to Moore’s Law and the advent of the World-wide Web, touchscreens, and so on. Ditto most everything in the data center, with servers and networks getting roughly 1000X faster (thanks again, Dr. Moore) and seismic shifts like virtualization and network switching.
But then there’s storage. Most enterprise storage solutions in use today were designed 20+ years ago. Storage has been stagnant because of mechanical disk. Hard drives do not follow Moore’s Law – yes, they have gotten denser, but not faster. As a result, there has been little impetus to innovate. Until now.
Disruption #1: Flash memory
Over the past decade, flash memory has redefined the consumer technology experience. It is the storage inside your smart phone and is used heavily in modern data centers like those of Google, Apple and Facebook. It is only a matter of time before flash will supplant hard drives for all hot and warm data.
Of course, flash has been used for years as a cache or tier to accelerate disk storage. However, the performance disparity between flash and disk is so great that hybrids of flash and disk perform like disk as the “long” disk operations dominate. To use an analogy, the difference is like that between traveling internationally by jet or ship, and it is pretty hard to plan a business trip if you and your colleagues don’t know which one each will get. (Perhaps this is one of the reasons why next-gen hybrid solutions have thus far not done as well as hoped in the enterprise?)
The reason of course that flash was originally limited to a cache/tier was that it was expensive. Back in 2011, the price of consumer-grade multi-level cell (cMLC) flash was about four times higher than the price of a fast hard drive (15K rpm). Today, cMLC costs less than fast disk, the same fast disk that underpins the $24B performance-optimized (a.k.a. “Tier 1”) storage and related software market.
Disruption #2: The cloud
Cloud is a more nebulous concept than all-flash (pun intended), but it is having a similarly disruptive impact on the storage industry. Amazon Web Services has simply reset the bar for IT by making it dramatically easier for developers to deploy and scale applications. As a result, all data centers aspire to be cloud-like – to meld together commodity hardware with software and automation that delivers agility, elasticity, resiliency, security and most of all, simplicity.
Generally, the term public cloud is used to refer to Infrastructure/Platform as a Service (IaaS/PaaS) offerings like AWS, Microsoft Azure and Google Compute Engine. However, like most, we also include multi-tenant Software as a Service providers like Salesforce and Netsuite, as well as consumer Internet giants like Apple and Facebook (B2C SaaS).
So what will the storage industry look like in the next few years? Expect even more major shifts in the storage industry going forward. It is high time, for at least the performance storage market, to join servers and networks on a Moore’s Law curve. At the same time, storage must shed the complexity, consulting overhead and unfriendly business practices that have helped make AWS so appealing. Alone, either the transition to flash or the cloud would be profoundly disruptive. Taken together, none of the storage solutions designed for mechanical disk and the traditional data center will make the leap to the solid-state cloud. Now the competition is on to see which storage solutions will deliver the most compelling business value in all-flash.
