When it comes to the data storage industry: is there any scope left for real innovation?

Federica Monsone, founder and principal, A3 Communications, takes a look at the state of data storage today, examines if there is any room for truly ground-breaking innovation, and explores how current and future breakthroughs with storage technologies will affect the industry and its organizations.

Over the past nine decades, the storage industry has brought to market numerous ground-breaking technologies and architectures, many of which are still in use today, years after they first appeared on the market. Every day, vendors announce new products that seemingly offer something different from the myriad of available competing solutions; but are these simple updates to existing technologies, with a few tweaks and additions? Or are they truly original? At a time when so much has already been developed, can data storage vendors still be innovative?

To find out, let’s go back to the very start of the storage industry as we know it today. It was nearly a century ago, in 1928, that Austrian scientist Fritz Pfleumer patented the first magnetic tape. Since then, tape has played a significant role in the storage of data. NAKIVO’s director of product management, Veniamin Simonov, agrees: “There have been numerous breakthroughs in the data storage industry since the times of magnetic tape which, over the years, has helped to revolutionize how data is stored. Since then, we’ve seen the move to disk storage, which has included floppy disks, hard disk drives, optical disk drives, and so on.”

But how has storage technology evolved over time? Since the inception of data storage, the industry has developed different media, from tape to disk and from SSDs to flash, strived to increase areal densities, introduced architectures that aim to maximize data location and retrieval speeds. A lot more is on the horizon, according to Andrew Buss, research director, European enterprise, infrastructure and European edge strategies at IDC: “We’ve seen the move from tape to disk, and now the migration to flash is underway. We’re in the early stages of new approaches such as Intel Optane persistent memory, which allows high performance and highly durable storage media, as well as to add another level of caching for memory when added to the memory bus as DIMMs. In addition, new materials and lining technologies like graphene may well bring the humble spinning disk back to prime relevance once again for situations where real-time performance is not so critical. In the future there are likely to be many advances such as holographic storage or even DNA-based storage.”

Innovation begets innovation, tells us Antonio Barbalace, senior lecturer at the School of Informatics, University of Edinburgh: “With the introduction of flash-based storage devices, new hardware interfaces have been introduced, including the bespoke NVMe. NVMe is under continuous evolution, and several diverse variations to the interface/protocol have been introduced, such as OpenChannel, Zoned Namespaces and computational storage.”

Curtis Anderson, software architect at Panasas, takes this idea and develops it further: “If you describe tape, disk, flash, NVDIMMs, etc, as ‘base technologies’, then as each new type of base technology is added to the mix, the combination allows for a much wider range of possibilities. Where innovation truly happens, however, is in how the various base technologies are assembled into functional systems.”

Kam Eshghi, Chief Strategy Officer at Lightbits Labs agrees with this view: “We will continue on an iterative path until we make a leap – like from paper tape to magnetic tape, to disk drives, to solid state media and the next jump will be to some kind of technology like phase change or memristor – and finally to quantum storage. In between, there will be lots of iterative improvements in speed, density and lower cost.”

Has it therefore become harder and harder for vendors to come up with truly innovative products? Yes and no, according to Randy Kerns, senior strategist at Evaluator Group: “On the surface, it seems that way, but new innovations aren’t necessarily just a new technology but really the application of new technologies.”

David Trachy, senior director of emerging markets at Spectra Logic, believes that the current battle against attacks on data will have an impact on where vendors will focus their R&D efforts: “We are likely to see the greatest innovations in software, specifically around data protection, because of the increase in security attacks, such as ransomware.” Billions of dollars are still spent on R&D, justifying the thought that something brand new is around the corner. William Toll, head of product marketing at Acronis, says: “There is always a continuous string of new innovations in storage, much of it stemming from the pace of innovation in the “nano” technologies that reduce the size of physical storage medium. IBM, Microsoft, and others continue to discover and publish research in future technologies that are very different from what we have in the market today. For example, Microsoft’s project silica,” the first storage technology for the cloud conceived and constructed from scratch. Reliant on ultrafast laser optics to store data in quartz glass, this technology is billed to potentially lead to a complete re-think of traditional storage system design.

“There have been some real significant storage product innovations in recent years,” according to Paul Speciale, chief product officer at Scality: “Some examples are new solutions for data backup with deduplication to eliminate backing up the same data redundantly many times. New hyperconverged solutions have simplified deployment and efficiency, and object storage has enabled massive scalability for billions of files. All of these were driven by changing customer needs, such as increased data growth rates, access by more users via the cloud, and the need to be more efficient,” he added.

End user requirements are indeed a significant driver behind data storage innovation. Industry veteran Alex McDonald, chair of a number of technical groups at SNIA EMEA and USA, agrees: “Necessity is the mother of invention. Users are very needy, and the storage industry has been hugely inventive. Since everyone wants to store their stuff faster, cheaper, and bigger, I think the industry has delivered on those criteria. External demands have driven innovation over the years, finding applications for Pfleumer’s tapes to spinning disks of iron oxide all the way to persistent memories made of glass”.

Software-defined storage (SDS), where hardware and software are uncoupled, is one stand-out innovation that has helped the industry enormously, for example by allowing greater administrator productivity thanks to its ability to leverage automation to adapt to changing requirements, and by giving end users the flexibility to select controller software and hardware from different vendors. Adoption rates of cloud storage enabled the industry overall to grow by bringing costly technologies within reach of budget-conscious organizations.

Interestingly, ESG’s senior analyst Scott Sinclair disagrees with the notion that traditional features of technology were recently revolutionized: “You don’t see a tonne of innovation in traditional features, such as snapshots, for example. But storage vendors are providing plenty of innovation in other areas such as combining artificial intelligence (whether integrated into the array or part of an external management platform) that collects telemetry data from the storage array to provide actionable insights or even automation to administrators. For example, leveraging this data and integrated intelligence the system can more efficiently scale workloads, to recommend actions to diagnose or remediate issues, or automatically optimize the storage resources when application requirements change. Vendors are also providing innovations to help their storage systems better support the needs of cloud-native or container-based workloads. Another area of innovation is offering a consumption-based model to procure storage resources, rather than just a CAPEX-based model. These are just some examples. The key takeaway is that there is a tonne of innovation going on, but not all of it might show up on a traditional data sheet.”

We are likely to see R&D around both new and current technologies in the future: “I think we’ll see both the development of existing technologies and ground-breaking innovations,” adds Steve Ashurst, European managing director at FalconStor.

The degree of innovation varies depending on where within the storage industry we look. When it comes to storage features for example, we have witnessed developments ranging from enhanced support of cloud-native or container-based application environments, to streamlining hybrid or multi-cloud storage management, incorporating intelligence into the storage system to simplify, or even automate IT operations.

Competition is another driver behind innovation. The storage market is full of different solutions to end users’ storage issues, with vendors competing to market the same cure to their storage woes. But, as with every industry, competition is healthy, and can lead vendors to push harder with product development to make it as marketable and user-friendly as it can be. FalconStor’s Ashurst explains: “We see existing vendors competing with emerging players to create something completely new or make improvements in order to capitalize on the new underlying trend.”

New trends, new challenges, are another clear driver behind data storage innovation. Scality’s Speciale agrees: “New problems require new, innovative solutions. We see this happening now as the technology landscape is changing again during this shift to cloud-native. This will once again spark a broad range of innovations. We see this happening again now as the technology landscape is changing again during this shift to cloud-native. This will once again spark a broad range of innovations.”

Despite an array (pun intended) of products launched on the market on a regular basis, most end users tend to be cautious when it comes to the adoption of new technologies and solutions, as Tim Klein, president and CEO of ATTO, points out: “Datacentres are generally more conservative in nature than early adopters in new storage technology and connectivity. As reliability is core to their mission, there certainly is a tendency to stick with the known technology and keep updating it to incrementally improve.” The lifecycle of IT products is also a factor in the adoption curve, as Speciale explains: “Most customers are not diligent about discovering what data is stored and on what system, and therefore tend to be lax and concerned about deleting data to reclaim space. This leads to bounded 3-5 year life spans for most solutions, especially on the hardware platform side. This, in turn, means that you will find a prevalence of solutions newer than 5 years within the data centre.”

To truly succeed, a new technology or architecture must also be cost-effective, as adding a new technology to hundreds or thousands of servers can be a costly exercise. “Fundamentally, especially at scale, it is not affordable to only use the latest and greatest technology,” tells us Lightbits Labs’ Eshghi. “Most of the capacity in datacentres and the cloud is still based on spinning disk drives. Spend is different though – because (for example) flash drives are far more expensive than spinning drives, there is more spent on flash drives than spinning even though the capacity of flash sold is less than spinning drives.”

What about vendors who do not innovate quickly enough? Well, they run the risk of becoming irrelevant, and therefore customers will look elsewhere. Lack of innovation may also damage vendor brands as these will unlikely be seen as market leaders. The result is the company going out of business or getting acquired. “I think that the majority of technology is dead after ten years, as are the manufacturers aligned to it unless they manage to evolve,” says FalconStor’s Ashurst.

READ MORE:

• Dot the I’s and cross the T’s: data management requirements for the legal sector
• Hackers target home Wi-Fi routers to steal data
• Data security and compliance: why prevention is better than cure
• 4 easy tips for to ensure data protection

Innovation is a necessity for most vendors looking to maintain a strong position in the market, but the degree to which the vendor can innovate is crucial. Being able to disrupt the market with a new product or service allows a company to cut through the noise and become an industry leader, instead of following the same technology lines as everyone else. Data storage may have been around for many years, but there still is room for vendors to innovate and surprise end users with ground-breaking products. This may be because they leveraged existing technologies or architectures, or possibly because they developed a brand-new solution. Either way, it seems that there will always be something new and shiny on the data storage horizon for the industry to explore.

For more news from Top Business Tech, don’t forget to subscribe to our daily bulletin!

Follow us on LinkedIn and Twitter