Do we need to rethink storage?

sphinx99

[H]ard|Gawd
Joined
Dec 23, 2006
Messages
1,059
This is a topic that's been on my mind ever since I began seriously working with solid state storage.

Traditionally, storage existed solely for data retention. If you wanted performance, you either multiplexed your storage or used tricks (cache, compression, whatever) to get what you needed. Controller cards, interface technologies were rarely insurmountable bottlenecks.

Now, things are changing. SSD is obsoleting several links in the chain:

- drive interfaces. SATA2 has gone from plenty to bottleneck in 12 months flat. SATA3? I doubt that it (and dual-port SAS, FC, etc) have that much headroom left given the progression of SSD storage.

- controllers. Controllers didn't need to be powerful. You could pay four figures for a LSI or Areca controller, plug in a bunch of disk, and sit back and wish you had faster disk. In comes SSD: now you wish those controllers had 4x the compute power and far better active cooling.

- storage chassis interfacing. Suddenly, 4 and 8 gbit multipath fiber isn't looking like the 5+ year shelflife storage interface fabric that it used to look like.

I am finding, in my professional work, that a few concepts aren't obsoleted so much as redefined:

- cache. Generally, I used to cache to reduce access time hit. With SSD, those days are gone. Now, cache exists to reduce sequential throughput hit. Who would have thought?!

- operating system. The OS and in particular tuning of the I/O scheduler and basic multipathing support are suddenly the de facto determining factors of storage performance. If the OS does not treat disk I/o with the same efficiency that it treats memory operations, it starts to fall behind with SSD.

- irrelevance of file system. Oh, a decade of using lightweight or raw filesystems to tweak every ounce of performance out of a storage system primarily by minimizing access time as much as possible.... all rendered unnecessary. The I/O scheduler matters but the filesystem? Not so much.

So, I find myself starting to think of solid state storage as on par with GPUs and multicore processers in terms of the paradigm shift it likely references. Projecting forward, I also to question the role of persistant storage in computing. With access time dramatically improved but throughput still far behind, will we see a rebalancing between what system RAM does, and what SSD does? Are we approaching a point where the very notion of a "drive" is obsoleted? When the performance characteristics of disconnected (e.g. disk) and connected (e.g. RAM DIMM) start to converge, does the very notion of a storage interface come into question?

I thought this might be an interesting topic for discussion and I'd love to hear others' thoughts on this. The approach I think should be taken is not of what to do in 2009 but where we think things are headed over a 10 year horizon. In 2009, this technology is still problematic but 2020 I suspect that the interface between "compute" and "storage" will look nothing like it does today.
 
Well, if memristors become as fast as RAM (a potential last I heard) Volatile Storage as we know it may cease to exist. RAM and HDD would converge. Allowing us instant boot ups regardless of how complex the OS is.
 
Well, if memristors become as fast as RAM (a potential last I heard) Volatile Storage as we know it may cease to exist. RAM and HDD would converge. Allowing us instant boot ups regardless of how complex the OS is.

If and only if the prices come down. For the amount of storage they have for the price, SSD are enthusiast and company only.
 
Back
Top