Recent DDR SSDs

[bao__zhe]

I am planning to build my home storage system early next year. Currently I am researching for a ZFS ZIL disk and found 3 products: DDRdrive X1, ZeusRAM, AEONDrive.

For DDRdrive X1, the price is ~$1500 in 2009 when it's out but I'm yet to find its current price. Does anyone have an idea about its current price or has recently brought one?

For ZeusRAM is the price still ~$3000? It seems this one offers (more than) double of the capacity, throughput, and IOPS, as well as better interface. Is there any drawback?

For AEONDrive, is it out yet? Or if there's an approximate launch time? Any spec? Price?

Given the timeline of this build, any advise on if i shall wait a little longer until some even more awesome device to release would be appreciated too!

 

[tycoonbob]

DDRdrive X1 is around $1,995 according to :
http://www.ddrdrive.com/ddrdrive_brief.pdf

I do not like how this requires external power. I would assume since RAM is volatile, if this power gets unplugged, anything on the storage RAM is gone. But then again, I'm sure there is a specific purpose for this sort of hardware. Could you get by with a RAMDisk?

 

[AgentQ]

What makes you think any of those products are going to give you a performance boost? Are you accessing your NAS over gigabit ethernet? If so, then any modern SSD is going to be much faster than the throughput of your network link, so there is no need to look for a faster product.

Also, do you really mean ZIL? You don't want to put your ZIL on a volatile medium such as one of those devices or a RAMDISK. The ZIL exists to securely store data in a transaction group until it has been safely written to disk. If your ZIL is lost whenever the system loses power, that completely defeats the purpose.

I think you might really be thinking of a L2ARC. But even then, you would be much better off investing in more primary system RAM and a system that can support up to 64GB. Primary system RAM is going to be much faster than any solution which is attached to the PCI Express bus.

None of these devices have any place in a ZFS storage server. The good news is that you can stop lusting over strange super expensive hardware now.

 

[BigJayDogg3]

At $1500, why would you do this instead of building a machine (2011/X79 comes to mind, but I'm unfamiliar with server sockets/chipsets) with a large RAM disk?

 

[wkearney99]

For a home storage server? Why would you think you'd even need this?

There's always going to be something better down the line. The key is identifying whether current products can be proven to work reliably given your current and near-term needs. Better to have something that's KNOWN to work rather than the hassles of beta testing the latest and greatest. At least not unless you're certain there's something about the newest gear that cannot be done as effectively with known-to-work solutions.

 

[bao__zhe]

emmm...if i understand correctly DDRdrive X1/ZeusRAM uses flash to backup which makes it non-volatile. RAMDisk on the other hand is volatile.

well now here is one more question, does DDRdrive X1 backup automatically in a power loss scenario or it's always manual backup/restore? I know ZeusRAM does.

I am thinking of InfiniBand 20/40Gbps which is pretty cheap on eBay. ZFS SMB over IPoIB or Windows 2012 SMB 3.0 with RDMA over IB.

 

[AgentQ]

emmm...if i understand correctly DDRdrive X1 uses flash to backup which makes it non-volatile. RAMDisk on the other hand is volatile.

well now here is one more question, does DDRdrive X1 backup automatically in a power loss scenario or it's always manual backup/restore?

On one hand, it actually lists 'ZIL Accelerator' as a target market, which makes me think that it would automatically backup/restore itself. But note that it takes 60 seconds to backup and has no internal battery. It looks like it relies on an external AC adapter to power this operation. So you'd have to attach that AC adapter to a known-good UPS system to make it all work. And also to make sure that AC adapter plug never falls out.

But you really need to read that datasheet further. The maximum 4KB IOPS and the sustained sequential read/write speeds are actually worse than a modern SSD. And it's worse by a wide margin, too. 512B read/writes might be faster, but that's not going to help you much.

So for $1999, you can get a device that performs worse than a cheaper SSD, has less space, and requires consistent external power to be non-volatile. All while not providing you any performance benefits for your intended application.

Stick with an SSD and don't look back.

 

[madrebel]

For ZeusRAM is the price still ~$3000? It seems this one offers (more than) double of the capacity, throughput, and IOPS, as well as better interface. Is there any drawback?

can be finicky about placement within the JBOD. yes they're still ~3000 but they're awesome. i have 10 .

 

[drescherjm]

But you really need to read that datasheet further. The maximum 4KB IOPS and the sustained sequential read/write speeds are actually worse than a modern SSD. And it's worse by a wide margin, too. 512B read/writes might be faster, but that's not going to help you much.

So for $1999, you can get a device that performs worse than a cheaper SSD, has less space, and requires consistent external power to be non-volatile. All while not providing you any performance benefits for your intended application.

Stick with an SSD and don't look back.

I agree. From the specifications it does not look that this device would be a good choice now. Maybe 3 years ago but not now.

 

[bao__zhe]

On one hand, it actually lists 'ZIL Accelerator' as a target market, which makes me think that it would automatically backup/restore itself. But note that it takes 60 seconds to backup and has no internal battery. It looks like it relies on an external AC adapter to power this operation. So you'd have to attach that AC adapter to a known-good UPS system to make it all work. And also to make sure that AC adapter plug never falls out.

But you really need to read that datasheet further. The maximum 4KB IOPS and the sustained sequential read/write speeds are actually worse than a modern SSD. And it's worse by a wide margin, too. 512B read/writes might be faster, but that's not going to help you much.

So for $1999, you can get a device that performs worse than a cheaper SSD, has less space, and requires consistent external power to be non-volatile. All while not providing you any performance benefits for your intended application.

Stick with an SSD and don't look back.

good catch...that pretty much makes it out of consideration. ZIL device really needs to preserve its data.

The main concern I'm having with flash SSD is its wear leveling as ZIL is write intensive. A secondary concern is latency, since IB has very low latency I prefer not to have bottleneck on this aspect.

can be finicky about placement within the JBOD. yes they're still ~3000 but they're awesome. i have 10 .

not a good news on the price...and you are RICH...

 

[ddrdrive]

good catch...that pretty much makes it out of consideration. ZIL device really needs to preserve its data.

Your correct, a log device (ZIL) must power protect all on-board volatile caches and the DDRdrive X1 does using either a SuperCap PowerPack or small external UPS.

As far as comparing to a Flash based SSD, my OpenStorage Summit 2011 presentation might be of interest: http://www.ddrdrive.com/zil_rw_revelation.pdf or the prior year at: http://www.ddrdrive.com/zil_accelerator.pdf

 

[AgentQ]

good catch...that pretty much makes it out of consideration. ZIL device really needs to preserve its data.

The main concern I'm having with flash SSD is its wear leveling as ZIL is write intensive. A secondary concern is latency, since IB has very low latency I prefer not to have bottleneck on this aspect.

What is your application?

I doubt latency will be a problem unless your application is doing huge numbers of successive sync writes.

Wear leveling probably isn't a problem either unless you're really writing massive amounts of data all day every day.

 

[bao__zhe]

What is your application?

I doubt latency will be a problem unless your application is doing huge numbers of successive sync writes.

Wear leveling probably isn't a problem either unless you're really writing massive amounts of data all day every day.

My application will be storing some documents, photos, audios and videos. Also I have an ESXi containing Exchange server, AD DC, IIS, etc where I'd like to put the data (not the system) on it. For my documents, photos and all server data I need high reliability whereas for audios and videos I need moderate reliability and high capacity. In addition, for all server data I need low latency and high IOPS as well.

My end goal is to have performance similar to DAS consumer-grade flash SSD for small data transfers (i.e. <1GB). For large files, e.g. video files, I expect HDD JBOD performance or maybe HDD RAID0/Z performance, which i suppose is less relevant here.

In terms of the amount of data I'd say 2-10GB per day on average.

 

[madrebel]

I agree. From the specifications it does not look that this device would be a good choice now. Maybe 3 years ago but not now.

SSDs still slow down if/when you write to them constantly. the rated IOPs for SSDs is not for sustained 24/7 writing.

the ddrdrive and the zeusram NEVER slow down, ever!

yes, they're expensive however they're worth every penny.

not a good news on the price...and you are RICH...

i didnt write the check . 196 3TB drives and 37 400gb STEC SSDs .... the zeusRAMs are a tiny fraction of the total cost.

 

[obrith]

SSDs still slow down if/when you write to them constantly. the rated IOPs for SSDs is not for sustained 24/7 writing.

the ddrdrive and the zeusram NEVER slow down, ever!

yes, they're expensive however they're worth every penny.


i didnt write the check . 196 3TB drives and 37 400gb STEC SSDs .... the zeusRAMs are a tiny fraction of the total cost.

Madrebel is right. Most SSDs are very poor for ZIL use. If you want a non-volatile SSD, you need a supercap model, such as the Intel 320's or X-series, there are not a lot of options. On top of that, SSDs datasheets contain numbers achieved immediately after a secure erase. After a short while of high-write use (as we're talking about) an SSD is probably going to do about 1/4 the IOPS written on the box. I have yet to find a consumer SSD that can push 1gbps writes with ZFS as a ZIL.

However, DDRx1 and ZuesRAM can, and their IOPS numbers do not diminish under load.

 

[AgentQ]

Madrebel is right. Most SSDs are very poor for ZIL use. If you want a non-volatile SSD, you need a supercap model, such as the Intel 320's or X-series, there are not a lot of options. On top of that, SSDs datasheets contain numbers achieved immediately after a secure erase. After a short while of high-write use (as we're talking about) an SSD is probably going to do about 1/4 the IOPS written on the box. I have yet to find a consumer SSD that can push 1gbps writes with ZFS as a ZIL.

However, DDRx1 and ZuesRAM can, and their IOPS numbers do not diminish under load.

All valid points. The DDRx1 and ZeusRAM definitely have their place in high-throughput ZFS systems.

But it all depends on the application. It's going to be very, very hard to justify using a $2000 to $3000 ZIL accelerator in a home application when an Intel 320 can be had for $200. Especially when the intended use probably won't ever involve a huge number of sync writes anyway.

 

[madrebel]

why use a zil at all in a home environment? just disable sync on the shares/zvols.

 

[danswartz]

Depends on what the client for those shares is. If it's a zvol shared out as an iSCSI target, that can be dangerous, because a crash with sync disabled can corrupt the guest. I run that way, but then, I take a snapshot every night (up to 7 old ones) of the ESXi datastore share, so worst case, a crashed guest loses 24 hours work.

 

[AgentQ]

Disabling sync is generally a very, very bad idea unless you know your backup strategy works (like danswartz's setup) and you're prepared to restore from backup whenever something goes wrong.

 

[madrebel]

Depends on what the client for those shares is. If it's a zvol shared out as an iSCSI target, that can be dangerous, because a crash with sync disabled can corrupt the guest. I run that way, but then, I take a snapshot every night (up to 7 old ones) of the ESXi datastore share, so worst case, a crashed guest loses 24 hours work.

unlikely. more likely you corrupt the uber block and roll back to one of the previous 127 transactions.

in a home environment there is really no reason to run a zil. your money is MUCH better spent on a decent UPS than a zil.

 

[bao__zhe]

Your correct, a log device (ZIL) must power protect all on-board volatile caches and the DDRdrive X1 does using either a SuperCap PowerPack or small external UPS.

As far as comparing to a Flash based SSD, my OpenStorage Summit 2011 presentation might be of interest: http://www.ddrdrive.com/zil_rw_revelation.pdf or the prior year at: http://www.ddrdrive.com/zil_accelerator.pdf

So DDRdrive X1 does provide data protection. That's nice to know.

i didnt write the check . 196 3TB drives and 37 400gb STEC SSDs .... the zeusRAMs are a tiny fraction of the total cost.

Now that will be the target for the rest of my life...i need to work really hard now

But it all depends on the application. It's going to be very, very hard to justify using a $2000 to $3000 ZIL accelerator in a home application when an Intel 320 can be had for $200. Especially when the intended use probably won't ever involve a huge number of sync writes anyway.

Thinking more about this, my application may not justify for using them. And I have to say they ARE EXPENSIVE
In terms of Intel 320, should i use one or two for RAID0/1?

And given the data reliability requirement of my application I will not disable sync.

 

[obrith]

Thinking more about this, my application may not justify for using them. And I have to say they ARE EXPENSIVE
In terms of Intel 320, should i use one or two for RAID0/1?

And given the data reliability requirement of my application I will not disable sync.

My suggestions on the Intel 320 (I use several as ZILs);

Mirroring and striping will both gain you very little. If you can't afford a bit of downtime to replace a drive if it fails, then go ahead and mirror - the drive failing does not put the pool at risk though (in newer ZFS versions). Striping gains nothing in most cases. In fact in my testing it can harm performance slightly - you can look for posts by "Nex7" and he goes into why this is the case.

The best thing I've found to get good performance out of the Intel 320's is to buy a large capacity (300gb), and use a very small slice/partition/protected area of it. You will gain two things; The drive will perform MUCH better and have less degradation after high-writes, and you'll get a very considerable boost in life expectancy (my 300gb drives, using 15gb slices, are showing a 4.3PB+ expected life, vs much closer to 1PB using the full capacity). You will still not be able to saturate gigabit with 4k packets though. Be very careful if you use a GUI to manage your pool and use this method though, things like Napp-IT will not respect slices.

You'll get arguments against using a slice, mostly related to not allowing a cache-flush command (or vague references to 'because someone told me so')... however in my research this does not seem a problem because we have the supercap which flushes the cache in the event of power failure, and the Intel 320 likely will ignore the cache-flush anyways because of the supercap.

Intel 320's are the best value, easy to get ZILs I've found, if you don't need the amazing performance of products like ZuesRAM and DDR x1's.

 

[msitpro]

My suggestions on the Intel 320 (I use several as ZILs);

Mirroring and striping will both gain you very little. If you can't afford a bit of downtime to replace a drive if it fails, then go ahead and mirror - the drive failing does not put the pool at risk though (in newer ZFS versions). Striping gains nothing in most cases. In fact in my testing it can harm performance slightly - you can look for posts by "Nex7" and he goes into why this is the case.

The best thing I've found to get good performance out of the Intel 320's is to buy a large capacity (300gb), and use a very small slice/partition/protected area of it. You will gain two things; The drive will perform MUCH better and have less degradation after high-writes, and you'll get a very considerable boost in life expectancy (my 300gb drives, using 15gb slices, are showing a 4.3PB+ expected life, vs much closer to 1PB using the full capacity). You will still not be able to saturate gigabit with 4k packets though. Be very careful if you use a GUI to manage your pool and use this method though, things like Napp-IT will not respect slices.

You'll get arguments against using a slice, mostly related to not allowing a cache-flush command (or vague references to 'because someone told me so')... however in my research this does not seem a problem because we have the supercap which flushes the cache in the event of power failure, and the Intel 320 likely will ignore the cache-flush anyways because of the supercap.

Intel 320's are the best value, easy to get ZILs I've found, if you don't need the amazing performance of products like ZuesRAM and DDR x1's.

I second this

 

[drescherjm]

The best thing I've found to get good performance out of the Intel 320's is to buy a large capacity (300gb), and use a very small slice/partition/protected area of it. You will gain two things; The drive will perform MUCH better and have less degradation after high-writes, and you'll get a very considerable boost in life expectancy (my 300gb drives, using 15gb slices, are showing a 4.3PB+ expected life, vs much closer to 1PB using the full capacity). You will still not be able to saturate gigabit with 4k packets though. Be very careful if you use a GUI to manage your pool and use this method though, things like Napp-IT will not respect slices.

Fully agreed. This is an excellent way to reduce wear and also the performance drop off. I have done similar with ssd caching on linux.

 

[bao__zhe]

My suggestions on the Intel 320 (I use several as ZILs);

Mirroring and striping will both gain you very little. If you can't afford a bit of downtime to replace a drive if it fails, then go ahead and mirror - the drive failing does not put the pool at risk though (in newer ZFS versions). Striping gains nothing in most cases. In fact in my testing it can harm performance slightly - you can look for posts by "Nex7" and he goes into why this is the case.

The best thing I've found to get good performance out of the Intel 320's is to buy a large capacity (300gb), and use a very small slice/partition/protected area of it. You will gain two things; The drive will perform MUCH better and have less degradation after high-writes, and you'll get a very considerable boost in life expectancy (my 300gb drives, using 15gb slices, are showing a 4.3PB+ expected life, vs much closer to 1PB using the full capacity). You will still not be able to saturate gigabit with 4k packets though. Be very careful if you use a GUI to manage your pool and use this method though, things like Napp-IT will not respect slices.

You'll get arguments against using a slice, mostly related to not allowing a cache-flush command (or vague references to 'because someone told me so')... however in my research this does not seem a problem because we have the supercap which flushes the cache in the event of power failure, and the Intel 320 likely will ignore the cache-flush anyways because of the supercap.

Intel 320's are the best value, easy to get ZILs I've found, if you don't need the amazing performance of products like ZuesRAM and DDR x1's.

WOW that's a really comprehensive suggestion! Thanks for it!