Would these be good for ZIL?

R

Ripley

Limp Gawd
Joined
Nov 4, 2004
Messages
240
I was thinking about adding one or a mirrored pair of these as ZIL for my new ZFS box. I've read that SLC is better than MLC for ZIL. Any thoughts? Anyone using them or something similar for ZIL?

http://www.ebay.com/itm/Intel-SATA-...?pt=US_Solid_State_Drives&hash=item19d50c9eeb

http://www.ebay.com/itm/20-GB-SLC-S...?pt=US_Solid_State_Drives&hash=item4846a43b06

If not either of these is there something close to this price range that would be better? I'd love to get some of the STEC stuff but there's no way I can afford it.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Is it a home box with light use or a work box with lots of use? SLC is always preferred, and if you are doing a mirror even better. If its a work box with lots of users, get something bigger.
 
I think you're thinking of l2arc. For almost any modern config, 8GB is more than enough.
 
I have been up too long, I had an arc thread up in another window. We use the 8GB STEC DDR SSD's for Zil, the Intels he is looking at are fine.
 
For a home setup I think you are losing speed with those drives. They are too small and too slow to help.

It might help if you were trying to cache a database server getting 1000s of hits a minute, but I dont think that is what you are going for?
 
danswartz said:
Again, this is for ZIL, not L2ARC, so caching doesn't enter into it.
Click to expand...

His wording wasn't correct, but his sentiment was.

The drives are probably going to make your top-end sync-write bandwidth rather low. They will provide pretty good improvement on a pool of spinners if you're demanding a decent IOPS number and don't want to spend much - IE you value latency at a lower bandwidth for a handful of VMs rather than maxing out gig-e (obviously this only applies to sync). These drives are only rated for 3300 write IOPS though, and the ratings are very generous.

Neither of these have capacitors FYI.

I've had good success buying larger Intel 320's and massively under-provisioning them when I need a cheap ZIL.
 
ZIL is all about iops, and you really need some really really fast devices to actually get any performance improvement.
Intel 320 will help, though they are still too slow to be a good ZIL.
 
Agreed. I have tried random SSDs and the performance has been universally disappointing. Given that I'm not rich and this is an all-in-one setup, I just disabled sync.
 
So is there no performance gain to be had from using one of these? Is the only option for getting noticeable improvement a $2k STEC?
 
No, it's better than running sync=enabled for something like NFS, it's just disappointing compared to not needing sync (e.g. having stec or somesuch or just disabling sync mode entirely.) How much all depends. You might want to borrow or buy a cheap ssd and do your own test. If it really sucks too much, return it (assuming your vendor doesn't mind and/or you don't mind the restocking fee...)
 
What's the best way to test ZIL? Or the difference between have an SSD for ZIL and not having one?
 
I generally prefer to test my ZILs in the use-case they'll actually see. So, for example, I often test from within a VM attached to them because most often that's how I'm targeting my ZFS boxes.

I just actually did some pretty extensive testing of an Intel 320 (300gb @ 15gb usable) vs my DDR X1.

The Intel 320 is backing a VMware 4.1 cluster. The test subject is a Ubuntu 10.04 server. IOzone with 12 threads @ 4k blocks can write 35,000 IOPS (over gigabit, using a 10GigE network). A single threaded test comes back at 22,000 IOPS. Mind you this is NOT the type of IOPS I see all the time on this ZIL, but it's capable of them. I generally see no less than 60MB/sec in transfers hitting this SAN.

The DDR X1 is backing a VMware 5 cluster. The test subject is an Ubuntu 12.04 server. IOzone with 12 threads @ 4k blocks can max out gigabit (~30,000 IOPS). Single threaded is about the same. Again, I dont see those IOPS out of the ZIL a lot of times. I sometimes get measly 40MB/sec transfers on this ZIL, but it's clearly capable of more than the 1GbE it's connected with in some situations.

I don't have data on what the DDR X1 can do on 10GbE yet. I've got a new system with a ZuesRAM on order, which will be replacing the Intel 320's primary duties.

I personally think the Intel 320's doing a damn fine job for a sub $500 ZIL.
 
obrith said:
I personally think the Intel 320's doing a damn fine job for a sub $500 ZIL.
Click to expand...

I believed the same until i found that there's a 8MB bug in Intel 320 and it still exists after a firmware upgrade which is supposed to solve it.

Just wondering how often the bug is happening in general and how big loss there can be if used as ZIL? (say, latest ZFS version.)
 
bao__zhe said:
I believed the same until i found that there's a 8MB bug in Intel 320 and it still exists after a firmware upgrade which is supposed to solve it.

Just wondering how often the bug is happening in general and how big loss there can be if used as ZIL? (say, latest ZFS version.)
Click to expand...

I've actually never seen that bug occur (knock on wood!). I have several dozen deployed in our IT infrastructure and we've got hundreds if not thousands in the field in our products. I've never heard of our products hitting this bug, I should ask.

One thing I wasn't quick to dig up was if this bug actually hits anything but Windows users...

Maybe I'll find out the hard way one day.
 
What kinds of test have you used to see the differences that ZIL makes?
 
The ZIL is only going to do anything when you're doing synchronous writes - meaning the program doing the writing is asking for a confirmation of every block of data written before it moves on to the next block. The most common scenario most of us see using mostly/all sync-writes is when we use NFS, especially with VMware.

You always have a ZIL. Typically it's just spread across your pool, so it's roughly as fast as your pool (on it's worst day). In most cases people choose to implement a ZIL because they want/need better response time and/or IOPS out of their ZFS machine. The time the ZIL really shines is when it has a random, synchronous, threaded workload. If you're using a ZFS box to host a VM cluster, this is nearly a requirement.

For example, I've got 2 servers that are roughly equal, save a ZIL. They've got 7 vdev's each of 10x 500gb laptop drives in RAIDZ2's (don't ask, the drives were free). If I do random sync writes to the one without a ZIL, the pool can do maybe 300 write IOPS. When I write to the one with a ZIL, it can do up to 35,000 write IOPS.

It's a narrow but common use case that you will want or need a ZIL.
 
obrith - I've read that SLC is preferable to MLC for ZIL. But the 320s are MLC. What made you choose them?


obrith said:
I generally prefer to test my ZILs in the use-case they'll actually see. So, for example, I often test from within a VM attached to them because most often that's how I'm targeting my ZFS boxes.

I just actually did some pretty extensive testing of an Intel 320 (300gb @ 15gb usable) vs my DDR X1.

The Intel 320 is backing a VMware 4.1 cluster. The test subject is a Ubuntu 10.04 server. IOzone with 12 threads @ 4k blocks can write 35,000 IOPS (over gigabit, using a 10GigE network). A single threaded test comes back at 22,000 IOPS. Mind you this is NOT the type of IOPS I see all the time on this ZIL, but it's capable of them. I generally see no less than 60MB/sec in transfers hitting this SAN.

The DDR X1 is backing a VMware 5 cluster. The test subject is an Ubuntu 12.04 server. IOzone with 12 threads @ 4k blocks can max out gigabit (~30,000 IOPS). Single threaded is about the same. Again, I dont see those IOPS out of the ZIL a lot of times. I sometimes get measly 40MB/sec transfers on this ZIL, but it's clearly capable of more than the 1GbE it's connected with in some situations.

I don't have data on what the DDR X1 can do on 10GbE yet. I've got a new system with a ZuesRAM on order, which will be replacing the Intel 320's primary duties.

I personally think the Intel 320's doing a damn fine job for a sub $500 ZIL.
Click to expand...
 
The primary reason people choose SLC is life expectancy under a write-heavy load. A few years ago SLC was the only way to go because you'd wear out your MLC drives very rapidly. Now the controllers have improved greatly and drive makers put aside reserve capacity. On top of that you can further under-provision drives to increase their lifespan under heavy writes. SLCs are also slightly faster, but it's pretty minor any more.

One of my Intel 320's, very under provisioned (15gb in use of 300), shows 15TB written @ 0% wear. Last I calculated it I figured it should exhaust its wear indicator at about 4.4PB written. That wont be for several years in this server... long before it's retired.
 
One thing of note; if you use Napp-IT, it doesn't support using slices (underprovision). You have to do it manually and be careful not to re-use the disk in the Napp-IT GUI. Napp-IT will gladly stomp on your in-use slice to use the whole disk.
 
obrith said:
One thing of note; if you use Napp-IT, it doesn't support using slices (underprovision). You have to do it manually and be careful not to re-use the disk in the Napp-IT GUI. Napp-IT will gladly stomp on your in-use slice to use the whole disk.
Click to expand...

Info
Next napp-it nightly 0.8m has improved disk hotplug detection and supports partitions.
So this may no longer a problem.
 
_Gea said:
Info
Next napp-it nightly 0.8m has improved disk hotplug detection and supports partitions.
So this may no longer a problem.
Click to expand...

I wasn't complaining (much ;) ). I would appreciate it though - I know your take on slices.
 
obrith - Do you run you ZIL in mirrored pairs or as single drives?

General question, I believe that you can add a ZIL after a pool is created but not remove one is that correct?
 
Depends on how important the server is. Generally if I'm using a 320 it's because I want to keep the server cheap and the data isn't invaluable (or is well backed up)... so I go with single ZILs. The ZIL failing simply slows the server down if you're using a current version of ZFS. If you can't afford the down time (or pool slowdown if you have hot-plugable drives) to replace the ZIL in a failure, use a Mirror.

You can both add and remove a ZIL after the pool is created.
 
Worth noting,

If you upograde to FreeNAS 8.3 (final release is probably any day now) and upgrade your storage pools to the final ZFS volume, there is no longer any reason to mirror the ZIL. The most recent ZFS revisions can survive ZIL failures.
 
You must log in or register to reply here.
Back
Top