Best way to make use of 160GB SSD in OpenIndiana ZFS server?

M

Maxwell Smart

Limp Gawd
Joined
Nov 15, 2005
Messages
139
Hi all, I have an ESXi server all-in-one build based off of _Gea's excellent guide with the following specs:

Quad-Core Xeon w/ Hyperthreading
16GB of RAM
IBM M1015 in IT Mode
8x3TB Hard drives in a RAID-Z2 with ~15TB of usable storage.
2x16GB SSDs set up to hold the OpenIndiana VM image

I just acquired a 160GB Intel G2 SSD and I'd like to make use of it in my server. What I'm not sure of is what is the most effective use for it. ZIL? cache? Would I need to rebuild my current array? Additionally, can I put it on the motherboard SATA ports or would I need to acquire another raid card and pass it through to the OpenIndiana VM with VT-d? Thanks for any potential help and advice.
 
I'm confused. I assume you have a datastore for OI? Are the two SSDs for that raided somehow? If you want to use the 160GB SSD for a l2arc, yes, you'd need to pass in some other controller. A separate ZIL is honestly a waste with an all-in-one, just disable sync. Or do you have the datastore spread over the two SSDs? If so, kinda overkill (unless the two SSDs were free), since OI does very little I/O with the root pool...
 
The SSDs were cheap, only 40 bucks for both, my intention was initially to RAID 1 them together, but I ended up needing the other for another VM, so they are both independent and on the system SATA bus. So the best use of the SSD would be for an l2arc, and I'd need to pass through an additional controller? How much of a potential performance gain would I see by enabling L2arc?
 
Depends on how much of your working set fits in it. If most of your working set fits in l2arc, you eliminate almost all disk reads, which gets your latency down to network only.
 
Only by trial and error. You set up the l2arc device(c) and let 'er rip. After a few days of typical use, what has been read is (for the most part) your working set. If the SSD is/are too small, you might still be okay, depending on the l2 hit/miss ratio, which you can get by:

kstat | egrep "(l2_hits|l2_misses)"
 
Perhaps before that he should see how much of his working set fits in ARC first. If he has a very high hit ratio in his ARC maybe the L2 would be a waste. Perhaps revitalize or upgrade a laptop or desktop system with that drive instead?
 
That's a valid point. I have about 256GB served up to a handful of VMs. I only have 8GB RAM (mobo restriction) with about 6GB going to ARC. I see a consistent 75% ARC hit rate. I had tried adding a 64GB M4 SSD but it made virtually no difference for an arcane reason it took me a long time to understand. I was seeing almost no hits in cache, but looking at all the L2ARC stats, I saw almost all adds to L2ARC were being evicted later. Some googling revealed that L2ARC entries require "pointers" in ARC, so if your ARC is 'too small', you will just thrash on the L2ARC without ever utilizing it.
 
Maybe you can try just creating a VMDK on your SSD of x amount of GB and see if it makes a difference after a while.
I am curious to see how big (if any) speed advantages arise when copying data to your storage and also if there is a large difference between creating a VMDK or "offering" the complete SSD to your storage.

Your test might just prove that it is possible the carve up the SSD if it has IOPS to spare. That way multiple volumes might benefit from a SSD.
Maybe as described here:
http://www.livingonthecloud.net/2011/09/hp-microserver-building-san-in-box.html
 
You must log in or register to reply here.
Back
Top