Improving low small read/writes on OpenIndiana iSCSI

L

lizardking009

Gawd
Joined
Nov 18, 2005
Messages
663
I have a PowerEdge 2950 with 2 2ghz quadcores and 32gb of RAM. I have about 50tb of storage spread across the internal bays and external JBODs. I have a ZFS pool made up of multiple RAID 6 volumes controlled by the PERCs. The drives are all green drives.

The software is OpenIndiana with the COMSTAR iSCSI target.

iSCSI performance is pretty much what you'd expect. 2mb/sec on small block sizes up to 115mb/sec (gigE limitation) on larger sizes. This is fine for most loads of my loads but it makes my VMs kind of sluggish.

Any suggestions on how to boost the low end performance? Any tuning suggestions that have helped?

Thanks.
 
lizardking009 said:
I have a PowerEdge 2950 with 2 2ghz quadcores and 32gb of RAM. I have about 50tb of storage spread across the internal bays and external JBODs. I have a ZFS pool made up of multiple RAID 6 volumes controlled by the PERCs. The drives are all green drives.

The software is OpenIndiana with the COMSTAR iSCSI target.

iSCSI performance is pretty much what you'd expect. 2mb/sec on small block sizes up to 115mb/sec (gigE limitation) on larger sizes. This is fine for most loads of my loads but it makes my VMs kind of sluggish.

Any suggestions on how to boost the low end performance? Any tuning suggestions that have helped?

Thanks.
Click to expand...
ZIL? L2ARC?

I imagine this is ESXi connected to your iSCSI LUNs for its datastores and not the VMs connected to iSCSI for additional volumes? I'd be interested on any tweaks you do that helps.
 
A little bit of both, but Hyper-V instead of ESXi.

ZIL and L2ARC are kind of a problem... I don't have any free bays in the 2950 and the power connections could get interesting. I'm still thinking on that, too. Those new Intel SSDs look tempting.
 
One more thing, is this an All-in-One? You mention GigE so I assumed the VMs are hosted on another box.
 
lizardking009 said:
I have a PowerEdge 2950 with 2 2ghz quadcores and 32gb of RAM. I have about 50tb of storage spread across the internal bays and external JBODs. I have a ZFS pool made up of multiple RAID 6 volumes controlled by the PERCs. The drives are all green drives.

The software is OpenIndiana with the COMSTAR iSCSI target.

iSCSI performance is pretty much what you'd expect. 2mb/sec on small block sizes up to 115mb/sec (gigE limitation) on larger sizes. This is fine for most loads of my loads but it makes my VMs kind of sluggish.

Any suggestions on how to boost the low end performance? Any tuning suggestions that have helped?

Thanks.
Click to expand...

Performance is bad because both your drives and Raid-6 (Raid-Z2) are slow on I/O
For VM-datastores, I would always:

- use an extra smaller pool of faster disks (7200, better 10k or 15k disks, SSDs in future)
- always use mirrors (2 way or 3 way) for much better I/O
(on a raid-6 or raid-z2, you must postion all disks for every read or write operation,
so your I/O of such a raidset is similar only to a single disk.)

Hybrid-Storage (SSD read and syncronous write cache) can help also
 
Last edited:
Agreed. Especially for a mix of small, random reads, an n-way mirror will be much better.
 
_Gea said:
Performance is bad because both your drives and Raid-6 (Raid-Z2) are slow on I/O
For VM-datastores, I would always:

- use an extra smaller pool of faster disks (7200, better 10k or 15k disks, SSDs in future)
- always use mirrors (2 way or 3 way) for much better I/O
(on a raid-6 or raid-z2, you must postion all disks for every read or write operation,
so your I/O of such a raidset is similar only to a single disk.)

Hybrid-Storage (SSD read and syncronous write cache) can help also
Click to expand...
Is destroying the zpool and rebuilding (zfs send it out and receive it back in if that's at all possible) a viable option? If not seems like ZIL+L2ARC would be the only reasonable "tweak". Should fix some of your random IO issues.. I guess it really depends on your IO profile (read or write heavy?)

Is there any way you can go external? Hell, SATA to eSATA bridge it. Or internal Sata and tape an SSD to the side of the case... :D

PS Is this a Production or lab environment?
 
L2ARC could maybe help. It is easy to connect, and disconnect. One command, and then you are done.
 
This is a home lab environment but I'd rather keep my RAID config as is. I'm doing my best to keep the capacity up. The performance isn't terrible on the VMs, but I'd like every little bit. Hopefully Dell left me a place to mount the L2ARC device with double sided tape.

It would be nice if there were a PCIe device with onboard SATA and power and a mounting place for an SSD. Kind of like a BYO SSD Revodrive.

EDIT: Such a product exists! http://www.apricorn.com/products/desktop-ssd-hdd-upgrade-kits/velocity-solo.html
 
Last edited:
You must log in or register to reply here.
Back
Top