ZFS on HP SmartArrays - poor performance?

ajm83

n00b
Joined
Jan 25, 2012
Messages
32
Is it expected that ZFS/Nexenta performance will be absolutely dire on SmartArray P400 hardware?

For example P400 card, mirrored pair of 5400rpm SATA discs, (for testing only before purchasing faster discs) writing at only 2.9mb/s (expect at least 40mb/s):

root@nex385:/volumes# dd if=/dev/zero of=/volumes/v1/tempfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 363.651 seconds, 2.9 MB/s

We've tried changing MAX/MIN PENDING, and disabling the write cache flush. Neither improved performance.

Is there anything we need to do to make this card ZFS friendly? Or are these smart cards totally unsuitable for ZFS?

(As an aside this system is intended to replace another SmartArray system - 6400/MSA20 which also has poor performance - 30mb/s absolute peak from a 9 disk raidz2 - random writes are more in the order of 5-10mb/s...)

Edit: I should add that the server proc/mem is vastly over specced - quad core or dual quad and at least 8gb RAM - we've tested with upto 32GB)
 
Sounds like some kind of write through cache issue for the controller?

It's interesting that you say that - if we enable the 'array accelerator' (battery backed read/write cache) we'll get 1.6gb/s and an immediate return from dd. IOSTAT -x will then show the disks 100% busy for 2 or 3 minutes and if we kick off another dd, it'll be run at the 3mb/s speed.

I think that means that the cache part of the card is working, but for whatever reason, its interface to the physical disks is not right.
 
Is there anything we need to do to make this card ZFS friendly? Or are these smart cards totally unsuitable for ZFS?

Those P400 and P410 don't support a real JBOD setting. Only thing that can be done is putting every disk in a One-Disk-Raid 0 so the OS sees single drives. However this means there will be metadata on the drive from the controller.
 
Yes battery backup (tested and working). Think the card has 512MB, with a 25R/75W split.

dunno. it says the BBU is working, but these abysmal write numbers usually seem to be when the controller is set to write-thru. maybe it's not the BBU but a setting in the card bios that is wrong? or something else?
 
dunno. it says the BBU is working, but these abysmal write numbers usually seem to be when the controller is set to write-thru. maybe it's not the BBU but a setting in the card bios that is wrong? or something else?

Hmmm, so 2-3mb/s would be expected if set to write through mode (I assume by this you mean "Array Accel" is switched off?
 
I don't know about expected or any specific numbers. Just saying that every time I've seen a complaint about horrific write perf, it's been something like this. Sorry I can't contribute more :(
 
How is your zpool laid out? Is the card presenting a single volume or individual disks?
 
If an array is setup on the P400,

have you given it enough time to do it's parity check?
As it will be dog slow until that is performed.

Leave it running (turned on) for say 6hrs then test again.... see if performance improves

But really Toss the P400 for ZFS and just use plain HBA's

.
 
It's not setup as a RAID array in the controller, just individual disks in raid-0 setup.

Problem for us is that we have a quite a few HP servers spare with loads of memory & CPUs, but none of them have simple SATA, or SAS ports; they all have smartarrays. We also have an MSA20 SATA array, but that will only talk to a SmartArray card, despite using a standard SCSI port, it's not normal SCSI. :mad:

I think at this point it might be easier to invest in something like the Norco 4020 and roll our own setup out. The bummer then is that we need to buy everything x 2 for redundancy as this is our main ISCSI VMWare filer.

Cheers for the help everyone.
 
Back
Top