ZFS performance test with ZFSguru

bao__zhe · Dec 10, 2012

tangoseal said:
Well zfs will use all the ram it has available for caching. You really dont need a hba unless you want bleeding edge io but you would be surprised at how fast an intel ich10 can be and works perfect with 99% of set ups. However you only have 6 ports on an ich10 controller.

I would go with an ecc supporting amd 965 quadcore and m1015 for the best performance per dollar or an intel i3-3000 series dual core with ich10 and 6 disc for another highly cost effective rig. I just shelled out $2000 for my e5-1620 piweres 32g zfs box.

Per my research ECC RAM is crucial for ZFS, or you are risking your data.

ZFS does eat lots of RAM if available as @ tangoseal points out. Smaller RAM will work but will reduce performance. ZFS eats CPU as well, especially if compression is turned on, even only LZJB. per my current experience over the several days, if another VM on the same ESXi host reads files for the full speed the host CPU can go to 100%. the Solaris VM is given 6 vCPU and the other VM is using 8 reading threads with 4 vCPU.

Billy_nnn · Dec 11, 2012

bao__zhe said:
Per my research ECC RAM is crucial for ZFS, or you are risking your data.

ECC memory is no more crucial for ZFS than it is for any other data storage filesystem/lvm!
It's desirable of course, but it can depend on the nature of the data too!
Certainly lack of ECC memory shouldn't put anyone off deploying ZFS - it's still "safer" than many other filesystems/lvms would be on the same hardware!

If you insist on ECC memory on the storage server, then you should really also have ECC memory on all systems which manipulate the storage server's data.
In many cases, dirty data will spend more time sat in the memory of client systems than it will sat in the storage server's memory, and the storage server is no more susceptible to random bitflips than the client systems!

Of course, ECC also helps with availability, and can make it easier to identify failing memory (as long as the ECC is implemented correctly).

tangoseal · Dec 11, 2012

I am ZFS Guru testing as we speak. I will let you know what I find soon.

tangoseal · Dec 12, 2012

So here is some results from raid testing 8 Western Digital Drives using onboard C602 controller. All drives are on 3gbps Sata II ports.

Not sure what is actually useful to the community here but I will post everything text wise.....

Code:

SGURU-benchmark, version 1
Test size: 32.000 gigabytes (GiB)
Test rounds: 3
Cooldown period: 2 seconds
Sector size override: 4096 bytes
Number of disks: 8 disks
disk 1: gpt/disk1.nop
disk 2: gpt/disk2.nop
disk 3: gpt/disk3.nop
disk 4: gpt/disk4.nop
disk 5: gpt/disk5.nop
disk 6: gpt/disk6.nop
disk 7: gpt/disk7.nop
disk 8: gpt/disk8.nop


* Test Settings: TS32; SECT4096; 
* Tuning: none
* Stopping background processes: sendmail, moused, syslogd and cron
* Stopping Samba service

Now testing RAID0 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	587 MiB/sec	606 MiB/sec	583 MiB/sec	= 592 MiB/sec avg
WRITE:	987 MiB/sec	960 MiB/sec	953 MiB/sec	= 967 MiB/sec avg
raidtest.read:	84	107	83	= 91 IOps ( ~6006 KiB/sec )
raidtest.write:	84	115	86	= 95 IOps ( ~6270 KiB/sec )
raidtest.mixed:	74	79	75	= 76 IOps ( ~5016 KiB/sec )

Now testing RAIDZ configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	627 MiB/sec	594 MiB/sec	626 MiB/sec	= 615 MiB/sec avg
WRITE:	729 MiB/sec	709 MiB/sec	670 MiB/sec	= 703 MiB/sec avg
raidtest.read:	61	60	62	= 61 IOps ( ~4026 KiB/sec )
raidtest.write:	70	68	70	= 69 IOps ( ~4554 KiB/sec )
raidtest.mixed:	57	59	59	= 58 IOps ( ~3828 KiB/sec )

Now testing RAIDZ2 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	602 MiB/sec	583 MiB/sec	605 MiB/sec	= 597 MiB/sec avg
WRITE:	585 MiB/sec	585 MiB/sec	548 MiB/sec	= 573 MiB/sec avg
raidtest.read:	59	59	59	= 59 IOps ( ~3894 KiB/sec )
raidtest.write:	63	63	73	= 66 IOps ( ~4356 KiB/sec )
raidtest.mixed:	58	59	57	= 58 IOps ( ~3828 KiB/sec )

Now testing RAID1 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	552 MiB/sec	500 MiB/sec	583 MiB/sec	= 545 MiB/sec avg
WRITE:	102 MiB/sec	98 MiB/sec	98 MiB/sec	= 99 MiB/sec avg
raidtest.read:	83	73	75	= 77 IOps ( ~5082 KiB/sec )
raidtest.write:	73	65	68	= 68 IOps ( ~4488 KiB/sec )
raidtest.mixed:	50	52	51	= 51 IOps ( ~3366 KiB/sec )

Now testing RAID1+0 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	582 MiB/sec	579 MiB/sec	588 MiB/sec	= 583 MiB/sec avg
WRITE:	462 MiB/sec	488 MiB/sec	467 MiB/sec	= 472 MiB/sec avg
raidtest.read:	82	79	81	= 80 IOps ( ~5280 KiB/sec )
raidtest.write:	79	81	79	= 79 IOps ( ~5214 KiB/sec )
raidtest.mixed:	67	67	68	= 67 IOps ( ~4422 KiB/sec )

Now testing RAIDZ+0 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	458 MiB/sec	473 MiB/sec	468 MiB/sec	= 467 MiB/sec avg
WRITE:	672 MiB/sec	659 MiB/sec	657 MiB/sec	= 663 MiB/sec avg
raidtest.read:	64	64	64	= 64 IOps ( ~4224 KiB/sec )
raidtest.write:	69	69	70	= 69 IOps ( ~4554 KiB/sec )
raidtest.mixed:	62	63	63	= 62 IOps ( ~4092 KiB/sec )

Now testing RAID0 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	398 MiB/sec	401 MiB/sec	400 MiB/sec	= 400 MiB/sec avg
WRITE:	486 MiB/sec	489 MiB/sec	483 MiB/sec	= 486 MiB/sec avg
raidtest.read:	90	74	74	= 79 IOps ( ~5214 KiB/sec )
raidtest.write:	93	78	77	= 82 IOps ( ~5412 KiB/sec )
raidtest.mixed:	68	67	66	= 67 IOps ( ~4422 KiB/sec )

Now testing RAID0 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	444 MiB/sec	444 MiB/sec	449 MiB/sec	= 446 MiB/sec avg
WRITE:	618 MiB/sec	615 MiB/sec	590 MiB/sec	= 608 MiB/sec avg
raidtest.read:	96	94	97	= 95 IOps ( ~6270 KiB/sec )
raidtest.write:	99	99	100	= 99 IOps ( ~6534 KiB/sec )
raidtest.mixed:	73	71	72	= 72 IOps ( ~4752 KiB/sec )

Now testing RAID0 configuration with 6 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	505 MiB/sec	501 MiB/sec	502 MiB/sec	= 503 MiB/sec avg
WRITE:	696 MiB/sec	738 MiB/sec	748 MiB/sec	= 727 MiB/sec avg
raidtest.read:	85	79	80	= 81 IOps ( ~5346 KiB/sec )
raidtest.write:	90	80	81	= 83 IOps ( ~5478 KiB/sec )
raidtest.mixed:	73	73	74	= 73 IOps ( ~4818 KiB/sec )

Now testing RAID0 configuration with 7 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	571 MiB/sec	589 MiB/sec	587 MiB/sec	= 582 MiB/sec avg
WRITE:	855 MiB/sec	804 MiB/sec	864 MiB/sec	= 841 MiB/sec avg
raidtest.read:	81	91	81	= 84 IOps ( ~5544 KiB/sec )
raidtest.write:	83	91	83	= 85 IOps ( ~5610 KiB/sec )
raidtest.mixed:	75	74	74	= 74 IOps ( ~4884 KiB/sec )

Now testing RAIDZ configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	256 MiB/sec	240 MiB/sec	261 MiB/sec	= 252 MiB/sec avg
WRITE:	343 MiB/sec	358 MiB/sec	322 MiB/sec	= 341 MiB/sec avg
raidtest.read:	62	61	62	= 61 IOps ( ~4026 KiB/sec )
raidtest.write:	65	64	65	= 64 IOps ( ~4224 KiB/sec )
raidtest.mixed:	60	59	59	= 59 IOps ( ~3894 KiB/sec )

Now testing RAIDZ configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	360 MiB/sec	349 MiB/sec	348 MiB/sec	= 352 MiB/sec avg
WRITE:	433 MiB/sec	591 MiB/sec	440 MiB/sec	= 488 MiB/sec avg
raidtest.read:	58	58	60	= 58 IOps ( ~3828 KiB/sec )
raidtest.write:	63	71	71	= 68 IOps ( ~4488 KiB/sec )
raidtest.mixed:	57	56	57	= 56 IOps ( ~3696 KiB/sec )

Now testing RAIDZ configuration with 6 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	482 MiB/sec	468 MiB/sec	451 MiB/sec	= 467 MiB/sec avg
WRITE:	693 MiB/sec	555 MiB/sec	528 MiB/sec	= 592 MiB/sec avg
raidtest.read:	64	60	63	= 62 IOps ( ~4092 KiB/sec )
raidtest.write:	75	66	76	= 72 IOps ( ~4752 KiB/sec )
raidtest.mixed:	58	58	57	= 57 IOps ( ~3762 KiB/sec )

Now testing RAIDZ configuration with 7 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	527 MiB/sec	514 MiB/sec	512 MiB/sec	= 518 MiB/sec avg
WRITE:	645 MiB/sec	616 MiB/sec	609 MiB/sec	= 623 MiB/sec avg
raidtest.read:	59	62	62	= 61 IOps ( ~4026 KiB/sec )
raidtest.write:	66	76	75	= 72 IOps ( ~4752 KiB/sec )
raidtest.mixed:	57	58	58	= 57 IOps ( ~3762 KiB/sec )

Now testing RAIDZ2 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	230 MiB/sec	208 MiB/sec	202 MiB/sec	= 213 MiB/sec avg
WRITE:	231 MiB/sec	250 MiB/sec	254 MiB/sec	= 245 MiB/sec avg
raidtest.read:	56	56	56	= 56 IOps ( ~3696 KiB/sec )
raidtest.write:	62	60	61	= 61 IOps ( ~4026 KiB/sec )
raidtest.mixed:	53	41	53	= 49 IOps ( ~3234 KiB/sec )

Now testing RAIDZ2 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	294 MiB/sec	295 MiB/sec	284 MiB/sec	= 291 MiB/sec avg
WRITE:	339 MiB/sec	358 MiB/sec	399 MiB/sec	= 366 MiB/sec avg
raidtest.read:	56	46	56	= 52 IOps ( ~3432 KiB/sec )
raidtest.write:	59	47	60	= 55 IOps ( ~3630 KiB/sec )
raidtest.mixed:	40	38	54	= 44 IOps ( ~2904 KiB/sec )

Now testing RAIDZ2 configuration with 6 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	379 MiB/sec	392 MiB/sec	371 MiB/sec	= 381 MiB/sec avg
WRITE:	460 MiB/sec	588 MiB/sec	397 MiB/sec	= 482 MiB/sec avg
raidtest.read:	65	81	80	= 75 IOps ( ~4950 KiB/sec )
raidtest.write:	69	81	79	= 76 IOps ( ~5016 KiB/sec )
raidtest.mixed:	45	48	47	= 46 IOps ( ~3036 KiB/sec )

Now testing RAIDZ2 configuration with 7 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	476 MiB/sec	465 MiB/sec	476 MiB/sec	= 472 MiB/sec avg
WRITE:	549 MiB/sec	465 MiB/sec	504 MiB/sec	= 506 MiB/sec avg
raidtest.read:	59	58	58	= 58 IOps ( ~3828 KiB/sec )
raidtest.write:	65	62	62	= 63 IOps ( ~4158 KiB/sec )
raidtest.mixed:	57	56	57	= 56 IOps ( ~3696 KiB/sec )

Now testing RAID1 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	341 MiB/sec	351 MiB/sec	357 MiB/sec	= 350 MiB/sec avg
WRITE:	117 MiB/sec	126 MiB/sec	118 MiB/sec	= 121 MiB/sec avg
raidtest.read:	79	74	71	= 74 IOps ( ~4884 KiB/sec )
raidtest.write:	73	70	67	= 70 IOps ( ~4620 KiB/sec )
raidtest.mixed:	46	58	57	= 53 IOps ( ~3498 KiB/sec )

Now testing RAID1 configuration with 5 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	419 MiB/sec	415 MiB/sec	415 MiB/sec	= 417 MiB/sec avg
WRITE:	105 MiB/sec	104 MiB/sec	103 MiB/sec	= 104 MiB/sec avg
raidtest.read:	72	76	78	= 75 IOps ( ~4950 KiB/sec )
raidtest.write:	67	72	73	= 70 IOps ( ~4620 KiB/sec )
raidtest.mixed:	59	48	49	= 52 IOps ( ~3432 KiB/sec )

Now testing RAID1 configuration with 6 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	484 MiB/sec	495 MiB/sec	539 MiB/sec	= 506 MiB/sec avg
WRITE:	104 MiB/sec	105 MiB/sec	141 MiB/sec	= 117 MiB/sec avg
raidtest.read:	77	73	93	= 81 IOps ( ~5346 KiB/sec )
raidtest.write:	71	66	89	= 75 IOps ( ~4950 KiB/sec )
raidtest.mixed:	48	49	62	= 53 IOps ( ~3498 KiB/sec )

Now testing RAID1 configuration with 7 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	537 MiB/sec	572 MiB/sec	541 MiB/sec	= 550 MiB/sec avg
WRITE:	103 MiB/sec	108 MiB/sec	103 MiB/sec	= 105 MiB/sec avg
raidtest.read:	75	65	75	= 71 IOps ( ~4686 KiB/sec )
raidtest.write:	67	59	66	= 64 IOps ( ~4224 KiB/sec )
raidtest.mixed:	59	56	60	= 58 IOps ( ~3828 KiB/sec )

Now testing RAID1+0 configuration with 4 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	382 MiB/sec	408 MiB/sec	402 MiB/sec	= 398 MiB/sec avg
WRITE:	233 MiB/sec	255 MiB/sec	258 MiB/sec	= 248 MiB/sec avg
raidtest.read:	72	75	71	= 72 IOps ( ~4752 KiB/sec )
raidtest.write:	69	74	70	= 71 IOps ( ~4686 KiB/sec )
raidtest.mixed:	61	60	60	= 60 IOps ( ~3960 KiB/sec )

Now testing RAID1+0 configuration with 6 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	497 MiB/sec	491 MiB/sec	491 MiB/sec	= 493 MiB/sec avg
WRITE:	411 MiB/sec	346 MiB/sec	342 MiB/sec	= 367 MiB/sec avg
raidtest.read:	82	76	76	= 78 IOps ( ~5148 KiB/sec )
raidtest.write:	82	74	76	= 77 IOps ( ~5082 KiB/sec )
raidtest.mixed:	67	66	67	= 66 IOps ( ~4356 KiB/sec )

Now testing RAIDZ+0 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	458 MiB/sec	477 MiB/sec	477 MiB/sec	= 252 MiB/sec avg
WRITE:	660 MiB/sec	646 MiB/sec	663 MiB/sec	= 341 MiB/sec avg
raidtest.read:	63	63	63	= 61 IOps ( ~4026 KiB/sec )
raidtest.write:	70	69	69	= 64 IOps ( ~4224 KiB/sec )
raidtest.mixed:	62	61	62	= 59 IOps ( ~3894 KiB/sec )

Now testing RAID0 configuration with 1 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	145 MiB/sec	138 MiB/sec	126 MiB/sec	= 137 MiB/sec avg
WRITE:	140 MiB/sec	114 MiB/sec	109 MiB/sec	= 121 MiB/sec avg
raidtest.read:	75	74	81	= 76 IOps ( ~5016 KiB/sec )
raidtest.write:	66	66	70	= 67 IOps ( ~4422 KiB/sec )
raidtest.mixed:	40	41	52	= 44 IOps ( ~2904 KiB/sec )

Now testing RAID0 configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	243 MiB/sec	256 MiB/sec	255 MiB/sec	= 251 MiB/sec avg
WRITE:	250 MiB/sec	256 MiB/sec	296 MiB/sec	= 267 MiB/sec avg
raidtest.read:	71	66	71	= 69 IOps ( ~4554 KiB/sec )
raidtest.write:	69	65	68	= 67 IOps ( ~4422 KiB/sec )
raidtest.mixed:	58	55	59	= 57 IOps ( ~3762 KiB/sec )

Now testing RAID0 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	326 MiB/sec	318 MiB/sec	314 MiB/sec	= 319 MiB/sec avg
WRITE:	386 MiB/sec	356 MiB/sec	357 MiB/sec	= 366 MiB/sec avg
raidtest.read:	76	82	83	= 80 IOps ( ~5280 KiB/sec )
raidtest.write:	77	85	87	= 83 IOps ( ~5478 KiB/sec )
raidtest.mixed:	66	64	66	= 65 IOps ( ~4290 KiB/sec )

Now testing RAIDZ configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	128 MiB/sec	129 MiB/sec	149 MiB/sec	= 135 MiB/sec avg
WRITE:	110 MiB/sec	130 MiB/sec	132 MiB/sec	= 124 MiB/sec avg
raidtest.read:	67	68	71	= 68 IOps ( ~4488 KiB/sec )
raidtest.write:	65	64	68	= 65 IOps ( ~4290 KiB/sec )
raidtest.mixed:	56	40	44	= 46 IOps ( ~3036 KiB/sec )

Now testing RAIDZ configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	222 MiB/sec	224 MiB/sec	213 MiB/sec	= 220 MiB/sec avg
WRITE:	242 MiB/sec	240 MiB/sec	221 MiB/sec	= 235 MiB/sec avg
raidtest.read:	56	61	59	= 58 IOps ( ~3828 KiB/sec )
raidtest.write:	60	64	64	= 62 IOps ( ~4092 KiB/sec )
raidtest.mixed:	56	56	39	= 50 IOps ( ~3300 KiB/sec )

Now testing RAIDZ2 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	142 MiB/sec	142 MiB/sec	150 MiB/sec	= 145 MiB/sec avg
WRITE:	88 MiB/sec	100 MiB/sec	96 MiB/sec	= 95 MiB/sec avg
raidtest.read:	85	71	79	= 78 IOps ( ~5148 KiB/sec )
raidtest.write:	71	63	70	= 68 IOps ( ~4488 KiB/sec )
raidtest.mixed:	45	66	66	= 59 IOps ( ~3894 KiB/sec )

Now testing RAID1 configuration with 2 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	215 MiB/sec	216 MiB/sec	194 MiB/sec	= 208 MiB/sec avg
WRITE:	110 MiB/sec	111 MiB/sec	112 MiB/sec	= 111 MiB/sec avg
raidtest.read:	71	69	68	= 69 IOps ( ~4554 KiB/sec )
raidtest.write:	66	66	63	= 65 IOps ( ~4290 KiB/sec )
raidtest.mixed:	56	56	54	= 55 IOps ( ~3630 KiB/sec )

Now testing RAID1 configuration with 3 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	301 MiB/sec	281 MiB/sec	300 MiB/sec	= 294 MiB/sec avg
WRITE:	116 MiB/sec	105 MiB/sec	109 MiB/sec	= 110 MiB/sec avg
raidtest.read:	71	71	71	= 71 IOps ( ~4686 KiB/sec )
raidtest.write:	66	66	66	= 66 IOps ( ~4356 KiB/sec )
raidtest.mixed:	57	45	56	= 52 IOps ( ~3432 KiB/sec )

Now testing RAIDZ+0 configuration with 8 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ:	462 MiB/sec	472 MiB/sec	486 MiB/sec	= 252 MiB/sec avg
WRITE:	667 MiB/sec	643 MiB/sec	623 MiB/sec	= 341 MiB/sec avg
raidtest.read:	64	63	65	= 61 IOps ( ~4026 KiB/sec )
raidtest.write:	69	68	70	= 64 IOps ( ~4224 KiB/sec )
raidtest.mixed:	62	61	62	= 59 IOps ( ~3894 KiB/sec )

Done

it would seem apparent that my best bang for the buck configuration performance and redundnacy wise with these drives and controler is going to be RaidZ-1, referred to as RaidZ in this particular benchmark suite.

ZFS Guru isnt mature yet but it looks VERY promising. And what I love about ZFS is that I can change operating systems across the board and simply import my VDEVS and POOLS into any other host OS that runs ZFS.

Lastly I will point out that I am very impressed with the performance of these 5900RPM Drives as per stated here...

Now testing RAID0 configuration with 1 disks: cWmRzmId@cWmRzmId@cWmRzmId@
READ: 145 MiB/sec 138 MiB/sec 126 MiB/sec = 137 MiB/sec avg
WRITE: 140 MiB/sec 114 MiB/sec 109 MiB/sec = 121 MiB/sec avg
raidtest.read: 75 74 81 = 76 IOps ( ~5016 KiB/sec )
raidtest.write: 66 66 70 = 67 IOps ( ~4422 KiB/sec )
raidtest.mixed: 40 41 52 = 44 IOps ( ~2904 KiB/sec )

This is not bad for a single disk running 5900 RPM with 64mb cache on a sata II connection.

Bao-Zhe (OP) it would appear that these Western Digital REDs are almost and in some areas faster than your 7200RPM drives comparing my graph below to your graph in the original post... Now of course this could be my drive controller "Intel C602". However the other graphs yours is pulling ahead slightly.

Here is mine... (Sequential Writes) How they are faster than reads is beyond me??? haha

_Gea · Dec 12, 2012

tangoseal said:
ZFS Guru isnt mature yet but it looks VERY promising. And what I love about ZFS is that I can change operating systems across the board and simply import my VDEVS and POOLS into any other host OS that runs ZFS.

One Problem:

If the disks were formatted in *BSD with GPT partitions (which *BSD recognizes but Solaris doesn't), you cannot import to Solaris.

So you must use GEOM on BSD.

FlangeMonkey · Dec 12, 2012

Thank you very much tangoseal, I cannot tell you how I've been wanting to see some results from a WD30EFRX for comparison.

They do however look extremely fast compared to the reviews I read. I'm curious how long the test ran and therefore sizes of files. I'm not saying it is, because I want these results, but is this hitting the disk or is ZFS/Memory caching to a degree?

tangoseal · Dec 12, 2012

FlangeMonkey said:
Thank you very much tangoseal, I cannot tell you how I've been wanting to see some results from a WD30EFRX for comparison.

They do however look extremely fast compared to the reviews I read. I'm curious how long the test ran and therefore sizes of files. I'm not saying it is, because I want these results, but is this hitting the disk or is ZFS/Memory caching to a degree?

These test are solid disk and bypassing all caching except what is on the disk (64mb).

As far as time taken... these took about 20 hours solid if not a bit more. I loaded up ZFS Guru Live CD and ran the disk benchmark advanced suite.

FlangeMonkey · Dec 12, 2012

tangoseal said:
These test are solid disk and bypassing all caching except what is on the disk (64mb).

As far as time taken... these took about 20 hours solid if not a bit more. I loaded up ZFS Guru Live CD and ran the disk benchmark advanced suite.

Thanks for clarifying.

bao__zhe · Dec 12, 2012

FlangeMonkey said:
Thank you very much tangoseal, I cannot tell you how I've been wanting to see some results from a WD30EFRX for comparison.

They do however look extremely fast compared to the reviews I read. I'm curious how long the test ran and therefore sizes of files. I'm not saying it is, because I want these results, but is this hitting the disk or is ZFS/Memory caching to a degree?

It is said in the text report: "Test size: 32.000 gigabytes (GiB)" which is the default. Due to how the spindle HDD is constructed the starting area of the LBA space has the best performance. Thus the test of 32GB is only meaningful for benchmarking and comparisons not for real life storage performance (unless you only use 100GB of your 10TB pool

.

@tangoseal: yes that does seem to be somewhat faster. what's the model of the Western Digital Drives? Mine is Seagate ST3000DM001 and i got them for $90 during those crazy days

Also my test is conducted within ESXi environment so virtualization layer and PCIe passthrough layer, as well as a controller separate from the southbridge (as you mentioned) will hurt the performance, although i expect it not to be very significant.

Billy_nnn said:
If you insist on ECC memory on the storage server, then you should really also have ECC memory on all systems which manipulate the storage server's data.
In many cases, dirty data will spend more time sat in the memory of client systems than it will sat in the storage server's memory, and the storage server is no more susceptible to random bitflips than the client systems!

good catch. my next step is to incorporate ECC into my desktop...somehow... right now my solution is to do BT rehash/CRC all within the server. I hope MELLANOX IB adapters have ECC RAM?

ZFS performance test with ZFSguru

bao__zhe

Limp Gawd

Billy_nnn

Limp Gawd

tangoseal

[H]F Junkie

tangoseal

[H]F Junkie

_Gea

Supreme [H]ardness

FlangeMonkey

Limp Gawd

tangoseal

[H]F Junkie

FlangeMonkey

Limp Gawd

bao__zhe

Limp Gawd