Horribly slow matrix raid

[-Dragon-]

I've got 4 hitachi 5k3000 2TB drives in an array on a C204 board, 50GB RAID10 volume for the OS and a ~6TB RAID5 storage volume.

Transfering a multi GB file over the network to the OS volume gives the expected results, 100-110MB/s for the entirety of the transfer limited by Gbit connection, resource monitor shows spiking active time fluxuating quite randomly from 0 to sometimes 90, maybe average around 30-50, hard to tell. System cache memory slowly increases ~2GB and plateaus there.

Transfering to the RAID5 volume is a different story however, transfer starts out the same from the network side however disk active time pegs at 100%, write IO is only ~20MB/s, and cache RAM increases linearly until the RAM is full. Once RAM is full the network transfer slows to 10-20MB/sec for the rest of the transfer and the whole system becomes very unresponsive during this time due to the high disk activity.

Is this normal for software RAID5 on an intel board? Frankly I haven't used software RAID in years and the areca in my other box probably has me a little spoiled with 300MB/s+ write speeds to a RAID6 (never actually benchmarked it but I have done a file copy from the drive to itself and it did over 150MB/s on the transfer so I'm sure it could do 300+ if it was just writing), and I know the drives aren't the fastest but this seems excessively slow. If it IS normal then I'll definitely switch to a RAID10 on the data partition as well, I just want to make sure there's nothing actually wrong first...

 

[Dan_D]

RAID 5 sucks on any onboard controllers. At least, the integrated chipset variety in consumer boards. I had a RAID5 array running on an Intel D5400XS using 4 x 150GB Raptors and immediately changed to a RAID 0+1 setup. The performance was horrible for HD video and games being run off the volume.

 

[-Dragon-]

Well this is for a VM server but still 20MB/s is pretty pathetic... and the 100% active time is what's really hurting things. Looks like I'll be backing up to server 1 and switching to RAID10

 

[Brahmzy]

intel onboard RAID5 = FAIL

Not even an viable option IMO. I just went RAID10 instead. But dammit, I'm running out of space.

 

[UhClem]

I've got 4 hitachi 5k3000 2TB drives in an array on a C204 board, 50GB RAID10 volume for the OS and a ~6TB RAID5 storage volume.

That's probably your problem, right there. The OS and the DATA volume are competing for the same disk head assembly (x4). You're "seek-thrashing".

-- UhClem

 

[Dan_D]

That's probably your problem, right there. The OS and the DATA volume are competing for the same disk head assembly (x4). You're "seek-thrashing".

-- UhClem

Good catch. I didn't see that the first time I read it. Even so, using a single volume and partition for the OS and all applications, I've experienced Intel RAID 5 as being too slow for general usage.

 

[UhClem]

Good catch. I didn't see that the first time I read it. Even so, using a single volume and partition for the OS and all applications, I've experienced Intel RAID 5 as being too slow for general usage.

Hah, and me just a "n00bie" -- damn silly-ass quantitative stereotyping. Feh!! (I was doing Unix kernel development before many of you existed.)

-- UhClem

[This space left blank, out of respect for those trying to get meaningful Google results.]

 

[Dan_D]

Hah, and me just a "n00bie" -- damn silly-ass quantitative stereotyping. Feh!! (I was doing Unix kernel development before many of you existed.)

-- UhClem

[This space left blank, out of respect for those trying to get meaningful Google results.]

I read a lot of posts. I miss things sometimes.

 

[-Dragon-]

There was very little disk activity before and after the tests and according to resource monitor not much else going on during them so thrashing shouldn't have been an issue

 

[Brahmzy]

It is a well known fact that intel onboard RAID5 is not really considered a serious option.

There are threads and threads on performance issues and the like. I mean 6TB 7200RPM would take almost a week for the chipset to perform a full "verify". Write speeds are absolutely dismal (sometimes in the teens.) A rebuild on a 2TB drive would probably take a week.

Why bother? Buy bigger drives n go R10, or get a 3rd party controller.

 

[Dan_D]

There was very little disk activity before and after the tests and according to resource monitor not much else going on during them so thrashing shouldn't have been an issue

RAID 5 via Intel's ICHxxx chipsets and Pxx Express series chipsets is pathetic. End of story. Again many people have tried it and it's never offered good performance. I sure as hell wouldn't run the OS off that. I tried it and found the thing hitching playing movies and games, and even opening files. It's horrid. I bought a MegaRAID 8308ELP at the time so that I could have a decent RAID 5 array. This worked much, much better.

 

[-Dragon-]

Yeah I'm convinced, will be switching it over to 10 tonight, extra storage was moderately nice but I don't really need more than 4TB on that machine for the forseeable future since it's not really the file server.

 

[UhClem]

There was very little disk activity before and after the tests and according to resource monitor not much else going on during them so thrashing shouldn't have been an issue

I was alluding to a different "flavor" of thrashing (not paging/VM related). Also, for all we know, it could be resource monitor itself that is causing this "extra" activity, especially in a VM context. It doesn't take a lot of "extraneous" disk accesses to totally fubar a transfer test.

Sure, switch your DATA array configuration, but first, it might be interesting, if you've got a spare random drive around, to clone that 50GB OS partition and run (the OS) off that separate spindle, and test the RAID5 DATA transfer without head-contention.

I'm not "defending" the Intel Raid5 implementation, just saying that it may not be the only (or even the worse) contributor to your poor results. It is seldom a good idea to have OS & DATA sharing heads.

-- UhClem

 

[Red Falcon]

That's probably your problem, right there. The OS and the DATA volume are competing for the same disk head assembly (x4). You're "seek-thrashing".

-- UhClem

+1

This would totally kill performance. There should only ever be one RAID array per set of drives. Wow, I'm surprised the OP is getting the speeds he is.

 

[-Dragon-]

Just as an FYI since moving to a RAID10 on the second half of the volume instead of RAID5 my write speeds are around 110MB/s. So yeaeh, why is RAID5 even an option on these systems other than a check box on a feature list...