• A Great friend to the HardForum with a great kid that he is trying to get a scholorship to continue his schooling. Please give hime a vote! Only 24 hours left! Thanks.
    If you have an VOTE FOR KEENAN!

LSI 9260-8i RAID 5 Help Needed

B

btwiggles2

n00b
Joined
Jun 9, 2011
Messages
10
My RAID 5 array isn't quite performing as I'd hoped. I'm looking for any guidance or suggestions. This array is used for storing BR-Rips and TV Series, all files are at least 1GB, most closer to 10GB. I was expecting write speeds for large file transfers to be more inline with the read speeds I'm seeing for transfers that are greater than the stripe size. I realize XOR calculations are going to bring down write speeds, but this doesn't seem right.

System Specs
  • MSI H67MA-E35 with B3 stepping
  • Intel Core i3-2100
  • LSI 9260-8i with firmware 12.12.0-0036 (if I upgrade to latest firmware, card won't allow
    BIOS to get passed DMI pool check)
  • 4 GB Corsair XMS3 DDR3
  • 5 Samsung Spinpoint (EcoGreen) F4 2TB's, with updated firmware

RAID5 is setup with 256kb stripe size. Initialized as GPT partition in WHS 2011, formatted with 64kb allocation unit size.

ATTO on a single drive RAID 0 and 5 Drive RAID 5, No Read/Ahead, Write Through, Direct I/O
ATTO.jpg
 
Do you have the BBU for your raid card? Most HW raid will default to not caching writes if you do not have that installed and that will lead to very slow writes.
 
I don't have the battery backup. But I'm intentionally showing the bench without the write-back cache enabled to show the slow write speeds in the RAID5 as compared to a single drive. I realize enabling cache will improve the write speeds, but only to a certain point. The drives themselves, without the RAID cache enabled, should be capable of higher write throughput, or at least that's what I'm hoping someone will confirm.
 
I realize enabling cache will improve the write speeds, but only to a certain point.
Click to expand...

The write speed should be 3 to 4 times what you get. I get 300 to 400 MB/s writes on linux software raid with similar drives on 4+ year old dual core athlons.
 
The OP had it in his post. He needs to enabled Writeback to use the cache. The 9260 does it without a BBU. Try going into the webbios or using the Megaraid software interface to edit your virtual drive properties and change to read ahead, writeback and cached I/O. Your current settings is for an SSD not for HDD.
 
I'm in complete agreement, enabling the cache will increase the write speeds dramatically.

But humor me, if I was running a software RAID, or didn't want to enable the cache in order to avoid corruption, shouldn't I be able to get higher write speeds in a 5-Drive Raid5, knowing that a single drive is capable of close to 150 MB/s writes?
 
RAID 5 is actually pretty slow for writes. I don't know how it compares to a single drive but it will definitely not scale linearly like RAID 0 would.
This is because the same data needs to be written to two drives for redundancy and which drive in needs to be written on needs to be determined by an algorithm computed by a controller (or the CPU for software RAID). This adds latency and it could possibly make it slower than even a single drive if the latency is excessive. The more drive the more latency as the computation gets more complicated and it may outweight the gain of being able to stripe the data over several drives in parallel.
 
Rafale said:
RAID 5 is actually pretty slow for writes. I don't know how it compares to a single drive but it will definitely not scale linearly like RAID 0 would.
This is because the same data needs to be written to two drives for redundancy and which drive in needs to be written on needs to be determined by an algorithm computed by a controller (or the CPU for software RAID). This adds latency and it could possibly make it slower than even a single drive if the latency is excessive. The more drive the more latency as the computation gets more complicated and it may outweight the gain of being able to stripe the data over several drives in parallel.
Click to expand...

That is all wrong. Ignore it.

RAID 5 works much like RAID 0, except it interleaves a parity chunk for every (N-1) chunks written. Computing the parity (XOR) takes almost no time at all. RAID 5 should be close to RAID 0 speed for large sequential transfers (technically, 1 - 1/N of the RAID 0 speed).
 
btwiggles2 said:
I'm in complete agreement, enabling the cache will increase the write speeds dramatically.

But humor me, if I was running a software RAID, or didn't want to enable the cache in order to avoid corruption, shouldn't I be able to get higher write speeds in a 5-Drive Raid5, knowing that a single drive is capable of close to 150 MB/s writes?
Click to expand...

It should be possible to get much higher speeds than you are getting. I'm not familiar with the details of the 9260, but it should be more than capable of several times the speed you are seeing. Maybe someone familiar with the card will chime in.

Could you post some more benchmarks? Maybe AS-SSD, HD Tach, etc.?
 
Rafale said:
He needs to enabled Writeback to use the cache. The 9260 does it without a BBU. Try going into the webbios or using the Megaraid software interface to edit your virtual drive properties and change to read ahead, writeback and cached I/O..
Click to expand...

I did as suggested and as expected, the write performance, as well as reads, is greatly increased. However these speeds shown are for relatively small transfers. If I'm writing a 10GB BR-Rip, the effects will be diminished as the 512MB cache will fill up and the HDD's, at their current write speed, will be unable to write-back the data as quickly as the cache can be replenished.

RAID5 with Cache enabled
Raid5%20Cache.jpg


8GB Transfer from Single Disk Raid0 to RAID5
transfer.bmp
 
Yup, all correct. When the cache runs out you basically are back where you were.
What do you mean by single disk Raid0 as your source drive? I am asking because your single drive might be your limitation on your file transfer.

To John, what you are saying is correct in theory but not in real life. RAID5 writes are much slower than you would expect because of latencys. Different controllers will behave differently and software RAID is better because of lower latencies. Without cache I have never seen it anywhere near RAID0 (n-1) performance... maybe it is because I am dealing with SSDs and the latencies become a big factor.
 
Even if RAID 5 doesn't scale linearly like RAID0, writes shouldn't get worse. With the cache disabled writes are under 80MB/s, almost half of what I see using the 2TB by itself, or the speed shown in the file transfer after the cache fills up.

Single disk RAID 0 is using the drive connected to the RAID card. You get the benefits of the cache but no redundancy or striping, you have to define a virtual drive to use a disk connected to the 9260 and RAID0 is the only option when only selecting 1 drive. In my case it's the most direct way to compare the RAID5 write speeds with a single disk, i.e. connected to the same controller just not a member of a RAID array.

What am I missing?
 
Rafale said:
I am asking because your single drive might be your limitation on your file transfer..
Click to expand...

This is the "Single Disk Raid 0". The disc is identital to the RAID5 member disks, Samsung Spinpoint F4 2TB, easily capable of reading at 130-140MB/s. Surely the RAID 5 should be able to write as fast as it can read from one of these drives.

single.bmp
 
I see. The file transfer on the read side might also be running out of cache... Because both your virtual drives are running on the card and they share the cache. This is specific to your file transfer test though.

I agree that 5 drives should not be slower than 1 in writes in RAID 5. you should get 2-3x 1 drive.
I guess you have NCQ enabled? Something else is wrong but your array seem to work as expected with cache enabled.
 
Rafale said:
To John, what you are saying is correct in theory but not in real life.
Click to expand...

Maybe in your life, but if you configure things correctly, it is possible to achieve results close to what I wrote. I have done it many times.
 
btwiggles2 said:
This is the "Single Disk Raid 0". The disc is identital to the RAID5 member disks, Samsung Spinpoint F4 2TB, easily capable of reading at 130-140MB/s. Surely the RAID 5 should be able to write as fast as it can read from one of these drives.
Click to expand...
Would probably be a better test to configure a ram-disk to read from. Of course, getting a large enough ram-disk to be bigger than your cache could be a problem.
 
john4200 said:
That is all wrong. Ignore it.

RAID 5 works much like RAID 0, except it interleaves a parity chunk for every (N-1) chunks written. Computing the parity (XOR) takes almost no time at all. RAID 5 should be close to RAID 0 speed for large sequential transfers (technically, 1 - 1/N of the RAID 0 speed).
Click to expand...

I fully agree with all of this. The XOR does not take a lot of time with modern hardware.
 
Was your parity done being initialized? You're write speeds will be much slower if it's not.

Here's what I was getting 7k2000's single drive and 5 X RAID 5. on an older LSI 1078 based controller(I forget which model as I had 4 different models that used this chip at the time).

2tbx1.png


2tbx5.jpg
 
btwiggles2 said:
What is your stripe size? Did you format with the default 4kb size in Windows? What about your Raid Contoller settings, read ahead, write policy, I/O Policy?
Click to expand...

settings.gif


Windows format is GPT/4kb Cluster.

7628669 MB total disk space.
1126989836 KB in 7665 files.
2984 KB in 684 indexes.
0 KB in bad sectors.
312803 KB in use by the system.
65536 KB occupied by the log file.
6527785 MB available on disk.

4096 bytes in each allocation unit.
1952939519 total allocation units on disk.
1671113114 allocation units available on disk.
 
You must log in or register to reply here.
Back
Top