Performance issues with RAID5

[FrozenLiquidity]

I'm hoping that someone will be able give me a good idea of what is going on with the current issue I'm running into with my new RAID5 setup.

I recently built a new server using WHS2011 as the OS, with the following hardware:
CPU: Core i3 540 @ Stock
Mobo: ASRock H55M-LE
RAM: 8GB of Corsair XMS3 DDR3 1333MHz
RAID: Syba SY-PEX40016 (prime suspect )
Drives: The OS itself is installed on a separate drive and running perfectly connected to the motherboard.
The other drives are 5x 2TB Hitachi Deskstar 5K3000 HDS5C3020ALA632's

Despite the problems people have reported with this RAID card (which I realize is offloading work to the CPU) I was able to get it to work fine in RAID5 mode, showing 8TB of usable space, and 7.5 after the format using 16K stripe sizes.

Disk speeds and file transfer speeds are good. Sustained read and write speeds of 90-110 MB/sec - when it works. So here's the problem. While transferring some files, the drives or the raid card, stop responding, the CPU stops working and the transfer just stalls. The write speed drops from ~100MB/sec to essentially zilch, we're talking 2-3MB/sec, and I doubt if it's even doing that. I am able to transfer some files, a 1.5 GB movie here, or a few smaller files there, but often times with the files it will start out strong and just come to a grinding halt halfway through the transfer.

As far as I know, read speeds are not affected by this. This is difficult to verify though as I can't really get anything to read onto the drive when most of the transfers to it fail. However benchmarking software indicated that reading speeds were fine, while the write speeds were abysmal. I'm not sure what could be going on here, and I was hoping the excellent minds here at the [H] had some possibilities or ideas that they'd be able to share.

Any thoughts?

 

[FrozenLiquidity]

Some additional information:

I realize that this card does not have a supported driver for the WHS2011 OS, however I don't think that the driver is the culprit here.

Today I reset the array to JBOD and it seemed to work fine. Once in Windows I was able to transfer 50GB+ worth of files to the drives at a sustained speed of over 100MB/sec, which drive benchmarks supported (previously, on RAID5 I was encountering problems from the get go - a single 8GB file would fail). The only thing that had been changed was the operation mode of the Syba RAID card. Now, I'm not 100% sure how information across disks is written in JBOD; that is, if it stripes the data across all the disks simultaneously or if it writes to one disk until full, and then the next, and so on.

Either way, this indicates to me two places of potential failure here: first, if JBOD doesn't write to all disks, a potentially bad drive or sectors could be avoided until far later in the lifespan of the array, thus avoiding any read/write issues until those problems are encountered. The other possibility is the XOR calculations that are no longer present in the JBOD operation. I'm positive with this card that the CPU is doing those calculations, and that it is more than powerful enough to handle it, though throughout this process in RAID5, something, somewhere stalls.

Is there some BIOS setting for the CPU which could be affecting the parity calculations?
Given this additional detail, are there any suggestions that people could throw my way?

I'm not asking for expert opinions here, just ideas for things to try to narrow down the problem could be helpful. I'm sure some of you have some ideas...

 

[FrozenLiquidity]

I've probably reassembled this array over a dozen times now. Each and every time I've tried a RAID5 setting on the hardware it has displayed the abysmal writing speeds that I've come to expect... (one exception noted below)

Read speeds: ~80MB/sec
Write Speeds: <5MB/sec
Random Seek + RW: <3MB/sec

I've tried several sets of 3 drives in a RAID5 array with the same results. Only once was there an exception, and for a short while I had a properly functioning RAID5 array with 3 of the 5 2TB drives. When trying to reproduce this setup after additional testing, I was unable to. Again, in a JBOD setup with all of the disks, speeds are acceptable.

I can only make guesses at this point as to why this is performing as such...

I can only guess that you get what you pay for.

 

[TypeRazor]

You have answered your own question, your card is what we refer to as "fakeraid" and windows does not handle such well as you have seen. You either need to get a real raid controller or move to real software raid (ie *nix)

The reason the speeds are fine in JBOD is because the controller is doing nothing, its just acting as a HBA.

 

[FrozenLiquidity]

Yes, I already understand that the card is "fakeraid", in that it offloads the real RAID work to the CPU as opposed to handling it locally. I was under the presumption that an i3 probably wouldn't balk at that... Though you also seem to be pointing a finger at Windows too, and I have a hard time believing that it could be both. Perhaps a more detailed explanation is warranted?

The other solution I was considering, the Highpoint 2320 card; does the same thing (fakeraid). Are you saying I'm going to run into the same problem there? Frankly, a $300+ real raid controller solution holds little appeal to me at this time. For that amount of money I could nearly duplicate the setup I have now and keep the entire contents of the full JBOD array in cold storage as a backup rather than just a fault tolerant solution.

I'm willing to, and have been considering other hardware options at this point; although the OS will be staying and I'd like to get this working without tossing too much of what's already here out the window.

 

[drescherjm]

Yes, I already understand that the card is "fakeraid", in that it offloads the real RAID work to the CPU as opposed to handling it locally. I was under the presumption that an i3 probably wouldn't balk at that... Though you also seem to be pointing a finger at Windows too, and I have a hard time believing that it could be both. Perhaps a more detailed explanation is warranted?

Software raid 5/6 (even fake raid) does not have good performance under windows. I believe this is due to the way the stripe cache is handled. In most implementations on windows a stripe cache either does not exist or is not effective. Remember in parity raid without a stripe cache every write that is less than the size of the stripe the entire stripe needs to be read from disk then the parity calculated and finally then the entire stripe needs to be written back to disk. With the same exact hardware on basically any non windows os I would expect you to gave 150 MB/s reads and > 100 MB/s writes with a 3 drive raid 5 array.

 

[af22]

The "raid" features on that card are pretty much unreliable and should not be used. I use windows raid and it works well on my 4x2TB drives, its quite reliable and performs consistently. The only downside to windows raid, is that it doesn't notify you when a drive dies. And when i'm writing to my 4x2TB raid5 array, it pushes the cpu to 60%+ on my zacate e-350.

With my 4 drive array, i can push 200-300 meg/sec read, and a consistent 60 meg/second write (this may be cpu bounded, and will increase as more drives are added if you have the cpu power).

 

[FrozenLiquidity]

On that note, it might be nice to know what each of you are using for that, in that regard.

Given that the HP 2320 does essentially the same thing, would one generally expect the same results? No one has indicated that this one suffers from the same thing the current one does under working conditions...

 

[drescherjm]

and a consistent 60 meg/second write (this may be cpu bounded, and will increase as more drives are added if you have the cpu power).

I would consider that horribly slow but if you do not need write speed this may be acceptable to you. I can tell you that a linux software raid 5 with the same hardware will easily do over single drive speed for sequential writes. And it will do that at 12% cpu usage or lower.

 

[af22]

On that note, it might be nice to know what each of you are using for that, in that regard.

Given that the HP 2320 does essentially the same thing, would one generally expect the same results? No one has indicated that this one suffers from the same thing the current one does under working conditions...

I was in the same boat as you before i went software raid. Pretty much every $100 raid card is going to suck in raid5. Instead of trying to find a "affordable" raid card, you should buy a used "$300" raid card off ebay for $100. But, the only pain in the enterprise cards is that you may need to get those SAS to SATA cables which can add up depending on how many hard drives you have.