Recommend me a RAID card/setup

[nytrackstar]

I'm waiting to build my ultimate rig at the end of this summer with conroe/g80...I need help figuring out my hdd configuration.

I'm willing to throwdown a large amount of money on this...

I'm looking for the fastest possible configurations...

As far as storage i'm probably going for 3 of the upcoming seagate 800Gb hdd's in a raid 5 array, and for windows maybe 2 or 3 150gb raptors in raid 0/5?

does raid 5 suffer in speeds/throughput at all? I've tried to do some reading on the topic but would like to hear some real world knowledge and opinions from people that know. other Raid options that might work? keep in mind im mostly interested in speed.

last thing would be, what raid card will be best for this scenario?

Thanks in advance for any help!

 

[hokatichenci]

If budget isn't a concern, definitely go Areca. It has raid upgrading/downgrading/expanding, so you could start out with raid0, get another disk and go raid5, get another and expand, then buy another and upgrade to raid6, etc. Real hardware raid isn't as slow as software raid since everything is done on a processor on the raid card as opposed to by the CPU. If you really have no budget I'd suggest going with something like 150gb raptor's in RAID1 and then spreading across larger files on some larger disks in RAID5, which would provide a lot of speed on the O/S with redundancy and a second volume with a lot of space.

 

[unhappy_mage]

Ooh, no budget. /me opens ZZF...
2*Raptor 150 in raid 1 or as seperate drives. Raid 0 is a crock. ~$300 each. Subtotal: $600.
If speed is your main concern you don't want Seagate drives. That simple. Maybe they'll fix their huge seek time/buffer optimization problems, but if the 7200.8->9 transition says anything it says no. The WD 3200KS are some of the fastest non-raptor drives around, and at a decent capacity point. Say 8 at $140 each. Subtotal: $1720.
The Areca 1230 is a serial ata, 12 port, pci express (note: this means find a board with an extra x8 physical (probably x4 logical) slot - these will probably be more common with Conroe) at about $740. Subtotal: $2460.
Then, you'll need a power supply to deal with 10 disks, a dualcore chip, 4gb of ram, and dual fast video cards (I'm guessing here ). This is a job for the PSU forum guys, but I'm gonna say the PCP&C 850 will be a good pick for the job. About $450.

That brings the total for this uber disk system to about $3k. And I didn't go all out

As for speed, raid 5 will be slower by a good deal, but it gives you redundancy - you can have a disk die and not lose any data - and it only loses you the capacity of one disk. Raid 0+1 would be *hella* fast on that array+controller (and I'd loooove to see some benchies ) but it'd lose you half the capacity. 8 disks for only 1.2 TB doesn't strike my fancy. 8 in raid 5 would give you about 2.1, to compare. Raid 6 will lose you the capacity of two disks (1.8 TB) and it's a little slower than raid 5, but you can lose two disks and not lose data.

This is a lot to throw at you at once. Ask questions, even if you're pretty sure they're dumb. Lots of questions.

 

[hokatichenci]

8 in raid 5 would give you about 2.1, to compare. Raid 6 will lose you the capacity of two disks (1.8 TB) and it's a little slower than raid 5, but you can lose two disks and not lose data.

Once I can figure out how to get good benchies from Linux I can post exactly how fast my 1.8tb raid6 array is, which seems to be plenty fast at least for all the stuff I'm doing if the OP is interested.

 

[nytrackstar]

wow, lots of information here...hmm where to start? okay, the reason for me wanting to go with the seagate drives is because my father's company is one of the larger seagate distributors in the country, so i would be able to buy those drives at cost to him, which is a good deal cheaper than any other place, i would probably be able to get the WD's at a lower pricepoint than online retailers as well, and seeing how much cheaper those drives are it would probably be worth going for a number of those instead of the seagates...

i dont have much of an issue with my storage arrangement being in RAID-5, that is probably the plan because the only real usage is going to be at first when i move all my media onto it, i dont need more than 400 to 500gb's of storage for my media now, i would like to have atleast 1 to 1.5 tb so i have room for growth, my bigger concern is my windows drive, im looking for extreme speed and redundancy, not sure if possible, but maybe a RAID 0+1 with the older 74gig raptors? not sure if i can justify doing that with the new 150's since i wont ever come near ~300gb's of space on my windows drive...

thanks again for all the help, i really appreciate it!

 

[unhappy_mage]

Yeah, I agree - the raid 5 will be fine with either drive, get what's cheaper and use raid 5 and maybe a hot or cold spare. If you're going to buy a few drives now and expand as you need it, buy drives that are expensive per GB - prices will drop, and you'll be pretty much locked into whatever drive you buy when you go to expand - mixing drives is bad.

The interesting part of this is definitely the OS volume. At the ~$600 level you're looking at, with controller taken care of and an adequate PSU and ventilation, you have a fundamental choice - 2 raptor 150s or 4 raptor 74s. You'd probably want to run the 150s in raid 1 for a greater number of requests possible to handle at once, and the 74s in raid 0+1 or 10 for a larger number of requests and higher STR - you wouldn't hit a queue depth of 4 often enough on desktop to make 4-way raid 1 useful, I think. However, almost nobody has run desktop/gaming-type tests on raid 0+1 that I know of (it's all the rage in the DB world, though ) so I don't know that I can recommend it. I'd stick with 2 150s in raid 1 for best speed, least stress on components, and leaving an extra pair of ports open on the sata controller.

hokatichenci: Try bonnie++. It's a filesystem-based test, so XFS will be good for you The other interesting test is 'time dd if=/dev/zero of=/mnt/array/tempfile bs=64k count=160k' - this tests how long it takes to write a 10GB file. I would run these tests on my array for comparison, but it's 99% full and that hurts XFS, so I'll desist until I buy more drives.

 

[hokatichenci]

I'm not really sure how much of a speed benefit there would be over raid10 74gb vs raid1 150gb, but I applaud you for at least being concerned about redundancy I personally would say go for raid1 150gb's just because it takes up less drives, and raid0 from what I've seen hasn't really been a huge improvement for day to day tasks (as opposed to heavy vid editing). As for raid5/6, speed on that array will be fine as long as you get a card with real hardware raid. Expansion definitely won't be a problem as long as you get a card with those features, which any good raid card should do. I personally expanded my Areca array from raid5 4x drives to raid5 5x drives, then to 8x drives RAID6. Worked great and all I had to do was either expand my partition (which was reiser so I couldn't) or just add a new partition. If you went with a 12 or 16 port card you would have enough ports to probably last a lifetime, and if you have separate system drives you could safely do raid expansion without even having to reboot.

My other suggestion is that you might want to look into hot swap cages, since it'll make working with the drives a lot easier. Plus if a drive goes down I don't have to do shut down, I just pop the bad drive out, put in the replacement when it comes and I'm golden

 

[hokatichenci]

hokatichenci: Try bonnie++. It's a filesystem-based test, so XFS will be good for you The other interesting test is 'time dd if=/dev/zero of=/mnt/array/tempfile bs=64k count=160k' - this tests how long it takes to write a 10GB file. I would run these tests on my array for comparison, but it's 99% full and that hurts XFS, so I'll desist until I buy more drives.

real 0m47.865s
user 0m0.132s
sys 0m20.361s

Thats onto my XFS partition, its surprisingly slower onto my reiserfs partition (~1min16sec).

If you're going to buy a few drives now and expand as you need it, buy drives that are expensive per GB - prices will drop, and you'll be pretty much locked into whatever drive you buy when you go to expand - mixing drives is bad.

I hadn't thought of that, and since its such a damn good idea I 2nd that notion.

 

[unhappy_mage]

real 0m47.865s
user 0m0.132s
sys 0m20.361s

Thats onto my XFS partition, its surprisingly slower onto my reiserfs partition (~1min16sec).

That's 214 MB/s in (mostly) sequential writes. Even on reiser, 135 MB/s isn't small beans. XFS is probably so much faster because it keeps track of a list of free blocks sorted by size and another list sorted by location. So when you ask for a block of a given size to write to, it doesn't have to go look on disk, it has a list of where to go all ready. That means writes happen quickly. How about reading it back off? 'time dd if=/mnt/array/tempfile of=/dev/null'

bonnie++ gives much better results, though - it also does small file tests.

 

[hokatichenci]

How about reading it back off? 'time dd if=/mnt/array/tempfile of=/dev/null'

real 0m48.626s
user 0m3.716s
sys 0m32.166s

I tried running bonnie++ when I first got the array but I had problems with it getting to run, not to mention it turns out one of my drives was bad. I'll try and figure out all of the options and get some stress tests done and post a full review of the card at some point.

 

[Rocco123]

Just a couple of suggestions.

If money is not an issue, have you thought of going SCSI? Seagate makes an awesome 10k 300GB U320 drive. Of course, you're not going to have balls out capacity like a couple of 800s, but 4x300 in RAID 5 is still in the high 800 GB range. This of course would be many times more expensive than SATA (each of these drives are 8-900), but you have the reliabiltiy and performance of U320. These drives are built to last.

There are many good U320 RAID cards from intel and LSI, all around the 500-700 range for a decent one.

Also, if you are going for SATA RAID 5, you might want to choose a nice server case with a drive cage.

 

[nytrackstar]

alright, so looks like im going to go for 2x150gb raptors in raid 1 as my boot drive, and 3x800 for my raid-5, now i am able to have more than one set of raided cards on that areca that was recommended, right? like, both the raid 1 and raid 5 would be on the same controller card...

 

[Rocco123]

I'm assuming that the Areca they are recommending is this one:

http://www.newegg.com/Product/Product.asp?Item=N82E16816131006

or

http://www.newegg.com/Product/Product.asp?Item=N82E16816131003

The difference being the top card has 12x ports and has more memory.

And yes, this card should be able to support multiple arrays (any half-ass RAID card can do this).

My only beef with these cards is that there is no slot for the RAM, which means you couldn't expand this. Most of the decent SCSI cards have a slot to upgrade the RAM module, like this one:

http://www.newegg.com/Product/Product.asp?Item=N82E16816118021

Oh well, just a minor issue.

 

[unhappy_mage]

Rocco123: Yes it does. That's a stick of notebook ram onboard the 12x card there. It also has a network interface, for managing the card remotely. Beat that.

alright, so looks like im going to go for 2x150gb raptors in raid 1 as my boot drive, and 3x800 for my raid-5, now i am able to have more than one set of raided cards on that areca that was recommended, right? like, both the raid 1 and raid 5 would be on the same controller card...

Where have you seen the 800 GB drives? I haven't heard anything about them, and not even searching google or seagate turn up anything about it. Source please? I think you might be best off with Maxtor or Hitachi 500 GB disks right now, although they're sure to come out with new products by Christmas.

Edit: You might want to get some enclosures like the CSE-M35T from Supermicro for maximum awesomeness. They'll keep the drives cool and make cable management easier.

 

[Rocco123]

I stand corrected, I didn't even notice at a glance. Doesn't specify wether it is ECC/registered or not. Most RAID memory modules are ECC though.

 

[unhappy_mage]

It is; notice there are five chips visible? ECC is done with an extra chip to store parity bits, so there are 9 or 18 instead of 8 or 16 chips on memory with ECC. They thought of everything with this card, I swear.

 

[nytrackstar]

well, my dad's company ( www.cti-intl.com ) has very close ties with seagate, my pops told me that the 800gb drives havent been announced yet, but he's getting the first shipments in within a month or so...

 

[Rocco123]

Yeah it does look pretty impressive. It looks like it even comes with an extra heatsink? Also looks like he'd have more than enough SATA cables

 

[nytrackstar]

how loud are those drive racks? this system is going to be watercooled since im going for as quiet as possible, loudest thing is probably going to be the 1kw pcp&c...

 

[Rocco123]

well, my dad's company ( www.cti-intl.com ) has very close ties with seagate, my pops told me that the 800gb drives havent been announced yet, but he's getting the first shipments in within a month or so...

That's strange. I haven't heard of that either. I'm suprised at least a little info hasn't been released, 800 GB would be a huge leap forward in HD storage.

 

[unhappy_mage]

Yeah it does look pretty impressive. It looks like it even comes with an extra heatsink? Also looks like he'd have more than enough SATA cables

Funny story - when I bought my sata card (8 ports) I also ordered 8 sata cables so I'd never have to worry about buying more. So of course, when I opened the box for the raid card... 8 more sata cables So now I'll *never* have to worry about buying more.

The racks are probably pretty quiet; I don't own one yet (tho I'm working on it ) but worst case you can replace the fan with a quieter one. I'm guessing even at $120 they give you a cheap fan They're nice, though - Ockie owns four (I think those are this model...), maybe he'll stop by and post his experiences.

 

[nytrackstar]

ill be at the office today, so ill try and gather more info on them...

 

[nytrackstar]

i wanted to thank everyone again for all the help!

 

[hokatichenci]

I stand corrected, I didn't even notice at a glance. Doesn't specify wether it is ECC/registered or not. Most RAID memory modules are ECC though.

The 1210 and 1220 have 128mb of onboard RAM, and no network port, they cannot have their RAM expanded. The 1230, 1260, and 1270 (12, 16, and 24 ports respectively) all have an onboard NIC and expandable RAM (ECC up to 1gb, 2gb maybe?). All of the models also have a battery backup module. One of the other things that stands the Areca apart (at least my feeling is this) is their API, which directly accesses the controller so that you can write your own tools. Once I figure out how to get the tests running properly on this system I plan on writing some simple command line tools for use in shell scripts.