Should I do Raid 5?

[rottweiler]

Ok, this is probably the 5th time or so I've changed my mind, so I guess that's why I'm asking about Raid 5. jk.
Ok, after reading a ton of stuff here, I decided that Raid 0, was not the way to go, and was about to order more hard drives and a controller to do Raid 0+1.
This might be because I didn't really understand what Raid 5 was.
After talking to my roomates, I think I get it.
And if so, it seems really awsome.
Is Raid 5 really that great?
It seems as though you get N-1 drives of space (N being the number of drives in the configuration), and they're basically fully backed up, as long as only one drive fails and you replace it before another fails.
And this rebuild is done on the fly!!! Amazing.
The XOR of the data written to one drive is written to another drive in the setup, and all drives are used simultaneously.
Thats basically what I got out of it, someone else can explain in detail if they want.
I'd like to know is there a big difference that would favor Raid 0+1 over Raid 5?
It seems to me that Raid 5 is the way to go.
I already have 2 200GB Seagate drives, so 1 more and a good controller and I can set it up, right?
Also since this wasn't answered in a previous thread, what would be a good controller to get for this setup?
Getting the same storage space and only getting 1 drive instead of 2 saves me about $140, so I could definately get a better controller card.
Though 3Ware still seem too expensive.
Newegg does have a Highpoint for $68 that can do Raid 5.
The only thing it seems to be missing is 66MHz PCI 2.2 support.
And the cheapest comparible Promise is $168 sine theirs for $123 doesn't support Raid 5.
But the Promise cards are 66MHz PCI 2.2 compliant.
I have the MSI Master2-FAR, so I don't have PCI-X.
So I wouldn't need the 66MHz cards, right?
If there are better options, please let me know.
But the cheapest 3Ware from newegg is $339.
Is it really worth that much for their name?

Thanks

 

[Konstantinnovation]

If you want a RAID5 card then expect to spend money on it. The "expensive" cards do all of the calculations on the card, the cheaper ones would require your processor to do most of the work, which both slows down the performance of the array, and your system as a whole. Don't forget you get what you pay for or less... so dont expect a 150$ card to work miracles. The reason more people do not use RAID5 is because of the huge overhead cost of the card.

My advice is NEVER to buy "cheap" parts, I guarantee you will regret it in the end(i always did). If the cost is too high then you should re-evaluate your goals, but do yourself a favor and don't buy one of the budget raid5 cards. I just paid over $450 for an 8 channel 3Ware and it hurts now, but in the longrun It is worth it.

 

[dandragonrage]

I think you should tell us why you want RAID in the first place. If you are sure that you want it, however, I will also say that you'll regret purchasing a cheap card if you want RAID 5.

 

[J]

I agree 100% with sickpuppy. If you buy crap know then you will pay for it with performance and stability down the road.

Rebuild on the fly is great, but I have had the raid controler take a crap and take out the whole array anyway. No rebuilding on the fly there.

Are you really sure you want RAID5. You wont see much of a speed improvement with speed as far as the operating system booting up. You will see a speed improvent as far as when you do large sequential reads and things of that nature. You also get the added benifit of redundancy.

 

[rottweiler]

My machine will be for DVD ripping, video encoding, a basic web/ftp server at some point, and some gaming.
I was thinking that for the large DVD images and video files that raid would be helpful.
But I also want the redundency feature.
I will be bringing my work home, at least for backup on my main machine.
So in the unlikely event that my work laptop and one home harddrive fail, I'll still have the data.

I think you have convinced me to invest in a good controller, but the only half suggestion was a 3com card, and I can't really find them anywhere.
I checked their website, pricewatch, and newegg.
Nobody seems to have 3com sata raid controllers.

Should I just go with the 3ware on newegg?
whats the difference between the Escalade 8506-4LP and 9500S-4LP?
From 3ware's site, it seems the 9000 series is better, as the model would imply, but will this be anything noticable/useful to me?

Thanks

 

[dandragonrage]

3com? I think someone MEANT 3ware....

 

[rottweiler]

That would make sense, since newegg has an 8x one for $400.
Honestly an 8x one wouldn't be bad, since I could add more than 1 drives to it down the road.
Also, are all the 64bit ones 32bit compatible?
Some of the other manufacturers said 32/64, but all the 3ware seem to only say 64.

And how hard is it to move a raid 0 setup to a raid 5?
I'm gonna have to format everything won't I?

Thanks

 

[J]

looks like on the 9000 series you can upgrade the onboard memory and on the 8000 series that isnt possible. did'nt see anything that jumped out but I am sure there are some other differences.

 

[Zuht]

You should be aware that writes on a RAID-5 array will be kinda slow. Video work would probably suffer because of that. It's just dandy for archiving video, music, DVD images, etc, but it's no fun running the system and I/O hungry apps off of one (been there, done that).

 

[Ice Czar]

RAID 5 is ideal for a large amount of storage with redundancy
and keep in mind that as you increase the number of drives the effciency increase
3x100GB = 200GB
5x100GB = 400GB
8x100GB = 500GB
12x100GB = 1100GB
but you typically cant dynamically add drives and would need to rebuild from scratch

and while reads are quite good, writes are not do to the parity calculation, it great for storage but isnt a performance array

that is where RAID 0+1 has the advantage, but its capacity effciency suxs at 50%

a few benchmarks (SATA & SCSI) in Dutch, but the mainbenchmark is built with IPEAK SPT just like StorageReviews testbed
http://www.tweakers.net/reviews/400
http://www.tweakers.net/plan/233


Considering the large file access pattern (if thats all that is being addressed) there might be an alternative
cut & paste
---------------------------------------------------------------------------------------------------------------

Well, tell us anyway. Might be useful for another project

Oh its that Netcell SynchRAID, an optimized RAID3 in effect
http://www.netcell.com/technology.htm
download the Whitepaper at the bottom
"SynchRAID Removing RAID Adapter Bottlenecks" (PDF)
they come in a 3 channel and 5 channel in PATA or SATA
but at 32bit 33MHz only
specifically aimed at workstation and desktop, for CAD and Video\Image Production

Data Paths and Word Level vs Block Striping
SyncRAID supports a wide data path to and from the drive array, which is an integer sum of the individual widths of the individual drive I/O interfaces. This allows much Higher data throughput, especially when streaming large data files on and off the disk as in emerging 64-bit CPU desktop and workstation environments.

Figure 1 illustrates how 64-bit data word destined for the drive array in a SynchRAID controller is handled versus in a legacy controller. For a legacy RAID 16-bit ATA implementation, a block of 64-bit words from the host bus is divided into a stream of 16-bit words and issued to individual drives until the block stripe is full or the block write complete. If the block occupies more than a single drive stripe block or "chunk" then it moves onto the next drive to start using another stripe block until the block transfer has completed. As we shall see later, the parity generation or update method very much dictates the performance of array write operations.

SynchRAID, due to its innovative synchronization scheme, removes this data and blocking arrangement altogether, keeping the data words intact as 64 bits all the way to the drives. This effectively creates a 4x-speed ATA interface.*(with 4 drives) Applying the concept to multiple SATA drives, the interface becomes a 4x SATA interface, and so on.

of course if the data transfer falls below 64bit....
you see why its sort of "special"

that of course doesnt address evn half of it, but is a taste
effectively its supposed to be RAID 0 performance with redundancy
Tom's reviewed it, but then it was Tom's that reviewed it

 

[rottweiler]

Wow, that's very interesting.
Do you think it actually lives up to those statements?
It's not too expensive either, especially if it's supposed to be much better than Raid 5.
Is Raid 5 really that slow in writing?
I figured it'd be slower, but since it was still writing to multiple drives at once, I figured it'd be fast enough.
Maybe I need to do some more research before I decide on what I really want.
Obviously I want what everyone wants, max storage, speed, and reliability.
But since I don't like asking for the impossible, I'll look around and see what fits my needs.
Thanks for that info Ice Czar and others too.
I didn't realize I'd be taking such a performance hit with Raid 5.

 

[Ice Czar]

.
Is Raid 5 really that slow in writing?

those Dutch Benchmarks tell the tale as both RAID 0 and 5 where tested
along with a single drive

as far as SynchRAID goes, preliminary info I have suggests yes it works
but Ive yet to see exhaustive testing, and if your trying to use it with too small a file throughput, its not going to be the solution

personally, Id consider it for a dedicated "editing" performance array
but leave the big storage array to traditional RAID 5

you really dont need a performance array to serve up video or other media
or even to capture it most of the time (depends on the resolution) but for editing
thats where you need it, but having a huge capacity array for that is putting alot of storage space through its paces

just as typical is a large storage RAID 5 and a smaller RAID 0 for actual editing
which is only employed transitionally, if redundancy is required the 0+1 is typically employed

 

[Konstantinnovation]

Another noteworthy difference between the 8000 and 9000 series is that 3Ware plans a firmware upgrade to the 9000 that allows for upgrading the size of the array WITHOUT WIPING THE DATA. This is important if you dont have a way to back up a 500-1000gig array to upgrade.

 

[Ice Czar]

cool thats typically external RAID enclosure stuff

 

[M0rph3us]

Okay, my recommendations: Get a good RAID5 card. Like 6 or 8 ports. I only buy Adaptec, but that's me. Anyway, get the 2 drives (for now if money is an issue) and RAID1 them. Storage. Get 2 Raptors instead of 200GB drives, and RAID0 them. swap. Keep all your Work in Progress on the Raptors, and archive to the 200GBs. Or if you have the $$ now, get 2 or 3 more 200GBs for RAID5 storage. If not, that's why you got a capable card with room to grow.

That would be the best way to do disk intensive vid editing, etc. That's how I would do it in your situation. If you mess it all onto one drive, it'll be way fragmented from all the editing work, and overall will be slower than the Raptors/storage setup.

 

[Qwertyman]

my brother sells servers with raid 5 and he says they get over 900mb per sec which i find very hard to believe.

 

[M0rph3us]

uhh, that's REAL REAL fast. Like 12-16 drives on a 4-channel u320 controller. PCI-X mind you. A little outta his league, but fun no less. Very fun. Installs Win2k3 in what? 3 minutes?

 

[Ice Czar]

yeh but try to actually use that kind of throughput
definately requires a multiuser environment

a single user with the most demanding single ap out there would be hard pressed to use half a 64bit 66MHz PCI slots bandwidth which is 532MB/s

PCI-X 64bit 133MHz comes out to 1066MB/s

 

[rottweiler]

Ok, I think I've decided, please criticize some more.

Although SyncRaid looks good, I don't think I'll go that route.
I think I might get 2 74GB Raptors and Raid 1 them for my OS and critical data.
Newegg dropped the price and has a $20 rebate.
Then get the 3ware card, possibly 8-port, and another 200GB Seagate and raid 5 those for backup/storage.

Then hopefully they'll come out with a firmware to add drives w/o destroying the raid, and I can add more drives down the road.
I'm guessing that's what this line from 3ware means:

Online Capacity Expansion ready (available Summer 2004)

I'm thinking the performance of Raid 1 should be good enough for video editing, right?
So I'll edit on the raptors and backup to storage.
Or I could just get 1 Raptor and do a software backup to the Raid 5 every week or so.
Or Raid 0 the raptors and backup weekly.

Obviously this will be a lot more money than I originally planned on spending, but I think the performance and future expansion would be worth it.
The only big problem is what happens when I want to get a new system.
Will I have to destroy and rebuild the Raid5 setup with a fresh windows install?
Or could I backup the Raptor(s), rebuild and install windows on them, install drivers for the 3ware, and then add it to the system with data intact?
Let me know.

Also does anyone know the difference between 3ware's 9500S -8, and -8MI?
Thanks

So what do you think?

 

[SEALTeamSix]

Ok, I think I've decided, please criticize some more.

...Or Raid 0 the raptors and backup weekly.

So what do you think?

If you're going to do Raptors + more storage, I'd put those Rappies in RAID 0, and then you'd just backup to your other drives (presumably a redundant RAID array of some sort). Raptors are for speed, so why not take advantage of it, eh?

-SEAL

 

[Ice Czar]

If you're going to do Raptors + more storage, I'd put those Rappies in RAID 0, and then you'd just backup to your other drives (presumably a redundant RAID array of some sort). Raptors are for speed, so why not take advantage of it, eh?

-SEAL

RAID 0 Raptors would increase the Sustained Transfer Rate, which I dont really think he'd need considering the aps, the single Raptors being sufficient for Video Ripping, Encoding ect

but more importantly it would increase the access latency, so considering the application that might be a bad trade

if it was video editing Id change that assessment
of course a few descriptors doesnt really impart a typical usage pattern,
and how someone percieves performance can be very subjective
one person putting more value on say load times of games (which would dramatically improve) over the random access of data in another application.

which is why you get 2 individuals with diametrically opposed opinions of the "value" of RAID 0 yet they have identical rigs

 

[rottweiler]

So even in Raid 0 reading is slower than a single drive?
If so then I think I'll just get 1 raptor.
And that will be my OS/gaming/editing drive.
And the 3x 200GB Raid 5 will be my data storage/backup drive.
Hows that sound?

Also I remember reading in another post about the best setup for partitions.
It was probably by you Ice Czar.
Something with a seperate partition for My Documents and/or Program Files and changing the Registry.
Do you have a link to that, or if it's ingrained in your memory, like so many things seem to be by you fast responses, could you post it again.

Thanks,
Unless there are too many objections, I think this is the way I'll go.
Only another $792 and my system will be done for now.
Unless I decide to trade my Master2-Far for a Tyan K8W, which I'm thinking about.

 

[Ice Czar]

reading isnt slower per se
there is added latency in moving the arms reading the data and transfering it over a single drive
so say you have a large file that is fragemented into several locations, its read performance would suffer because it hits that latency for each location

but say you have the same file contiguously, well your penatized once, but then the faster sustained transfer rate comes in and your alot better off

in a nutshell thats the main difference
and it depends on the number of accesses vs the length of the file in overall performance
sequential reads and writes of larger files will be massively improved
but Random reads and writes of small files will be slower than a single drive

 

[rottweiler]

It sounded before like you were saying it wouldn't benefit me to have the raptors in Raid 0.
Is that still the case and your just giving me more information?
Or was I wrong in interpreting your last statements, and you think I should do Raid 0?

I don't really know the ins and outs of video editing.
I know a bit about DVD ripping, and I know there are big files.
However, I don't know how much of them are accessed at a time.

Personally, from everything I've read, I don't think I'd see a significant difference either way.
Maybe slightly, and maybe it would be a big difference to some people, but I've never really seen a lot out of small changes.
Out of all the overclocking I've done, I've really only seen a difference in benchmarks, not actually in real life.
I figure it'll probably be the same with this, I might even be able to encode a DVD in 4hrs. instead of 5.
But I'm at work for 8 (+2 for current commute) so no big deal.
And I'd rather it take an extra hour to encode than freeze for a 1/2s while I'm playing HL2.
Not because I will be gaming a lot, but because I'll actually be using the computer.
It's the same reason I always do full installs of Games, and applications, I'd rather it not have to read from the disc, cause it's soo slow, and I'll waste a little more hdd space.

That's what I think, but more comments are very welcome.
I'm trying to learn all I can since I'm very new to raid.

 

[Konstantinnovation]

When doing video editing the files are mostly contiguous. Don't worry about the latency from a RAID0 array. You should definitely RAID0 the raptors.

NOTE: I asked 3Ware and they responded via email that ONLY the 9000 series will be updated to allow increasing the array size without destroying data. Can anyone confim this?

 

[Warriorprophet]

For video editing on PATA drives I still recommend 0+1, but yes raid1 should give you good enough writes for what you are doing. Almost any raid 0/1 controller can keep up with 14mb/sec uncompressed avi captures, and any decent single drive (the fastest pata 8mb drives, and or the 10krpm raptors, etc) can do it well enough...

DVD rips aren't that big, 7-14gb, mostly in 1gb files, mostly you'll only be using one to re-encode to divx atleast, if you're doing multi-audio or subtitled ogms then it gets a bit different, but still, mostly the speed of your drives doesn't matter at all for dvd ripping, I been doing it on this rig for years, 5x dvdrom, ata100, and its fine...

 

[Ice Czar]

well this morning paladin0 posted an article at Anandtech that addresses the Raptors in RAID 0 specifically

Western Digital's Raptors in RAID-0: Are two drives better than one?

It sounded before like you were saying it wouldn't benefit me to have the raptors in Raid 0.
.

yup pretty much, as the article linked concludes (below)
And as Warriorprophet pointed out
encoding is CPU intensive not write intensive. (and I defer to him, having done little encoding myself )
Now if your where actually video editing then there would be a real benefit
in both reads and writes

Anandtech
If you haven't gotten the hint by now, we'll spell it out for you: there is no place, and no need for a RAID-0 array on a desktop computer. The real world performance increases are negligible at best and the reduction in reliability, thanks to a halving of the mean time between failure, makes RAID-0 far from worth it on the desktop.

There are some exceptions, especially if you are running a particular application that itself benefits considerably from a striped array, and obviously, our comments do not apply to server-class IO of any sort. But for the vast majority of desktop users and gamers alike, save your money and stay away from RAID-0.

Bottom line: RAID-0 arrays will win you just about any benchmark, but they'll deliver virtually nothing more than that for real world desktop performance. That's just the cold hard truth.

now if your running serious graphics programs that heavily use the pagefile (virtual memory,swapfile, scratchdisk) there is also a benefit, however the OS can do a software RAID 0, and without converting the disk to a dynamic disk either, like you would normally have to, so again the single drives would probably be better until you start to deal with real large files. There really isnt much difference between the CPU impact a software RAID 0 and a Hardware RAID 0 has until you get into the higher level cards anyway. (dedicated processors, large caches, quad channel and up, SCSI command overhead, TCQ ect)

 

[geofferama]

Okay, my recommendations: Get a good RAID5 card. Like 6 or 8 ports. I only buy Adaptec, but that's me. Anyway, get the 2 drives (for now if money is an issue) and RAID1 them. Storage. Get 2 Raptors instead of 200GB drives, and RAID0 them. swap. Keep all your Work in Progress on the Raptors, and archive to the 200GBs. Or if you have the $$ now, get 2 or 3 more 200GBs for RAID5 storage. If not, that's why you got a capable card with room to grow..

I also have an Adaptec card, the 2400A (4-channel pata) and it's kindof a tradeoff with this card. The good things about it is it's RAID5 performance (espescially if you get some fast onboard RAM).. The bad is that the card is a foot long & it has horrible RAID5 (re)build times. I run a 4 drive raid5 (formerly with 4 75gb deathstars, but two of them failed at once) and it was a multihour affair to build & rebuild the array.

rottweiler: I personally have a scsi boot drive w/the RAID as a storage/dumping area for dvd ripping etc. And I saw you mentioned OCing... Make sure your mobo has the proper PCI dividers because RAID (&scsi) cards are usually pretty touchy about the PCI frequency. And of course, don't forget to put the really important things on CD/DVD. Its a horrible feeling to come back and find your redundant backup wasn't so redundant.

 

[Ice Czar]

Also I remember reading in another post about the best setup for partitions.
It was probably by you Ice Czar.
Something with a seperate partition for My Documents and/or Program Files and changing the Registry.
Do you have a link to that, or if it's ingrained in your memory, like so many things seem to be by you fast responses, could you post it again.
.

DOH forgot something

http://www.hardforum.com/showthread.php?s=&threadid=741512
http://partition.radified.com/partitioning.htm
http://zapwizard.com/Partitioning/How-To.html
(I dont seperate out registry dependent aps from the OS as it complicates the hell out of repair and recovery, unless its a dedicated box, in which case, say its solely for Graphics or 3D, then the OS on one channel, and a Program on another, and a dedicated drive(s) or array for a swapfile, and mass storage somewhere might make a little sense and a slight improvement, provided they are all on seperate channels that dont have any interfering access going on, and more RAM is still a better performance investment)

in addition (old school)
http://www.windowsitlibrary.com/Content/435/07/1.html
some of which is applicable, other parts being exceeded these days or accomplished easier with say a 3rd party defragmenter like O&O Defrag
contains alot of insight, however it is NT oriented, and Ive yet to sort it out and include the applicable parts in the 1st tutorial