RAID5 NAS setup for video editing or not?

nakedhand

2[H]4U
Joined
Apr 21, 2008
Messages
2,127
A couple of days ago I knew absolutely nothing about storage arrays, so I have been reading here to equip myself with a basic understanding of RAID setups and what will suit my usage patterns. Now I am ready to purchase the first parts (HDs in august, since I am heading to the US at that point), and would like your opinion.

I primarily work with four types of tasks. 1: Loading (viewing) and adjusting hundreds of 15-25MB files with Lightroom. 2: Creating complex Photoshop files, often above 1GB. 3: Compositing and video editing.

Data Security is important and so is the gained read/write speeds, so I figure I need RAID5. I am not ready to sacrifice the performance loss in going RAID6, nor the space.

Forgive my potentially silly question and ignorance with network setups, but I am considering making this disk array a NAS. This instead of throwing it in my workstation avoiding the heat and noise. Does that make sense? A Gigabit connection will only carry 125MB pr. second as far as I understand, so my question is this; Is that enough throughput with 5 x 1TB SATA drives in RAID5? I have no idea what their throughput is going to be, sustained. SATA I theoretical limit is 1.2 Gbit uncoded I see..

I am aware that I am not going to benefit from the full potential of the SATA II drives due to these limitations, but is it an acceptable loss or ludicris? Will there be an unacceptable latency working from a NAS like I am settin gup here, with my type of usage? Do the component match?

DRIVES:
5 x 1TB drives, undecided. Leaning towards: Hitachi 7K1000 or Seagate ES.2


SWITCH:
NETGEAR FS726TP 10/100Mbps + 1000Mbps
http://www.newegg.com/Product/Product.aspx?Item=N82E16833122077

or

3com 3C1670800B 10/100/1000Mbps OfficeConnect Gigabit Switch 8
http://www.newegg.com/Product/Product.aspx?Item=N82E16833105136


CONTROLLER:
3ware 9550SXU-8LP 64bit/133MHz PCI-X SATA II RAID RoHS Controller Card - Retail
http://www.newegg.com/Product/Product.aspx?Item=N82E16816116054

ENCLOSURE:
Supermicro CSE M35T (fitted in a Lian Li PC82)
http://www.newegg.com/Product/Product.aspx?Item=N82E16817121405
...SATAII compatible, but disks will be detected as SATA I.

I assume the controller is backwards compatible, and chose a SATA II controller to go with the drives, should I decide not to go with the cheap Supermicro enclosure. I am really unsure about that component still. The controller card is probably overkill, but I want a good card that I can use with an 8 disk array later on, outside the Supermicro.

Any feedback is appreciated. A simple thumbs up or a "you are insane you got it all wrong" would be great. Be gentle please, I am totally new to all this :)
 
I have no idea what their throughput is going to be, sustained.

If you are a pro then I recommend you don't DIY. As an artist you have enough to be concerned about w/o also having to troubleshoot a complex system. When your raid card dies or you have two simultaneous drive failures or you can't make RT due to file system churning you won't have experienced help that you can call at 2AM so you can meet your deadlines. Also there is some reasonably affordable DAS you can purchase and most of the companies in this market are smaller and generous about payment terms.

If you're still feeling brave then here's some more advice...

1. You can use SATA drives but you need to do frequent backups; I have several (firsthand) experiences with two SATA drives going out at the same time in a single 5+1 R5 array.

2. I quoted you above because you MUST figure out what kind of speeds you NEED to sustain. Do you need RT uncompressed hd/2k streaming? Can you afford more drives? I don't think you can make enough sustained speed for RT hd/2k streaming with only 5 drives in R5; also you may want to look into raid 0 or raid 10.

3. If you want to playback uncompressed HD/2K files then gig ethernet will in no way cut the mustard. Either way, think about buying an external enclosure that you can attach via SAS because it is a faster technology and you will be investing for the future.

4. Plan your workflow! Filesystem churning can result in storage that could originally perform 380mb/s reads dropping to less than 5mb/s. If your application is constantly writing existing files and you are also creating new files it will lead to performance degradation and you need a plan in place to deal with that.

5. Whatever you do, don't cheap out here; remember this is your BACKBONE and all of your data and applications need both excellent reliability and sustained speed.
 
Thanks for the useful insights Shaggy3zx, much appreciated. I am still feeling somewhat brave (and scared..) and do acknowledge that RAID6 would be safer. Its unnerving to hear of your first hand experiences of several accounts of simultaneously failing SATA drives!

I work on a project oriented basis as a freelancer and have done so for a few years, and also study and work in a completely different field at the same time, so I am hesitant to spend what is needed to get a DAS (which in my language odly enough is old slang for a toilet..ok that was irrelevant), that fits the characteristics you describe. I have checked the market here in Denmark and I am not going anywhere for under $10.000, which is more than I am currently willing to currently spend. Seeing 2TB drives arent too far in the future, makes me want to wait that extra while too.

Real time uncompressed 2K is the ultimate goal in terms of throughput. Most of the work I do is photography, 2D and 3d, so maybe for now, I should settle for a much cheaper 5 disk RAID5 and do backups on separate disks that can be shelved as a precaution. Then I can get a chance to familiarize myself with the intricacies of RAID, until I feel a big purchase is necessary.

You emphasize workflow planing. I certainly agree that its crucial in all respects, especially in 3D production its vital. Is there specific "traps" that are especial heavy performance hitters, that you would like to elaborate on?

Any comments on the controller I have picked? I read a lot of positive feedback on the Dell Perc 5/I here, but am not up for that after reading through a dedicated 2cpu forum thread. Sometimes they work, sometimes they don´t apparently. Also the Areca 1220 seems to be popular here. I am not used to buying hardware blind fold, like I feel in this instance. It will be paired with a P45 chipset motherboard, if that makes any difference choosing.

Thanks for your time :)
 
I'm not an expert in video editing by any measure, but I can give you some advice that I hope will be helpful.

Firstly, for the sort of bandwidth you'll need, you will need a locally attached storage system. Whether you want to do this with internal drives or an external enclosure attached via SAS/Port multipliers depends on the scale of the system you're going to want. RAID 10 will give you good speed with some redundancy, or if you need maximum throughput possible, RAID 0 is the fastest, but 1 disk failure will bring the entire array down.

Secondly, plan on disks and arrays failing. Unless you need the absolute best uptime possible, it makes more sense to avoid things like Raid6 and instead use that money for better backup solutions.
I'm not totally clear on what your throughput and bandwidth requirements are, but a Raid 10 might be your best bet between uptime and speed, and doesn't require any parity calculations, so you don't need the best controller around.
Your backup solution again depends on the balance you want between uptime and cost. You could have a perfectly reliable raid5 or 6 NAS for fairly low cost, but in the event of a main array failure, you'd lose some time getting the system back, since you're limited to Gigabit speeds when you're restoring from backups. A directly attached external storage array would be faster to restore (and you could even do some work directly off it while your array is down), but the cost is greater.
 
Thanks The Hunter.

After more research more considerations and reading your post, I have decided to locally attach around 6-8 drives in my up and coming workstation in a RAID6 array, and use my current machine on a gigabit network as a fileserver with a the hotswapable enclosure, for daily backups.

I am still not sure about the controller card, but ithere is no rush, since I am not getting the motherboard before August. Thanks for the feedback once again. Expecting drive and array failures from now on :D...arghghh
 
Back
Top