NAS for video editing in Raid 5?

N

note235

Supreme [H]ardness
Joined
May 23, 2005
Messages
5,055
I edit raw videos between 2 very powerful pcs.
Basically I'd like to know if its possible to create a storage system that can allow me to write/read at around 500MB/s each for two pcs within a single one story house. They both have gigabit ethernet, usb3 and will get thunderbolt once motherboards come with that.
 
In a word no. You aren't going to get those kinds of speeds from NAS since gigabit maxes out somewhere around 120 MB/s. You need DAS. A SAS switch or a storage appliance could do it.
 
It's not completely out of the question it does however require Win8/WS2012 and a whole lot of GigE ports, for instance I did a copy of a 10GB VM from one Hyper-V host to another, each of which had 5 gig ports usable by the host:

ws2012_network_file_xfer_350MBs+.png

Note this is just with 5 regular NICs with no LAG/LACP from an intel 520 on one host to an intel 520 on the other.
 
Yeah its either 10GigE, or Win8/2012 (for the SMB Multichannel) + quadport 1GigE cards in each machine.

I get 500-600MBps from server to workstation with quad port card + two onboard NICs in each machine. SMB 3.0 is fantastic and almost makes me forgive the abortion that is Metro UI, almost.
 
10gigabit ethernet, fibrechannel, or infiniband to get to those speeds.

Or possibly a big LAG, with quad-port NIC's in each machine.
 
It is doubtful that software raid 5 in Windows is going to give you 500 MB/s writes to the server from each client simultaneously (I assumed software). Second, the use of multiple gigabit links is about as scalable as a brick wall. Every client that you might want to add in the future is going to get 4 links. That doesn't make a lot of sense when 3 cables total will be all you need in the case of IFB and SAS. In the case of 10GbE the aforementioned solutions are also cheaper unless the prices have finally come down on that.
 
Last edited:
Yes, you can but its not going to be Nas,USB3, or Thunderbolt.. I'm thinking infiniband, or even FC. what is your budget, most inifiniband and FC solutions are going to be 30k+?

But why not a hybrid?

I do a small amount of video editing. I have 2 smaller SSDs raid0'd together for the project I'm working on and for render space. Currently using Avid. When the project is done, and rendered out I copy the finals to my NAS. Gives me the most bang for my buck.

Your other machine would be able to get copy the renders/project files from the nas.
 
It won't be cheap, but it can be done with 10GigE, Infiniband (HBA in each box, connected -- 10Gbps is the cheapest), or a DAS (SFF-8088 ports/cables).

Infiniband would be the cheapest. Mellonox ConnectX HBAs can be had for $50 or so each, and you would need 2 1 ports for each PC and a 2 port card for your storage box. Then two cables, which will probably run $60 each or so. $250-400 depending on brands and used or new. Mellanox ConnectX (1 series) do not work with Server 2012, but the ConnectX-2 and ConnectX-3's do work with SMB Direct (SMB over RDMA) on Server 2012. IB can be done without a switch, which is the most expensive part.
http://blogs.technet.com/b/josebda/...connectx-3-using-infiniband-step-by-step.aspx

DAS would be the second cheapest assuming you build a storage device (pre-built will cost more) with some sort of RAID controller or HBA (if you wanted software RAID) and got a SFF-8088 HBA for each PC.

10GigE would require at least the cards if you went crossover, which will run $350+ each. If you wanted to do it off of a switch, well it will be several thousands.

Another option would be SMB Multichannel (SMB 3.0 with Server 2012). 4-6 NICs from the server to a switch, then another 4-6 NICs from your PC to the same switch. Messy, probably not cheaper than direct IB connections. Quad gigabit PCIe cards can be had for $150, and you will need at least a smart switch ($150-300 for a 24 port).
 
For that kind of speed, you likely want not RAID-5, but RAID-10, especially if you're doing nonlinear editing (which means the file may be accessed nonlinearly AKA non-sequentially AKA something RAID-5 sucks at). You will also need several disks. I'd say like 8 HDDs. 4-6 would probably give you enough speed with the right setup most of the time, but I think the minimum speed would dip below what you need. And you will want a quality RAID card (an IBM ServeRAID M1015 for $60ish in RAID-10 mode with 8 drives would be sweeeeeeeeeeeeeeeeeeet) IMO.

Also, thousands for 10GigE? LOL. Pick up a Myricom 10G-PCIE-8A-C for $50-100 (or an Intel Neteffect NE020 IF it has support for the OS you want to use, which IIRC is either Windows only or Windows and Linux (no Solaris, BSD, etc.) only) on Ebay plus a Woven Systems LB4 switch (which has 48x 1Gbps + 4x 10GbE CX4) for $150-300. There are also some similar Broadcom-brand switches on Ebay from time to time for like $200-400. I haven't upgraded my server to use 10GigE yet, but I am running one of those switches at 1Gbps, works well.

Non-CX4 10GbE stuff will either cost a good bit more and/or require you to be a lot more lucky on Ebay. CX4 is great for everything except running cables through walls in a home (because of the large connector and inability to cheaply/easily crimp your own). Well that and it has kinda low max lengths, but should be good enough for a home environment.

Another thing: if you're going to access the device using Microsoft's SMB protocol, make sure it supports at least version 2.0. Old SMB 1.x support does not mix well with the performance requirements you have.
 
Last edited:
I know macs with thunderbolt ports can connect to each other directly to transfer data. If PCs can do the same just have a fast drive array in 1 computer and share it over a point-to-point thunderbolt connection. I believe thunderbolt can handle two 10Gbps channels per cable.

I have a IBM M1015 like dandragonrage mentioned that can do over 800MB/s in raid0 with 8 drives connected. They are not good for raid5 though. I also had to get a new motherboard to use it since both of my existing boards wouldn't recognize it. They can be picky with consumer boards.
 
Last edited:
That Infiniband solution was good until 10GbE stuff got cheap, IMO. I can't see any reason to go Infiniband over what I recommended.
 
dandragonrage said:
For that kind of speed, you likely want not RAID-5, but RAID-10, especially if you're doing nonlinear editing (which means the file may be accessed nonlinearly AKA non-sequentially AKA something RAID-5 sucks at). You will also need several disks. I'd say like 8 HDDs. 4-6 would probably give you enough speed with the right setup most of the time, but I think the minimum speed would dip below what you need. And you will want a quality RAID card (an IBM ServeRAID M1015 for $60ish in RAID-10 mode with 8 drives would be sweeeeeeeeeeeeeeeeeeet) IMO.
Click to expand...

you can get the same amount of IOPS and transfer with 2 smaller SSDs and the cost would be cheaper. SSD size would depend on project size. The pain point is the final render out, not the preview renders while doing the editiing. Atleast for me anyway.
 
Yeah, SSDs would be helpful for speed, but if OP intends to do a LOT of high bitrate editing (and again, especially if nonlinear editing), I'd recommend SLC/enterprise-grade SSDs, or even an older good SSD with higher write cycles than today's mainstream drives, like the X25-M (but that would probably mean buying it used, which IMO disqualifies that option)

A good current mainstream drive could survive for a while, sure, but I'd want them to last "a really long time," personally. If he's not editing that often then today's mainstream drives would be okay. Other than Samsung 840 TLC garbage which I refuse to recommend for ANY purpose. I will NEVER recommend TLC.
 
The nice thing about SSDs, the price is definitly coming down on them. Even the high end ones. You could use some lower end ones, just to "get by" for now and it would get the job done. Most of them will still out perform any consumer spindle based disk. OP never really responded what his budget was so that would be the deciding factor.
 
kdh said:
The nice thing about SSDs, the price is definitly coming down on them. Even the high end ones. You could use some lower end ones, just to "get by" for now and it would get the job done. Most of them will still out perform any consumer spindle based disk. OP never really responded what his budget was so that would be the deciding factor.
Click to expand...

haha true
about 1k

my current setup is pretty adequate with raid 5 but its constant write/read
at 120MB/s
storage volume is preferred over speed since it's 250gb for 30 minutes of footage
 
Jesus. I hate working with uncompressed video. It just has ridiculous I/O requirements. Absolutely no reason to use it IMO. Even at 4k, lossless codecs will easily reduce your I/O requirements to easily manageable levels, and some of them are pretty fast.

I've spent a LOT of hours with AVISynth, Virtualdub, etc. Dabbled with creating a few AVISynth filters, too. Not sure what you're using, but damn.
 
dandragonrage said:
Jesus. I hate working with uncompressed video. It just has ridiculous I/O requirements. Absolutely no reason to use it IMO. Even at 4k, lossless codecs will easily reduce your I/O requirements to easily manageable levels, and some of them are pretty fast.

I've spent a LOT of hours with AVISynth, Virtualdub, etc. Dabbled with creating a few AVISynth filters, too. Not sure what you're using, but damn.
Click to expand...

blackmagic camera
for archiving I would use cineform or dnxhd or prores
my 1.32tb becomes 32gb haha
 
note235 said:
haha true
about 1k

my current setup is pretty adequate with raid 5 but its constant write/read
at 120MB/s
storage volume is preferred over speed since it's 250gb for 30 minutes of footage
Click to expand...

1k being your budget, is that for both machines?

really rough digging here.. but Newegg has some so so 256gig sata iii drives for a reasonable cost.

http://www.newegg.com/Product/Produ...IsNodeId=1&name=256GB&Order=PRICE&Pagesize=20

But 3 256 raid 0'd ssds would punch your raid5 array in the face in terms of performance, and should get you the space you need for 90 minutes of video. while straying just slightly over 1K in cost. Im obviously assuming your motherboard supports raid. being raid0 you'd want to do backups every few hours over to your nas.

hope that helps, best of luck. =)
 
kdh said:
1k being your budget, is that for both machines?

really rough digging here.. but Newegg has some so so 256gig sata iii drives for a reasonable cost.

http://www.newegg.com/Product/Produ...IsNodeId=1&name=256GB&Order=PRICE&Pagesize=20

But 3 256 raid 0'd ssds would punch your raid5 array in the face in terms of performance, and should get you the space you need for 90 minutes of video. while straying just slightly over 1K in cost. Im obviously assuming your motherboard supports raid. being raid0 you'd want to do backups every few hours over to your nas.

hope that helps, best of luck. =)
Click to expand...
yes 1k for both
it does but software
i was going to get areca card for my system
im thinking maybe i should get separate raids for each of the computers and use adobe cloud to just get the project files
 
B34M3R said:
I know macs with thunderbolt ports can connect to each other directly to transfer data. If PCs can do the same just have a fast drive array in 1 computer and share it over a point-to-point thunderbolt connection. I believe thunderbolt can handle two 10Gbps channels per cable.
Click to expand...

Greetings

Sadly Microsoft removed the direct cable connection capability starting from Vista, so that only leaves XP as the only other option assuming you can get Thunderbolt drivers.

http://support.microsoft.com/kb/930086/en-us

Cheers
 
note235 said:
yes 1k for both
it does but software
i was going to get areca card for my system
im thinking maybe i should get separate raids for each of the computers and use adobe cloud to just get the project files
Click to expand...

Thats about what I was talking about.. 3 drives and raid card per box. Dont do software raid. You'll be pulling cpu cycles and ram from your machine to keep raid working vs's using those cycles and ram for your renders. Then a regular nas for your finals/project files. Youre starting to push out to the 1.3k range. But I think if you spend a little extra, it might get you exaclly what you need or pretty close.
 
note235 said:
i was going to get areca card for my system
Click to expand...

Good brand, but typically pretty overpriced. I'd go with an LSI-based card if possible, because many of them are available for MUCH better prices than Areca stuff. For example the IBM ServeRAID M1015 which will be under $100 used though it doesn't support RAID-5 (which, seriously, you shouldn't be considering anyway). There are other LSI cards with RAID-5/6 support that will still beat the crap out of Areca cards for value.

P.S. Software RAID won't steal CPU from his encode if it is a different machine, but I don't think software RAID would get him the kind of speed that he wants, either.
 
You must log in or register to reply here.
Back
Top