Video editing workstation RAID help

P

Pultzar

Limp Gawd
Joined
Jun 22, 2010
Messages
198
Hello!

I am setting up my storage for a video and photo editing workstation. I will be using this machine to edit 4-6k RAW footage which can sometimes be 25-50MB per frame. I have been narrowing my choices to:

RAID 10 - Western Digital 3TB Red Drives
or
RAID 6 - Western Digital RE 4TB Drives

These have roughly the same cost per TB after the parity drives have been purchased for Raid 6. I can see my array growing to around 24TB. The Raid 6 has a lot less drives to deal with. However will it be as protected as the Raid 10? URE with Raid6 is the issue if I understand things correctly.

Backup Solution - LTO-6 Drive

Choice for controller card

Adaptec 71605Q with 1.5TB of SSD Cache

or

Areca 1882ix with 4GB of Cache

The downer with the Adaptec is that I would be forced to use an internal LTO drive as it does not have external ports. The Series 7 have been getting some nice reviews.

Thoughts? I would appreciate it that if you give me a suggestion that you also say why, since I'm trying to learn something here as well :)

Thanks!
 
It has also occurred to me that the Areca might not support LTO drives. Does anybody know if this is true? The adaptec can put the drive in a pure HBA mode which I hope would work.
 
is the a reason why you want to use tapes for backup? offsite requirements or something? maybe a USB RDX chassis would work better. but is disk to disk just not an option at all

areca and adaptec and adaptec both make great products. i have not used those specific ones but i have never been disappointed by offerings from either company.

RDX can also be done in SATA so it would work on a controller or directly on an MB and save the controller for actual HD's.
 
I am leaning towards tape because it is easy to have offsite and really cheap per TB. My footage is 500GB per hour. With the above schemes this is $50/hour of footage not to mention offsite backup requirements. An LTO-5 tape is about $18/TB which results in a backup or storage cost of $9/hour of video.
 
If this were me I would personally do a RAID 10 of 4TB - RE drives. I would definitely spend the bit extra up front and have a better performing solution.

If this is somthing that you make money off of, and where latency will cost you productivity, I would really want the best quality drives I could get and not the El cheapo "red" drives.
 
staticlag said:
If this were me I would personally do a RAID 10 of 4TB - RE drives. I would definitely spend the bit extra up front and have a better performing solution.

If this is somthing that you make money off of, and where latency will cost you productivity, I would really want the best quality drives I could get and not the El cheapo "red" drives.
Click to expand...

Won't the read performance be about the same in Raid 10 as in Raid 6? I can always put some Raid 0 drives in for scratch drives.
 
Both read and write will be faster with RAID 10 in comparison to the same number of drives in RAID 6


(With a good modern RAID card)
 
Both read and write will be faster with RAID 10 in comparison to the same number of drives in RAID 6


(With a good modern RAID card)
 
Both read and write will be faster with RAID 10 in comparison to the same number of drives in RAID 6


(With a good modern RAID card)
 
still look into RDX. faster easier to use interfaces, and VASTLY more robust media. cost can be skewed to either tape system depending on set up. (RDX is a tape system, in case i did not make that clear)
 
You could easily get an internal to external adapter for the Adaptec.

As an owner of a 71605Q, and as I've said in other threads, I'm fairly impressed with the caching so far with my fairly bastardized setup. One test I did involved 4 intel 520 240GBs and 4 2TB WD Red (or equivalents, all 5k RPM drives) in RAID-5. The SSDs were split up in pairs, one pair in RAID-1 for the 240GB cache and the other in RAID-0 for a 480GB source disk. I did two tests one with write caching one without. I tested by using the Hyper-V manager to create a new VHD by copying the contents of the 480GB RAID-0 to the 6TB RAID-5 volume. Without caching the drive averaged 100-200MB/s, with most of the test being 150MB/s or slower. With caching the copy for the most part averaged 200-300MB/s with much of the test being in the 250MB/s+ range. A notable exception was at about the 90% point where the cached test suddenly dropped to 4-5MB/s for a minute or two before resuming back to 200-300MB/s for the rest of the test. I figure this was due to the fact that at this point the cache had written almost 2 full write cycles to the drives in a 25 minute period and had to do some emergency and heavy duty GC, which to me is understandable and acceptable. In the end the non-cached copy took almost an hour, the cached test took just under a half hour, so for large, almost 500GB file copies, the SSD cache halved the amount of time necessary to do the copy.

It seems Adaptec uses the SSD write cache to bypass the whole RAID-6 write penalty by only doing writes when it can write a full stripe, effectively reducing the write penalty from 6 to less than 2, which is actually better than RAID 10.

Just to be clear where are you getting your 1.5TB SSD cache number from? To do write caching you need to have a redundant maxCache array so either RAID-5 or RAID-1E (which is like a striped RAID-10). I'd recommend the Adaptec with RAID-6 spinners and RAID-1E cache, honestly I've had a hard time accurately benchmarking mine but last night I did kick off full virus scans on all 20 of my VMs and while disk utilization held steady at around 50-80% for the duration, everything was still plenty responsive the whole time despite 20 antiviruses scanning the same storage subsystem at once.

As far as protection goes technically RAID-6 worst case is better than RAID-10 worst case. While RAID-10 can theoretically survive the loss of half of its disks, it can also theoretically completely die with the loss of 2 if it's 2 in the same mirror group. With RAID-6 you always need to lose 3 for a complete loss (or lose 2 and get a URE to lose that part of an array, but to be fair in RAID-10 if any drive fails and the wrong drive gets the URE you lose data too, so RAID-6 is still better in worst case)
 
Sorry I was wrong about the 1.5TB. I have 3 Sandisk Extreme 480GB drives which would yield 720GB of cache. I would consider using SuperSSpeed S301 drives but I can't find them in stock.
 
Zedicus said:
still look into RDX. faster easier to use interfaces, and VASTLY more robust media. cost can be skewed to either tape system depending on set up. (RDX is a tape system, in case i did not make that clear)
Click to expand...

Everything that I can find on RDX indicates that it is a hard drive system. Do you have a link please?

1.5TB RDX drive is about 10X the cost of a 1.5TB LTO-5 cartridge.
 
Pultzar said:
Sorry I was wrong about the 1.5TB. I have 3 Sandisk Extreme 480GB drives which would yield 720GB of cache. I would consider using SuperSSpeed S301 drives but I can't find them in stock.
Click to expand...
That's still plenty and RAID-1E will use 3 drives just fine. Unless you plan to consistently do 1.5TB+ writes in an hours time the performance should be superb. Adaptec + 720GB cache + RAID-6 would give you way more performance than areca + RAID-10.
 
Pultzar said:
Everything that I can find on RDX indicates that it is a hard drive system. Do you have a link please?

1.5TB RDX drive is about 10X the cost of a 1.5TB LTO-5 cartridge.
Click to expand...

the backup media is a ruggedized HD and it is used in a similar fasion to a tape (LTO) system.

the chassis for LTO will be about 1000$. the chassis for RDX will be 100$. RDX interfaces more easily, RDX is OVER 10x faster then writing to tape, so if this is production that alone may cover the media difference. also the media difference scales not equally, it may be vastly cheaper to use 2 RDX drives. and with the difference in interface and write speed, waiting to change tapes is not an issue.

RDX is not ALWAYS better, but if speed and mobility are important, it it worthwhile. the RDX chassis can even be external on USB2 and still be faster then LTO systems. plus then you can grab the RDX and take it with you if you need to do an import at a customers site (though depending on the software you may need to bring a laptop to do the interface).
 
Pultzar said:
I am leaning towards tape because it is easy to have offsite and really cheap per TB. My footage is 500GB per hour. With the above schemes this is $50/hour of footage not to mention offsite backup requirements. An LTO-5 tape is about $18/TB which results in a backup or storage cost of $9/hour of video.
Click to expand...

tape is horrible...and slow, If you have the cash get a raid controller with external ports and go that route. I have also had a lot of luck with USB 3 devices from i\o safe. The are fire and water resistant...so I keep them in a safe.
 
I finally ran across this review of the non-Q series, seems like the controller is living up to most of the stated specs, still would love to see a head-to-head of an Adaptec Q vs LSI CC.
 
As Dragon has pointed out, a raid controller with SSD cache is the way to go. As far as backups, if these are videos, why not use a cheap method such as bluray discs? 50gb discs should be enough to fit a couple of videos in for long term storage (5-10 years). I have a LSI 9285-8e and I was thinking of getting SSDs to build a cache but I really have no use for it, I just wanted to try it out.

Just a thought.

*Edit*

So after doing a little math, for example 12TB that need to be backup:

Tape:
3k for the unit
100 per tape (2.5TB per tape)
So about 3500 to backup and only have to change tapes 5 times

Discs:
150 for 50 dics (50gb per disc)
240 needed but let's round up at 250, so 750 and you need to change discs 240 times.

I would hate to change the damn thing 240 times, so it depends on your budget, but if you can find a cheap LTO6 tape unit then it's worth having a tape unit for backups.
 
Last edited:
What's your budget? If this is your job or something you do frequently, I would try to get it is fast as possible. Go with a pool of Samsung 840 Pro in RAID 0 on PCIe or onboard adapter. In addition, add a pool of 3/4TB traditional disks in RAID 10 for your static storage.

I don't have experience with RDX, but it sounds better than dealing with an LTO infrastructure.

Just keep in mind that the bottleneck on your system will eventually be the PCIe bus. But it all goes back to what you are willing to spend.
 
So in setups where you have a raid controller + hdd + ssd, do you still split up your working data from the OS? I was thinking of a super budget solution that involved the simple setup above and install the OS and video edit from the same drive. Is that advised against?
 
Last edited:
You must log in or register to reply here.
Back
Top