snapshot raid recommendations for media backup

dirtrider

Limp Gawd
Joined
Feb 3, 2008
Messages
135
I'm pretty new to raid. I do have a rather large amount of data to backup though. I'm around 12tb now. I had been using a bunch of external hard drives and mirroring them but that is stupid expensive and tedious now with these many drives. I was looking into a hardware raid solution but didn't like the fact that the drives are spinning all the time creating heat and using energy when they won't be used 99% of the time. I also didn't like the high cost of the raid cards plus expanders necessary for a large amount of drives. I will start with 10 drives in raid. Beyond that I'll need to have another case for direct attached storage. I started seeing references to snapshot raid and many mentions of flex raid. I really like that I can run it on windows since I don't plan to build a separate server for this. Just want to run 10 of the 12 drives I am putting into my new build in raid. Also already have much data on these drives so I was hoping to be able to preserve it.

What programs are out there that are relatively proven for snapshot raid? What is the general consensus here for using flexraid?

Will I be able to use this supermicro sata controller I currently have for flexraid or other snapshot raid? I already have this.
http://www.newegg.com/Product/Product.aspx?Item=N82E16815121009&cm_re=supermicro_controller-_-15-121-009-_-Product
Then will I be able to use this supermicro sas controller to connect to a separate direct attached storage? Other recommendations?
http://www.newegg.com/Product/Product.aspx?Item=N82E16816101358&cm_re=supermicro_controller-_-16-101-358-_-Product
 
The first card is definitely out. That is unless you have a server based system from 2+ years ago. That has PCIX slots instead of PCIe.

What programs are out there that are relatively proven for snapshot raid? What is the general consensus here for using flexraid?

These are both software solutions and thus you do not use a hardware raid controller. Although from the looks of it these are not raid5/6 controllers but will do raid0 or 1 which you will not use in your situation.
 
Last edited:
@dirtrider:

FlexRAID should work perfectly for your requirements.

The nice thing about FlexRAID is that if you have existing drives with data on them, you can install FlexRAID and add or use existing drives as parity drives and off you go. You can run the parity generation any time you need to (usually after you added or made any changes to the data in your storage pool).
Another nice feature about FlexRAID is that you can use different sized drives (for your data drives as well as your parity drives).
The only requirement that FlexRAID has is that the parity storage needs to be the same size or bigger than your largest data storage drive. For example, say you have 2 x 500GB, 4 x 750GB, 2 x 1TB, 2 x 1.5TB and 1 x 2TB drives. This means your Parity drive has to be at least 2TB in size. Now you could either use a single 2TB drive or 2 x 1TB drives or 4 x 500GB drives or 3 x 750GB drives, it doesn't matter as long as the total storage is larger than your largest data storage drive!

Also, in case your computer dies, you can pull any drive and hook it up to any other windows based computer and read the data (something you can't really do with hardware based RAID solutions!).

FlexRAID gives you different option in regards to the RAID (protection) level you want...

You'll find a lot more helpful information on FlexRAIDs website!
 
Hi treadstone - Glad to see you replied to this one. You seem to be one of the most experienced here with flexraid so I really appreciate your feedback. I am ready to give this flexraid a shot once I get the hardware figured out here which should be quick. I was just a little afraid of what some have said here and there about the creator being rather aloof but it seems to be pretty well proven at this point. I haven't heard of a better option for me especially being that I am running windows 7 ultimate 64bit on my main pc and I am starting out using this as my media server too.

An hba is basically just the controller card right? What hba + expander do you recommend? I have the supermicro 8 port sata controller as referenced in the first post but running on pci may be a limitation??? I saw the thread on the hp sas expander and wondering if I should get a hba compatible with that for future expansion options. Also not sure if that hp sas expander will work nicely with my mobo. I am maxed out on this new build as far as drives I can fit in the box is concerned so I want to have the option to connect another case shortly. Perhaps a norco 24 bay or something. Though now that I think of it I have 6 sata ports on the mobo. With many controllers I get 2 sas connections that can be split into 4 sata ports each. So that would give me 14 sata ports in my box which is exactly what I need for 1 OS drive, 1 non-raid drive, 10 drives on flexraid, and 2 optical drives. So perhaps I should just get the single controller for now. Just need to be mindful of what one to get and if possible it would have an external connection for the SFF-8088. I haven't seen anything like that though. Recommendations?

Finally, with flexraid can all the drives be spanned into one volume? Not sure if I am saying that right but it would be nice if it was treated all as one drive or at least everything but the parity drive(s). Then stuff doesn't need to be sorted around to be alphabetical ;-)

My pc build as follows:
Case: COOLER MASTER COSMOS 1000
PSU: CORSAIR HX Series CMPSU-1000HX 1000W
Motherboard: ASRock X58 Extreme LGA 1366 Intel X58 ATX Intel Motherboard
CPU: Intel Core i7-930 Bloomfield 2.8GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1366 130W Quad-Core
Video Card: MSI N460GTX CYCLONE 1GD5/OC GeForce GTX 460 (Fermi) 1GB 256-bit GDDR5 PCI Express 2.0 x16 HDCP
Ram: 2x for 12gb - CORSAIR XMS3 6GB (3 x 2GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800)
OS HD: Corsair Force CSSD-F120GB2-BRKT 2.5" 120GB SATA II MLC Internal Solid State Drive (SSD)
iStarUsa 5 into 3 - http://www.newegg.com/Product/Product.aspx?Item=N82E16816215001&cm_re=istarusa_raid-_-16-215-001-_-Product
*Case has 6 hard drive bays. Add the 5 bays from the iStarUsa housing. I mounted the ssd in the case separately as it is so small it was easy to find a place. That gives me 12 hard drives and 2 optical. Upgraded and added more fans too.
 
Last edited:
@dirtrider:

I am far from being a FlexRAID expert but it works for my needs which sound a lot like yours :)

As far as what HBA to use with the HP SAS expander card, I bought a AOC-SASLP-MV8 and connected it to two HP SAS Expanders for some tests I ran on my setup. Basically I hooked each SFF8087 port to one HP SAS expander and each expander was connected to 24 drives each. So I can tell you that this setup works. I have switched my setup back to the ARC-1680i (in HBA mode) since I like the Arecas out-of-band management functionality.

I don't know if the HP SAS Expander card would play nice with your MB, that is something someone else might be able to answer or you would have to bite the bullet and try it yourself...

I agree that it would be nice if the author of the program would be around all the time, but then again, he does have a life outside of FlexRAID! And just like everyone else, if something else comes up that is more important, he has to take care of it and I can't (and neither should anybody else) blame him for it...
I am greatful for all the work and effort he has put into this project so far and appreciate the fact that he is kind enough to share his hard work with all of us ... for FREE nonetheless!!!

FlexRAID also has another feature called FlexRAID-view which allows you to create a virtual folder view of different drives (local or even networked drives). I use it to combine my 48 x 2TB drives from my storage pool into a virtual folder that I share over the network as my movie storage.

My particular setup is as follows:

1 x SSD drive connected directly via SATA to the MB for use as the OS drive
1 x CDRW drive connected directly via SATA to the MB
1 x 2TB drive connected directly via SATA to the MB for use as Music storage
1 x 2TB drive connected directly via SATA to the MB for use as the FlexRAID Parity drive
48 x 2TB drive connected via HP SAS expanders to an ARC-1680i SAS controller (in HBA mode) for use as the main Movie storage pool
 
I have switched my setup back to the ARC-1680i (in HBA mode) since I like the Arecas out-of-band management functionality.

Can you explain this out of band mgmt? Not sure what that is or why the areca is better. The 1680i seems to be discontinued for the 1880 series now. Not sure what you take is on whether the new one is worth it or not. Doesn't seem to be many of either of these available used either. What should I look to spend for a used 1680i if I go that route? Is it worth the premium over the supermicro assuming I don't have either of these yet?
 
Last edited:
What I think I really need is a card with 2 sff-8087 ports internal and one sff-8088 port external. This would only use 1 slot. Then I can hook up 8 sata drives internal and later daisy chain on another array right? The only card I know of like this is the areca 1680ix series of cards. Is this a good choice? I read some on the hp sas expander thread I think saying that these cards don't work with many expanders for daisy chaining though. Recommendations?
 
no recommendations on the card to use? Was my point above not valid? One other advantage I see of going with the above card or another like it is that I can switch to raid 6 using the same card if for some reason flex raid didn't work for me.
 
I just purchased an areca 1880i card. I wanted the most future proof and compatible card I could get my hands on that should work nicely with expanders as I need down the road. I know this was more money than I needed to spend with the flex raid solution mentioned here but this also allows me versatility to use either flex raid or raid 6 on the card without having to get new hardware.

So given that I have this card now I'm wondering if I should still go with flexraid or if I should just use raid 6. If I decide to go raid 6 I just need to buy a bbu from what I understand which isn't much and doesn't influence my decision. If you couldn't tell already this isn't influenced so much by $$$ as other ease of use and reliability factors.

I will have issues with raid 6 and the fact that I have 2tb and 1.5tb drives right? Does any raid setup basically treat the 2tb drives like 1.5tb? Would 2 separate arrays make sense here?

Another negative of the raid 6 is having the data wiped on the drives for the initialization of the array. That isn't a deal breaker though as I do have space to copy everything to and from.

How easy is it to add new disks with this areca raid 6 setup? Are the drives able to power down during the 99% of the time they aren't in use? Will an array show up as a single disk (volume)?

Other thoughts?
 
Back
Top