Cost Effective Data Backup Solution - 6.5TB

[pirivan]

Hi all,

Due to a recent HD failure (it caused 0 issues thanks to duplication) I've been thinking a bit about how best to backup the data on my WHS box in case of multiple disk failures, theft, fire etc (probably in that order). It should be noted that this is NOT business data, this is simply media files etc so there is no need for enterprise solutions here; a very cost effective solution is what I would be going for here. So before I get into detail my system setup is more or less:

Norco 4020 Case
6x 1.5TB Seagate drives
4x 1TB WD drives
Total Storage Space: 13TB
Actual Storage Space (duplication is enabled on all data shares): 6.5TB
Total Currently Used: 4.3TB Used (duplicated and stored)
Total Currently Free: 3.9TB

I should also note that I have 2x AMS 5 bay eSATA enclosures with hot swap drive bays that could be used to easily connect backup hard drives to be taken to another location (IF I decided to go to the HD route).

So my first thought was to simply buy some additional hard drives, put them in the eSATA enclosures add them as backup drives, backup the data to them once a month or so (or however often I really felt like) and then take them off-site. So, in theory, I would need more or less 6TB worth of drives total for backup (not exactly but that is just what I started quoting). Thus the cost for that would look like:

1.5TB = 4x120 = $480
2TB = 3x200 = $600

Looking at the cost I began to wonder if LTO3 or LTO4 tape backup might be a better (read cheaper and more portable) solution. Would it be cheaper? It seems to me (from what I can tell) the primary cost of this solution would be the initial cost of an external LTO3 or LTO4 tape drive (new LTO4 drives are pricey). But after that, the tapes seem like a pretty cost effective solution as I wouldn't be concerned with maintaining multiple versions (I would just overwrite the same tapes/hard drives each month with a full backup, there would be no rotation with tapes or hard drives). It looks like the best bet (if I went the tape route) obviously would be to try to buy a tape drive on e-bay from someone looking to unload an LTO3 drive. A few examples I found:

http://cgi.ebay.com/Dell-PowerVault...ents?hash=item2a00ffb215&_trksid=p3286.c0.m14

http://cgi.ebay.com/Quantum-LTO-3-H...rnal?hash=item20ad39170a&_trksid=p3286.c0.m14

http://cgi.ebay.com/Quantum-Ultrium...rnal?hash=item1c0c563a7b&_trksid=p3286.c0.m14

http://cgi.ebay.com/QUANTUM-TC-L32A...rnal?hash=item2302be3dc4&_trksid=p3286.c0.m14

If I went with tape, I think I would want something external like the powervault or Quantum LTO-3-HH I linked there as it is my guess that there really isn't any room to fit in internal tape drive in the Norco 4020. I do have rack space to mount a tape drive if I was to get a rack-mountable device, but that wouldn't be a requirement. If I got an external drive, would it be easy to get a compatible card to put in a PC (a PCI-E/PCI slot in the server) if one wasn't sold with the tape device?

What are the major drawbacks to using compression on LTO3 tapes to double the storage capacity (800GB)? I assume with my simplistic requirements and non-enterprise data status, I could use something simpler and cheaper (open source?) than say Backup Exec to do full backups. I assume that the usage of tape makes restores a bit more complicated but with the right software I would guess it isn't too bad (my limited BE experienced has made it seem fairly easy).

Anyhow, any thoughts on all of this? Anyone considered anything similar for home backup use? What are the major drawbacks I am unaware of? Any additional/better/cheaper solutions for this issue that people have used? I don't even know where to begin looking for cheap, easy to use, tape drives with a controller card/interface that would work with my server.

After the drive failure I have become more acutely aware of how important my data is and that (of course) like RAID, WHS duplication is not a backup. I am at a point in my life where I can probably spend some money for peace of mind but probably not TOO much. I will look forward to reading the replies; thanks!

 

[Blue Fox]

Tapes are only going to cause headaches for you in home use. I would not recommend them.

 

[epimetheus]

I'm working on a solution for pretty much the same problem - offsite backup of my WHS. My problem is my Videos share is sitting at 3.57TB right now. The WHS backup solution will only let you select one physical backup disk to backup a folder to, unless I am missing something. Right now I'm working on setting up a second WHS in a Chenbro ES34069 case with a mini-ITX mobo. I tried unRAID, but it doesn't seem to support the Intel NICs on my ITX mobo, so scratch that.

 

[pirivan]

Tapes are only going to cause headaches for you in home use. I would not recommend them.

My initial thought about tapes was exactly as you stated but I didn't have any reason to justify thinking they wouldn't fit the bill here. I understand that restores are annoying but based on what I want to protect myself from that should be pretty darn rare. What are the major sort of headaches that I would run into? Are these headaches enough to justify the additional cost that using the HD based rotation scheme I discussed would entail? It really sounds like an e-bay tape drive + LTO3 tapes is cheaper but I may be missing something (or many things) here.

I'm working on a solution for pretty much the same problem - offsite backup of my WHS. My problem is my Videos share is sitting at 3.57TB right now. The WHS backup solution will only let you select one physical backup disk to backup a folder to, unless I am missing something. Right now I'm working on setting up a second WHS in a Chenbro ES34069 case with a mini-ITX mobo. I tried unRAID, but it doesn't seem to support the Intel NICs on my ITX mobo, so scratch that.

So, having never actually used the "backup" feature, does it allow you to select a certain amount of folders to backup to a disk (ie split your videos share between multiple disks) or does it only allow you to select a share? It sounds like if this is the case it might be best to just hook a external storage device up to your WHS server (or to any other PC on your network), format the disks as regular NTFS and then manually copy the files from the shares (splitting them between drives appropriately). Or setup a scheduled task/sync job to do this that you can run when your external device is plugged in. Sounds like this is what I might have to do if I go the HD route for offsite backups, if the "backup" feature is so limited (my videos share is at 1.8TB). I will be interested to hear what you end up doing as a "solution" to this.

 

[Blue Fox]

Personally, if I were going to invest in a tape backup, it would have to involve an autoloader. Maybe I just think tapes are a pain from having to deal with them at work. I had a bloody AIT-3 drive in my workstation since that's what we backed up desktops to. When I finally do implement a decent backup solution, I'm just going to build another server without RAID (probably using WHS). It seemed like the easiest solution (and most cost effective).

 

[geiger]

May want to look into using Openfiler to present an iSCSI volume to WHS server.

 

[pirivan]

Personally, if I were going to invest in a tape backup, it would have to involve an autoloader. Maybe I just think tapes are a pain from having to deal with them at work. I had a bloody AIT-3 drive in my workstation since that's what we backed up desktops to. When I finally do implement a decent backup solution, I'm just going to build another server without RAID (probably using WHS). It seemed like the easiest solution (and most cost effective).

I think I can see where your distaste is coming from on this. I would hope that if I went this solution I wouldn't have to use them as much as you probably are; particularly if you have been using those for desktop backups! I agree having a redundant server isn't a bad idea but for me it won't really work to achieve my original goal (off-site backups for a decent value).

May want to look into using Openfiler to present an iSCSI volume to WHS server.

I am not sure if I follow how this might help me with off-site backups(or my queries about tapes vs hard drives for this)? Perhaps you could explain further? This sounds like a good way to have the data backed up to another place in the same location.

 

[geiger]

I think I can see where your distaste is coming from on this. I would hope that if I went this solution I wouldn't have to use them as much as you probably are; particularly if you have been using those for desktop backups! I agree having a redundant server isn't a bad idea but for me it won't really work to achieve my original goal (off-site backups for a decent value).



I am not sure if I follow how this might help me with off-site backups(or my queries about tapes vs hard drives for this)? Perhaps you could explain further? This sounds like a good way to have the data backed up to another place in the same location.

Mount the iscsi volume as a local drive on the WHS box, then add it as a backup drive. I'm not sure if this will survive a reboot though.

 

[ilkhan]

Id say fill one of the 5 bay eSATA enclosures with 1 or 1.5TB drives, put them in a software JBOD (would that transfer from machine to machine?) and present the JBOD volume as a backup disk. But I don't know if a software JBOD (unless the enclosure supports doing that, which would seem strange to me if a dumb enclosure and not a NAS device) array would survive once removed from a computer and presented to a "new" computer.

Or get a 4 bay NAS and use that for the backup device. But a dedicated NAS device would probably double your costs.

 

[spectrumbx]

Hi all,

Due to a recent HD failure (it caused 0 issues thanks to duplication) I've been thinking a bit about how best to backup the data on my WHS box in case of multiple disk failures, theft, fire etc (probably in that order). It should be noted that this is NOT business data, this is simply media files etc so there is no need for enterprise solutions here; a very cost effective solution is what I would be going for here. So before I get into detail my system setup is more or less:

....

Without being biased, can I suggest FlexRAID + FlexRAID-View?

FlexRAID-View is just a feature of FlexRAID (not a separate product, but can be used independently) and presents you a JBOD view of your local and network data and supports virtual views.
FlexRAID give you parity protection.

What's so cool about FlexRAID-View? => http://www.openegg.org/forums/posts/list/155.page

 

[pirivan]

Id say fill one of the 5 bay eSATA enclosures with 1 or 1.5TB drives, put them in a software JBOD (would that transfer from machine to machine?) and present the JBOD volume as a backup disk. But I don't know if a software JBOD (unless the enclosure supports doing that, which would seem strange to me if a dumb enclosure and not a NAS device) array would survive once removed from a computer and presented to a "new" computer.

Or get a 4 bay NAS and use that for the backup device. But a dedicated NAS device would probably double your costs.

To fill in some of the gaps for you the enclosure(s) I have are listed here:

http://www.newegg.com/Product/Product.aspx?Item=N82E16817332017&cm_re=ams-_-17-332-017-_-Product

I believe I could use this enclosure to create JBOD (or RAID5 or whatever) but I don't think I would want to JBOD it and present one large disk to WHS. Without GPT the limit on a volume that I could present to WHS for backup would be 2TB. If I use the enclosures I would just have it look at the disks individually to backup files to; I wouldn't want to use JBOD to make one large volume. However, if I DID use JBOD or RAID of some kind, it looks like that RAID config is stored on the enclosure. So, you can move the RAID between PC's as long as you have the appropriate Sil controller card in the PC to connect it to.

I don't see much or any point to buying any kind of a NAS. It would be much slower than the DAS 5 bay enclosure I already have for backups. I already have the enclosure controller cards for connecting them in two PC's (one in the WHS, one in another PC) so I could connect the DAS 5-bay to either to backup files to (either locally or over the network). After I did that, I would simply take the entire DAS/hard drives off-site until I wanted to run another "full backup".

Without being biased, can I suggest FlexRAID + FlexRAID-View?

FlexRAID-View is just a feature of FlexRAID (not a separate product, but can be used independently) and presents you a JBOD view of your local and network data and supports virtual views.
FlexRAID give you parity protection.

What's so cool about FlexRAID-View? => http://www.openegg.org/forums/posts/list/155.page

I am not sure how RAID even fits into what my original goal is. I already have WHS running it's "duplication" (aka simplistic-automated RAID1 style redundancy). I simply wanted to get the pros and cons for backing up the data on the WHS server for tapes vs hard drives that I could take off-site (in terms of convenience and cost). I don't see any need to have whatever device I use for backups that I take off site running RAID of any sort. Duplication + an off site backup is probably good enough don't you think? I am not sure if I need the redundancy of RAID to protect monthly full backups that would go off-site. Does this make sense?

 

[spectrumbx]

.....

I am not sure how RAID even fits into what my original goal is. I already have WHS running it's "duplication" (aka simplistic-automated RAID1 style redundancy). I simply wanted to get the pros and cons for backing up the data on the WHS server for tapes vs hard drives that I could take off-site (in terms of convenience and cost). I don't see any need to have whatever device I use for backups that I take off site running RAID of any sort. Duplication + an off site backup is probably good enough don't you think? I am not sure if I need the redundancy of RAID to protect monthly full backups that would go off-site. Does this make sense?

I guess I should have explained a little more.

Instead of using WHS's drive duplication feature, many users prefer to use FlexRAID + WHS: http://www.openegg.org/forums/posts/list/28.page
This enables them to have the best of both worlds (all of the features of WHS + parity protection instead of duplication).

In fact, some users still have WHS's duplication feature for their most valuable data (which is often a very small subset of their total data) while having the bulk of the data protected through parity.

This would greatly simplify your storage space need, and therefore leave you with enough resources (money, drives) for your backup endeavor.

Basically, WHS's duplication feature is wasteful.

So, all you need to make your protection + offsite backup work would just be buying one additional hard drive and not a whole new set (you will use half of your existing drives for the offsite backups).

 

[orubap]

Nice bro. As a relative newcomer to 'home storage systems' WHS had all the features I wanted but duplication did seem to me a rather inefficient method of protection. I was aware some people used WHS with hardware raid setups but with the cost and complexity that's not something I could be bothered with.

My machine has duplication on for all shares except my largest: Videos. Just seemed a bit wasteful duplicating that much data.

Never heard of FlexRAID or unRAID before reading this but after scoping out your site, I'll be trying it later.

 

[pirivan]

I guess I should have explained a little more.

Instead of using WHS's drive duplication feature, many users prefer to use FlexRAID + WHS: http://www.openegg.org/forums/posts/list/28.page
This enables them to have the best of both worlds (all of the features of WHS + parity protection instead of duplication).

In fact, some users still have WHS's duplication feature for their most valuable data (which is often a very small subset of their total data) while having the bulk of the data protected through parity.

This would greatly simplify your storage space need, and therefore leave you with enough resources (money, drives) for your backup endeavor.

Basically, WHS's duplication feature is wasteful.

So, all you need to make your protection + offsite backup work would just be buying one additional hard drive and not a whole new set (you will use half of your existing drives for the offsite backups).

Hi Spectrumbx,

Thanks for that extended explanation; I follow exactly what you were trying to get at now. Yes, you would be right, this would certainly be the most cost-effective solution to my issue as it wouldn't involve spending much money at all (which is always nice)! I also, do highly agree that the WHS duplication feature is very wasteful. When I was initially planning my storage box that was a significant negative vs RAID6 however I eventually ended up going with WHS duplication simply for the simplicity of both management and recovery; it didn't seem like there would be much question that I had a clear recovery path in any failure instance.

In any case, I was quite excited about your solution here and I have spent MANY hours today reading up on your forum under the post linked as well as the many threads about FlexRaid/WHS on avsforums. I really wish that the WHS team would take this exact 'FlexRaid' concept and integrate it tightly into WHS so that it is simple to use as duplication; this would be a killer feature. The ability to setup FlexRaid using multiple parity disks is also great, overall you have done a really wonderful job with this software.

However, admittedly, the end result of all this 'research' (4-6 hours of reading at least) was a lot of confusion as to how well this might work with WHS. That thread you linked on your forums was all over the place with the 'guides' about how FlexRaid should be setup using WHS and how recovery should work; users couldn't quite seem to agree (the threads on AVSForums descended into a lot of arguments and it was even more difficult to discern facts). It sounds like due to the way WHS utilizes drive extender/drive pooling and tombstones that it can significantly complicate both parity creation and recovery. FlexRaid sounds like a great product but that the additional WHS "features" complicate things. Also, to get WHS + FlexRaid to work a lot of users seemed to be mapping to C:\fs\U\DE\shares or D:\DE\shares and it was my understanding this was NOT at all recommended by the WHS folks.

I became increasingly nervous about how I would implement FlexRaid properly while running WHS and even more nervous that even if I did that I would have a clear recovery path. The last thing you want to be doing when a drive fails is trying to figure out HOW to recover rather than what to do. You want to have 100% confidence that the system you have in place works properly. From the sounds of it this is probably all because WHS isn't something you use all the time and thus isn't necessarily a priority to work out definitive guides or make sure that everything would work right. Nothing wrong with that at all (you do a great job), it just sounds like that might be the situation.

It also did make me wonder (based on the thread) how long it would take to recovery from a drive failure using FlexRaid as I have 4.3TB worth of data currently. Or heck, how long it might even take to run a parity 'update' nightly. I also wasn't 100% pleased with the idea that I would have to schedule a parity update to run regularly; meaning I would have to leave my server on nightly, wasting energy. Currently I only turn it on when I need it, which tends to be every evening from about 6-10PM. After that it's turn off to save energy (and in my mind) also reduce the number of 'running hours' on the hard drives (hopefully extending the time before they fail).

So it sounds like currently if I want to utilize FlexRaid I have a few options:

1. Wait for someone already using WHS + FlexRaid to write a more definitive 'guide' about how to set it up and recovery properly (with verified recoveries, not like some of the random opinions about how it works like on AVSForums)
2. Wait for you to finish FlexRaid Live and perhaps do some more testing with WHS + FlexRaid in your VM and see if things are a bit easier to implement then
3. Migrate completely away from WHS and move to Server 2008 x64 (I have a license) with a simpler file sharing setup. I sure would miss the drive pooling feature of WHS!
4. Keep reading and see if I can figure out exactly how FlexRaid + WHS should properly work on my own by synthesizing all the information on your forums + avsforums

I probably won't do option 3 because boy was slipstreaming my WHS installation with my SuperMicro card drivers and AHCI motherboard drivers a big pain in the ass; I am not sure I want to repeat that with Server 08 x64 drivers. Though it WOULD be nice to be able to have GPT disks for greater than 2TB volumes .

Anyhow, I wanted to let you know that I had spent quite a bit of time looking into your wonderful software and that's where I ended up. Thoughts (from you anyone else)?

Anyone else on HardOCP running WHS without duplication and using FlexRaid instead? How is that working out? How has recovering from a full drive failure gone? How did you set it up properly so that you have full confidence in recovery?

 

[orubap]

meaning I would have to leave my server on nightly, wasting energy.

Try the Lights Out add-in.

- My WHS box turns on automatically when any of the machines on my lan are switched on (great considering its hidden away in a cupboard).
- When all clients on the lan are switched off/sleeping the WHS box hibernates itself after 10 minutes.
- Using a php script hidden away on my business site I can wake up my WHS box over the internet so I can still remote access.
- You can schedule times when the box should wakeup to perform backups etc.

 

[spectrumbx]

@pirivan

You have raised a really important point about the WHS + FlexRAID integration.

I had hoped that the user community would have come up on its own with the best implementation guide for it.
However, as you have seen and without a clear guideline from me, users ended up going in all sort of directions.

With FlexRAID-View finally out, my next two priorities are to standardize the WHS + FlexRAID integration and finish up FlexRAID Live!.
I am also going to be setting up a wiki site.

More likely than not, I will be installing WHS in a VM for testing this coming weekend.
I will then be able to sort through everything and provide a clear set of guidelines for the integration.

As far as your other concerns, the beauty of FlexRAID is that it is very easy to test out, and it imposes no commitment that you would need to reverse out of.
You don't even need a new drive for the parity space as long as you have some free space somewhere to use for the parity data.

You also don't need to run the WebUI and its scheduler.
I have instructions on how to scripts the command line client: http://www.openegg.org/forums/posts/list/67.page
You could have your script run, for instance, right before the box shuts down.

That said, the WebUI's scheduler is very powerful and can be configured to run the updates during your "hours of operation".

 

[XS Janus]

I would just build another windows raid6 server...
I see no point in fiddling with the unknown.
Chances are that when you're gonna need it, you won't be familiar with or have a large enough user base to help you out.
Just MO

 

[thesmokingman]

Slap some drives into those AMS boxes and RAID5 them up, now you have two 6TB backup arrays. Buy a fire and quake proof safe and keep one array box in there for cold storage and keep on plugged in. Swap them every couple of months or so. You should figure out whatever swap ratio to time that is easiest for your lazy meter. Also, you can also use some cheap or free online storage provider for backing up just your documents. Price is about 1K (@newegg) for 10 1.5TB drives and whatever the safe is? The AMS R5 boxes with raid cards are about 220 at newegg atm just in case.

 

[epimetheus]

To update on the setup I mentioned above, I just built a second WHS using a Chenbro ES34069 and a mini-ITX mobo. Started off with a Jetway mobo for the SATA daughtercard, but the SATA chipset and the Realtek NIC sucked balls. Ended up with a Comell LV-67A mobo from Global American. It's way overkill for a backup WHS, but it works like a dream. Took a little over a day to backup almost 5TB. I set the backups up with Comodo Backup running on my main WHS administered through the console and Advanced Admin Console.

 

[pirivan]

@orubap

Good idea, if I decide to implement FlexRaid at some point I will keep Lights Out in mind. Though by the time I implement FlexRaid, FlexRaid Live! might be out and I won't have to worry quite so much about scheduling parity calculations!

@Spectrumbx

Thanks for the reply! I am glad I'm not the only one who felt confused by all the (mis)information about a FlexRaidWHS integration. The potential is really there for this to be an amazing combination with efficient disk space usage, recover ability and dual parity features of FlexRaid combined with the drive pooling and ease of management of WHS. I was discussing this with a co-worker today and agreed that it's extremely unfortunate that Microsoft doesn't hire someone like you to develop a 'dual parity' flex raid like system as an option for WHS. Disk duplication would be great to still have as an option but it does get pricey (especially if you also want to backup off-site).

Anyhow, sounds great that you are working on WHS + FlexRaid and FlexRaid Live along with a wiki. FlexRaid Live sounds like a wonderful piece of software and wiki documentation on top of that; couldn't be happier. Let us know how things go with FlexRaid + WHS in your testing; I am sure many users could benefit from clear information. I may give playing around with it a shot on another box just to get a feel for it. I will probably use the webUI, as I would openly admit that my scripting abilities are beyond poor.

@XS Janus

I mostly agree with you here. I want to feel 100% confident if I need to do a recovery; there is no worse feeling than realizing you might lose data. In particular when you have so much data that it would be very hard for you to tell that you lost a few files here and there. That is why I like the WHS duplication recovery so much. Drive dies, remove it physically, slip in new drive, tell WHS to remove the "failed" drive and it reduplicates to empty space. That is damn slick. However, I don't think I am going to build a whole other server with RAID6 that I can have here locally, back up to and then take off site. That seems a wee bit overkill.

@thesmokingman

Based on the responses here I am leaning towards something like this solution. I admit I was hoping that I would be able to save some money and go with the tape solutions I was discussing (drive cost is high but tape cost isn't bad) but it seems pretty clear either many people around here either don't do it/know anything about it for home usage or don't think much of that idea due to the complication of tapes/tape drives. I may not do something quite as complicated as you suggested though (complicated/safe = $$$). It would be nice to have the extra redundancy of RAID5 on backups but it might be a wee bit pricey as it costs you that extra disk for parity in the backup. Also, wouldn't I run into an issue here with volume size? Wouldn't WHS not be able to see anything larger than a 2TB RAID volume on my 5-bay AMS enclosure? It might be better just to have it in 'JBOD'.

I also concur that it would be great to have a 'hot' backup here in the AMS and a 'cold' one off-site in a safe. That would highly secure (and handy) however I don't think I can currently afford that level of security (aka dropping 1K just on drives). Plus of course it gets pricier as I add drives to the Norco 4020 and thus have to add additional hot and cold backup enclosures. I may have to just go with 'cold' off-site backup enclosures for now due to cost. At least I already have the 2 AMS 5-bay enclosures; so no cost for those.

@epimetheus

How big/how many drives did you throw in the backup enclosure? How are you connecting them for the backup; just over gigabit? If I decide to go with the AMS eSata enclosures I will probably use syncback or something to sync all the data to various drives in the enclosures. It will take a while the first time, but at least after that it won't be bad. How much did it end up costing you for the entire setup? Did you end up putting it off-site for 'safety storage' after you sync'ed it?