Backup for 5-10TB+ Systems

L

liamjbenett

n00b
Joined
Dec 1, 2012
Messages
11
Hi,
I am currently the owner of 7TB worth of data and I am looking for advice on how best to back it up.

At the moment it all lives on my CentOS fileserver as a single non-RAID LVM volume (yes ban me for my stupidity) so I am looking a way to safeguard it all. I want to move this to a RAID configuration but I do that I want to make sure I have a true backup of the data,

What solutions would you recommend?
 
Sounds like your existing volume is the backup. I'd leave it be, isolate it, and make a new RAID in another system for daily use. Which RAID, I can't tell you. There are arguments against 3TB disks in RAID-5 because of the extended rebuild times and probability of read errors that could fail the rebuild. ZFS or the like may make more sense.
 
I wrote my own script that backs up based on the MD5 on the file. I have a database of what files (MD5) relatives have and don't backup those.

If your data is "common" or if you share with friends or relatives this could reduce your offsite backup-requirements quite a lot.
 
You have to keep in mind that redundancy is not backup!
I would get a couple of 3TB disks; mirror them (or even better do a RAID 0+1) for redundancy for your backup server with Linux (Ubuntu server ?) installed and run BackupPC on it.

The main thing is just getting the HDD capacity to hold the backups.
 
I would build a ZFS system to support those 3tb disk since ZFS is amazing on handling massive array and disk sizes.

I would use the ZFS platform as your main server. I would go with a Raid-z1 if I were you.
 
A lot of what is "best" will depend on your budget for the backup package. How much are you willing to spend? If you have some $$$ to spend and you want VERY reliable backup, Tape is the answer.
 
Thanks for all the comments. I really wish my budget stretched to tapes because long-term I know it's a better option and the price per GB is not that bad it's just that the drive itself cost a small fortune.

Warm back-up would be great but it could add to my relatives bandwidth costs and I don't think it's worth it.

Given that most of the media is write-once read very-rarely then I think external drives kept offsite (with a relative) would be the best option for me. I just need to think of a suitable solution, like the MD5 system described above so that I know my back-ups are up-to-date.
 
Crashplan works great as a free backup software. You could backup to an external drive or even better put drive on your relatives computer with crashplan after you seed it then backup over the net using crashplan.
 
First step is definitely start uploading to an online backup service.

Now that you're uploading, we can talk about how to introduce redundancy locally. There's no way around it: you need a lot of disks. You might need to spend a bunch of money on a NAS too. The details depend on the data and I don't know about that.
 
I started backing up about 2.4TB with Crashplan in September and I'm about halfway now in December.

I've got about 16TB that I'm working on getting backed up. I'm running RAID 60 with 14 RE4 enterprise drives and a hotspare on an LSI 9285-8e.

I use a couple services- I used to use Backblaze, but when I needed a restore it ended up being about 900 bucks for them to send a couple drives. And then- the Backblaze backup was only about 2/3 of the way finished when everything crashed; naturally the most recent stuff wasn't backed up yet, so I spend another 2k to have the whole thing recovered by 24HourData.

I keep my current frequently used documents in a Dropbox folder.
 
You're smart and you have money. I say write up your detailed requirements and email qnap, thecus, and another nas company asking what to do with what product of theirs, with what disks, and what is the warranty and support policy in case of failures and data loss. You can repost the results here for fun and discussion, but I bet you'll figure it out on your own.
 
I currently have a 12TB array with 9TB of data on it.

I decided to use CrashPlan+ to back it all up. Cost is extremely cheap at only $34 a year. It took 6 months to upload the 9TB on my 5mbit upload I get with Time Warner.

In my experience CrashPlan+ has worked great and I plan to continue using it since it is unlimited. I don't have to change anything as my array grows.
 
^I'm surprised Time Warner didn't flip a shit at the amount of uploading you did.
 
liamjbenett said:
Hi,
I am currently the owner of 7TB worth of data and I am looking for advice on how best to back it up.

At the moment it all lives on my CentOS fileserver as a single non-RAID LVM volume (yes ban me for my stupidity) so I am looking a way to safeguard it all. I want to move this to a RAID configuration but I do that I want to make sure I have a true backup of the data,

What solutions would you recommend?
Click to expand...

Don't use an online service. To stay in business they need to charge you at least for the hard drives. But more importantly the time and cost to restore from a service is a bit much.

---

I keep a backup at my daughter's house - in a tornado safe room.

I also keep a backup in my brand new safe deposit box.

Recovery time is measured in several hours.

---

7TB of data backups to $350 of hard drives. A safe deposit box costs $20 a year. Access and restoration times and costs are reasonable.
 
GeorgeHR said:
Don't use an online service. To stay in business they need to charge you at least for the hard drives. But more importantly the time and cost to restore from a service is a bit much.

---

I keep a backup at my daughter's house - in a tornado safe room.

I also keep a backup in my brand new safe deposit box.

Recovery time is measured in several hours.

---

7TB of data backups to $350 of hard drives. A safe deposit box costs $20 a year. Access and restoration times and costs are reasonable.
Click to expand...

Services like Crashplan are rather cheap since they offer unlimited storage. While restore time on something that large is steep, you can pay a fee that is not too bad to have a drive shipped to you. While yes the cost is more than what you have spent IF you use the dropship drive restore, however if you are patient on the restore it actually is not much more. If you also make a local backup, you really only need the online in the worst case condition.

The KEY difference is the backups are ALWAYS up2date, the safe deposit box requires you to constantly shuttle drives and for most people that is too easy to be a failure point.

Crashplan single computer unlimited storage is $36/year. No harddrive cost and only $16/year more than your safe deposit box.
 
m1abram said:
Services like Crashplan are rather cheap since they offer unlimited storage. While restore time on something that large is steep, you can pay a fee that is not too bad to have a drive shipped to you. While yes the cost is more than what you have spent IF you use the dropship drive restore, however if you are patient on the restore it actually is not much more. If you also make a local backup, you really only need the online in the worst case condition.

The KEY difference is the backups are ALWAYS up2date, the safe deposit box requires you to constantly shuttle drives and for most people that is too easy to be a failure point.

Crashplan single computer unlimited storage is $36/year. No harddrive cost and only $16/year more than your safe deposit box.
Click to expand...

Yeah, for me my data changes a ton so I like some sort of live backup. Set it and forget it.

Restoring can take some time yes, but with FlexRAID my array isn't striped so if I lose a drive or 2 and cannot recover from parity from some reason then I only have to download 1 or 2 drives worth of data, not everything.

Also, At least with CrashPlan you have random access to your whole backup. So while you are restoring, you can also just go immediately grab whatever files you may need right away and the transfer is likely as short as going to the bank to get a drive and connecting it and copying the file.
 
m1abram said:
Services like Crashplan are rather cheap since they offer unlimited storage. While restore time on something that large is steep, you can pay a fee that is not too bad to have a drive shipped to you. While yes the cost is more than what you have spent IF you use the dropship drive restore, however if you are patient on the restore it actually is not much more. If you also make a local backup, you really only need the online in the worst case condition.

The KEY difference is the backups are ALWAYS up2date, the safe deposit box requires you to constantly shuttle drives and for most people that is too easy to be a failure point.

Crashplan single computer unlimited storage is $36/year. No harddrive cost and only $16/year more than your safe deposit box.
Click to expand...

So somehow you got your local backup drives for nothing.

So somehow you avoided having a failure while crashplan was taking months to backup the files.

I do incremental backups everyday (files that have changed during the previous several days are saved in the daily backup. I get copies of each change for several days) - very little time or disk space required; no manual labor required. I could do continuous backups, but I don't have a need for them.

I visit my bank several times a week - depositing business checks. I can swap a recent backup for a less recent backup. For those who find it difficult to get to the bank, backups can be stored in any nearby substantial structure. Most of my neighbors have tornado shelters of one form or another.

---

Why would I store my data with a service that gives me disk space for less than the cost of the hard drives?

They have a poor business model. They will not be around for long.

But why would anyone want to backup 5-10TB of ripped DVDs?
 
I would have local backups with the safe deposit method too, so that cost is the same across both methods.

If the time it takes to backup is critical (note you should have made your local backups) then pay the ontime fee to dropship your backup. It really is not that expensive.
Otherwise prioritize your data so that the most critical is backed up first during the initial seed.

The chance of a failure during the initial seed vs the chance of failure to keep my manual handled safe deposit box backup up2date is probably worth it.

Seriously who visits a bank anymore, last time I went into a bank was 10 years ago because I needed a certified check for closing on a home. Heck anymore I do not even visit ATMs, use CC for most things get cash when I buy groceries and use online check deposits for those rare occasions I have a paper check to deal with.

Well Crashplan and similar have been around for a good bit and are becoming more popular, i would expect them to stick it out.

My backups are 1.2TB with most of that being photos I created. Now I can see why someone would want to backup ripped music CDs or DVDs is because they put a great deal of time ripping those items and cataloging them. Sure they can recover from the originals but they would have to put a lot of time back into it versus clicking a button and coming back later :).

Also I hope with these drives your are taking to the bank that they are incremental backups, if they are just disk copies that is almost as bad as not doing backups. Incremental backups are very important.
 
GeorgeHR said:
They have a poor business model. They will not be around for long.
Click to expand...

With customers including but not limited to: Google, Adobe, Cisco, Mozilla, Netflix, HP, LinkedIn, Citrix, ICANN, Pandora, Zynga, Kraft, Los Alamos, NASA, National Geographic, San Disk, Bank of America, Harvard, Stanford, Johns Hopkins, Penn, MIT.

I do not think they will be going out of business any time soon...


Your backup solution would be OK if I could fit all my data on 1 or 2 disks. But I'm at 9TB now and it grows every day. I would need at least 3 4TB drives to start and that would cost currently $900. Or 4 3TB drives which would be $600. It would take me 17 years of paying for unlimited CrashPlan to make up the cost of the backup disks I would have to buy. And then I would have to transport them every week at least to do updates and such. CrashPlan is set it and forget it and I like that.

Obviously I didn't have all my data backed up for the 6 month initial seed, but that's over and now I do have it all backed up. Plus, I backed up the most important data first and that was done very quickly. With a standard 5mbit upload I can sync an additional 50GB/day which is plenty to keep up with.

Obviously if you have capped Internet it's not an option, but plenty of people in the US have or can get unlimited Internet.
 
Last edited:
SirMaster said:
With customers including but not limited to: Google, Adobe, Cisco, Mozilla, Netflix, HP, LinkedIn, Citrix, ICANN, Pandora, Zynga, Kraft, Los Alamos, NASA, National Geographic, San Disk, Bank of America, Harvard, Stanford, Johns Hopkins, Penn, MIT.

I do not think they will be going out of business any time soon...


Your backup solution would be OK if I could fit all my data on 1 or 2 disks. But I'm at 9TB now and it grows every day. I would need at least 3 4TB drives to start and that would cost currently $900. Or 4 3TB drives which would be $600. It would take me 17 years of paying for unlimited CrashPlan to make up the cost of the backup disks I would have to buy. And then I would have to transport them every week at least to do updates and such. CrashPlan is set it and forget it and I like that.

Obviously I didn't have all my data backed up for the 6 month initial seed, but that's over and now I do have it all backed up. Plus, I backed up the most important data first and that was done very quickly. With a standard 5mbit upload I can sync an additional 50GB/day which is plenty to keep up with.

Obviously if you have capped Internet it's not an option, but plenty of people in the US have or can get unlimited Internet.
Click to expand...

Agreed-

And not to mention the daughter's tornado room likely cost $300/sf to build, the average American's wage is 30 bucks an hour, Drives cost $75/TB (and fail! and get killed when they land on the concrete!) and mileage is almost 60 cents a mile.

Crashplan and the likes are quite a good deal if you ask me.
 
SirMaster said:
With customers including but not limited to: Google, Adobe, Cisco, Mozilla, Netflix, HP, LinkedIn, Citrix, ICANN, Pandora, Zynga, Kraft, Los Alamos, NASA, National Geographic, San Disk, Bank of America, Harvard, Stanford, Johns Hopkins, Penn, MIT.

I do not think they will be going out of business any time soon...

It would take me 17 years of paying for unlimited CrashPlan to make up the cost of the backup disks I would have to buy.
Click to expand...

If it is going to take 17 years for you to break even on the cost of CrashPlan v. the cost of hard drives. It is going to take CrashPlan 17 years to break even. In fact CrashPlan will never break even since the life of hard drives is much less than 17 years.

---

It is unlikely that the organizations you list are "real" clients. The same math that I applied to your situation applies to any real client.

(Businesses lie about their client lists a lot.)

---

Your 10TB of data backs ups onto 5 2TB drives. That is about $500 nnot $900. You really need to learn to do sharper math. More importantly your data is not worth backing up anywhere.
 
Last edited:
GeorgeHR said:
If it is going to take 17 years for you to break even on the cost of CrashPlan v. the cost of hard drives. It is going to take CrashPlan 17 years to break even. In fact CrashPlan will never break even since the life of hard drives is much less than 17 years.

---

It is unlikely that the organizations you list are "real" clients. The same math that I applied to your situation applies to any real client.

(Businesses lie about their client lists a lot.)

---

Your 10TB of data backs ups onto 5 2TB drives. That is about $500 nnot $900. You really need to learn to do sharper math. More importantly your data is not worth backing up anywhere.
Click to expand...

The profitability of crashplan does not work out the way you describe. Many users will not take full advantage of the unlimited plan. And yes those are REAL clients, and the more enterprise clients crashplan gets the better the deals consumers will get.

Also if crashplan dies tomorrow, I have my local backups. I can just move them offsite until the next player comes along and I move to them. This is not a market that will be going away, in fact this is the market that will be growing a good bit. Really do not understand why someone would not take advantage.
 
GeorgeHR said:
Your 10TB of data backs ups onto 5 2TB drives. That is about $500 nnot $900. You really need to learn to do sharper math. More importantly your data is not worth backing up anywhere.
Click to expand...

I also pointed out why backing up ripped music and dvds is important to people.

1. Ripping and cataloging takes a good deal of manual time, restoring a backup of that data may take awhile over the internet but it is a click it and forget it task.
2. I have had over 400 CDs stolen in the past. If I had backups of my rips of those stolen CDs it would not have hurt so much. Many of those CDs were no longer in print so replacing them was not possible. Too bad that was before I had ripped them :(
 
The way that Crashplan works is the same way that companies like Time Warner can afford to let you upload a thousand GB a month - overselling. Same as for things such as cell phone networks, gym memberships, public highways, classrooms, ... Most consumers will not take advantage of the space, and the enterprise market (which praises things like reliability, centralized management, etc) ends up paying for a lot of it.

I think the average crashplan "unlimited" backup is around 20-30GB, and that's a high estimate (normal users simply don't have that much content to back up, or that they care about). The audience that posts to forums like these (or even reads these forums) is not average. The backups from my non-primary computers (parents, etc) are also around that size. Of course my personal Crashplan is on the order of 2TB, but most aren't.
 
Last edited:
jimhsu said:
The way that Crashplan works is the same way that companies like Time Warner can afford to let you upload a thousand GB a month - overselling. Same as for things such as cell phone networks, gym memberships, public highways, classrooms, ... Most consumers will not take advantage of the space, and the enterprise market (which praises things like reliability, centralized management, etc) ends up paying for a lot of it.

I think the average crashplan "unlimited" backup is around 20-30GB, and that's a high estimate (normal users simply don't have that much content to back up, or that they care about). The audience that posts to forums like these (or even reads these forums) is not average. The backups from my non-primary computers (parents, etc) are also around that size. Of course my personal Crashplan is on the order of 2TB, but most aren't.
Click to expand...

The audience here does not have more than 20-30GB of data worth backing up. The audience here has that much spare disk space lying around and most have friends who would store their backup hard drives. They can do a better more secure backup for free.

---

I have 6 spare hard drives (4-5TB total space) and a spare SSD (128GB) that are just sitting around that I could use for free backups. I see anough people on a regular basis that I could impose on them to store my backups off site. No cost.
 
GeorgeHR said:
The audience here does not have more than 20-30GB of data worth backing up. The audience here has that much spare disk space lying around and most have friends who would store their backup hard drives. They can do a better more secure backup for free.

---

I have 6 spare hard drives (4-5TB total space) and a spare SSD (128GB) that are just sitting around that I could use for free backups. I see anough people on a regular basis that I could impose on them to store my backups off site. No cost.
Click to expand...

You really do not understand how important automated backups is. By automating it I have near continuous backups. Can recover from a specific point in time, so even if I have a failure that goes unnoticed for several backups I can still recover the data. As for security, it is encrypted. I trust my data in a enterprise datacenter with a companies rep on the line more than I would trust it in my mom's basement.
 
GeorgeHR said:
The audience here does not have more than 20-30GB of data worth backing up.
Click to expand...


I can't tell now if you are trolling... How could you possibly know this? I have ~1TB of photographs alone. Loads more video that I have recorded, documents, software projects, research data, it all adds up. I'm sure many other people here have similar stuff too...
 
Stop replying and hijacking a backup hardware thread with remote backup arguments

Back on topic, lets talk about 8 drive NAS
 
NAS is not a backup solution. It is a storage device. A backup solution is a combination of software and procedure. I recommend using a solution that requires no human interaction after setup other than monitor the status of the backups. Ie the software should email you regular status updates. The solution should also do incremental backups and keep at least 3 months of versions. The solution should backup up to at least 2 copies with one being offsite.
 
I think online archiving is great for off-site backups. It's easy, and cheap. In my scenario, I currently have 6TB of data.

I have used Crashplan in the past, as well as BackBlaze...and here are my thoughts on the two and why I don't use either of those. BackBlaze only works on a computer OS, and not a server OS. My storage server runs Windows Storage Server 2012, so BackBlaze just doesn't work. Crashplan throttles your upload speed once you hit a certain point, which I think is somewhere around 200GB.

Yes, and average user of Crashplan only back ups 20-50GB of data. Businesses aren't on the same $36/yr plan, and pay much more than that.

I use AltDrive, which is unlimited...no file size limits...and no throttling. I have a HomeDrive folder, which are user backups at home. That is about 100GB, and is continually backed up. I have another group set up in AltDrive that back ups my music on a daily schedule, and videos on a weekly schedule. I backup based on importance. I also back up my HomeDrives and Music to an external, so I can quickly recover all of that.

Worst case scenario, I would have to rebuild my storage server, rebuild my array, install the AltDrive client, and restore everything. I have a 50 down, 5 up connection...so downloading is 10 times faster than uploading (and I upload to them at about 60GB/day or so.

Using a NAS to backup a NAS is great, but not cost effective. If you set them to replicate, then you will have highly available storage, but a tornado or fire could still make you loose everything. I have so much data that I don't like relying on others to hold my backups, and I don't like shuffling around drives. What I do keeps my backups consistent and hands off, once set up and going.
 
Never heard of AltDrive, but it looks interesting. Will check it out thanks.

How do you like the client software versus crashplan?

One complaint I have with crashplan is the client software can eat some memory. It has gotten better in recent versions, have you noticed this with AltDrive?
 
Well the reason CrashPlan eats memory is that it is compressing and encrypting your data, so AltDrive does it as well. Not sure if AltDrive is any better at it though, but my storage server has a quad core Xeon with HT, and 8GB ram...so resources aren't an issue.

I really haven't found anything I don't like about AltDrive...yet. The client software is not as "pretty" as CrashPlan, but the features are all that matter to me. I have heard some say that it's more for advanced users (which anyone backing up 1TB+ data could be considered advanced). I like that I can create my own 256 bit passkey (instead of the client creating it and spitting it back at me) and export it to a file (that can be later imported in if I need to re-install or recover). Never have to type or copy/paste the key.
 
I have already been looking at CrashPlan and it looks good. The initial seed(s) will take a really long time but it's definitely worth it. I am now concern regarding the upload throttling though so I guess i need to do more research on this.

Altdrive would be great also, but it doesn't seem to support operating in a headless environment which is something I need.
 
tycoonbob said:
Crashplan throttles your upload speed once you hit a certain point, which I think is somewhere around 200GB.
Click to expand...

CrashPlan has never throttled uploads for any users, not sure where you got that from. They state this regularly on their support forums.

I can attest to it somewhat in that I uploaded 9TB unthrottled at my full 5Mbit upload speed. I know friends with several TB as well who have even faster uploads that have never been throttled.
 
@SirMaster Maybe I am wrong on that. It's been a while since I have used CrashPlan, and I was thinking it was them.

@liamjbenett What do you mean it doesn't seem to support operating headless? My storage server runs headless, and AltDrive runs on that box without a problem. It runs as a service, so it will run even if you aren't logged on.
 
liamjbenett said:
I have already been looking at CrashPlan and it looks good. The initial seed(s) will take a really long time but it's definitely worth it. I am now concern regarding the upload throttling though so I guess i need to do more research on this.

Altdrive would be great also, but it doesn't seem to support operating in a headless environment which is something I need.
Click to expand...

For other people looking at starting their backups, CrashPlan will do a hard drive based seed. Search on their site for "Seed Service". They ship you an encrypted drive, you seed it with your backup then the load it onto the servers.
 
thedge said:
For other people looking at starting their backups, CrashPlan will do a hard drive based seed. Search on their site for "Seed Service". They ship you an encrypted drive, you seed it with your backup then the load it onto the servers.
Click to expand...

Doesn't this cost a decent amount of money?
 
You must log in or register to reply here.
Back
Top