Cheapest/easiest way to store archived data in the cloud? 5TB and slowly growing

I

idea

Gawd
Joined
Jan 24, 2005
Messages
615
I have a storage system (ZFS) at home that is approaching 90% utilization so I am looking to archive about 5TB of data that I almost never access. I am interested in paying for a service, rather than building/upgrading on-site systems.

My goals are:
  1. Very low price that is comparable to building/upgrading a storage server
  2. Speed is not a concern, minimum requirement is about 50-100KBps of bandwidth and I/O doesn't matter at all
  3. The data needs to be online and accessible over standard protocols (either NFS, SSH/SFTP, FTP, etc). No proprietary apps (unless you can convince me)
  4. All my data needs to be backed up with at least 2 copies. So either this cloud service should have a "backup to the backup" feature, or I need to pay for 2 independant services.

This is how I imagine a perfect solution, if only there was a service that sold it
  1. Cloud instance running Linux and SFTP server
  2. Tape storage (for cheaper price) - Pay as I go
  3. Filesystems are always online and mounted
 
Last edited:
backblaze is cheapest for that much data, I have ~13TB on backblaze and would honestly have paid for the HDD's of theirs I'm using up front to have them store my data for this cheap. I feel kinda bad for them storing this much for so little but I'm not dropping them any time soon so they'll eventually make $$$ off me.

I've never had to restore from backblaze but I think you have to use their app so... yeah, that might suck.
 
CrashPlan is also cheap. $3.96/mo to be exact for unlimited storage. I have 5Mbit upload and it always maxes out my connection. Not sure how much faster it can go. I have 13TB backed up to them currently.
 
If I was doing this

Buy an internet domain name.

Pay a local serivce to host it.

Pay for the hard drives that my data resides on. Pay a monthly fee to store the data.

Now, I am a client who is paying enough that they want to keep me happy. $100/month and both the service and I am happy.
 
For me it sounds like rsync.net delivers what you need. They're pricey though.
By your description I guess you're willing to pay around 499 USD per year for 5 TB?
 
olavgg said:
Oh and why Backblaze isn't backup... https://twitter.com/backblaze/status/221135129853181952
Click to expand...

I am not sure the question and answer refer to the same thing.

I believe that as long as your annual fee is paid your data is kept.

I believe that if you are 1 month late in making your annual payment, you should expect your data is still there.

I believe that if you are 6 months late in making your annual payment, you should expect your data to be gone.

But I could be wrong.
 
GeorgeHR said:
I am not sure the question and answer refer to the same thing.

I believe that as long as your annual fee is paid your data is kept.

I believe that if you are 1 month late in making your annual payment, you should expect your data is still there.

I believe that if you are 6 months late in making your annual payment, you should expect your data to be gone.

But I could be wrong.
Click to expand...

Nope. It's as-written. They expect the data to be "live" so they want you to check in at least once every 6 months.

https://help.backblaze.com/entries/21809372-What-happens-to-my-backups-when-I-m-away-or-on-vacation-

This makes sense. Backblaze isn't data archive. It's backup. If your backed up system doesn't talk to them for 6 months it'll be gone.
 
Well I know crashplan doesn't have this issue. They keep all your files, even files you have deleted as long as you pay.
 
Amazon glacier services could act as a secondary backup. its 1 cent per GB per month. Although that adds up quickly... 5TB would be 5000GB or $50 a month. This ignores the cost/time delay if you needed to restore data though. You may be able to look into bitcasa too. Even with increased prices they are still one of the cheapest harddrive-in-the-cloud options around.

The real answer is you won't get any thing cloud based that is both reliable and cheap. Far cheaper on a multi year time scale is to build your own server and either host at a friend/family house or if thats not an option then pay for a colo. Other option would be to invest in a tape backup system.

Crashplan and backblaze are not valid options for your configuration because they require you keep a local copy of your data.

What you could do is build a secondary server with ZFS and even though performance would blow nuts... You turn on deduplication and set compression with gzip to 9. You would want OS to be vanilla freebsd or preferably one of the original ZFS OSes like openindiana, solaris etc. to stop box from crashing due to lack of ram. This would achieve your speed requirements and would give you greater bang for your buck on backup. compression on an average smattering of files with LZ4 typically hits 30% (no media) so you could potentially get 35-40% compression on data on gzip and dedup would (potentially) free that up even more.
 
Last edited:
Very good advice in this thread. If I wanted to archive the data to the point where it's a pain in the butt to restore with proprietary apps (but at least it's backed up), it seems I would use Crashplan.

However, it seems what I want does not exist. I have been spoiled by so many Cloud offerings for such low prices (sometimes free!). However, getting my Linux box an additional 5TB+ of SFTP/always-on storage simply does not exist for a price low enough to justify for personal use.
 
I've never thought about it this way before, but could you (reasonably securely) co-locate a server at a rackspace provider and just sync with it? Just ship them a built server and let them plop it in the rack. You could even pre-populate the data before you sent it, so you aren't having to transmit the whole shebang over the internet.
 
iroc409 said:
I've never thought about it this way before, but could you (reasonably securely) co-locate a server at a rackspace provider and just sync with it? Just ship them a built server and let them plop it in the rack. You could even pre-populate the data before you sent it, so you aren't having to transmit the whole shebang over the internet.
Click to expand...

Most colo are reasonably secure (no unauthorized persons on the data center floor, security cameras etc.) and they are pretty good about power/internet reliability due to having generators etc.

You ship them/bring them a preassembled server they add to their racks and then you are good to go.

Pricing for fully redundant, 100mbps up/down connection tends to run ~$50 per 'U' of rack space. i.e. 2U server case is 100, 4U case is 200.

So imagine if you got several people together to all pool together resources to make a server with a 2U (or 4U if you can find enough). But a 2U case could get you ~20ish HDD or you could do a 2U JBOD case with a 1U control head server which I think would cap at around 50 drive slots. All of this would run several thousand dollars. But if you had a group of storage horder friends you could make a very nice ZFS box that backs up everything for cheaper than cloud storage.

You could even add cloud like functionality with owncloud.

(perhaps you can tell I have thought about this a bit before, hehe)
 
PinkyThePig said:
Most colo are reasonably secure (no unauthorized persons on the data center floor, security cameras etc.) and they are pretty good about power/internet reliability due to having generators etc.

You ship them/bring them a preassembled server they add to their racks and then you are good to go.

Pricing for fully redundant, 100mbps up/down connection tends to run ~$50 per 'U' of rack space. i.e. 2U server case is 100, 4U case is 200.

So imagine if you got several people together to all pool together resources to make a server with a 2U (or 4U if you can find enough). But a 2U case could get you ~20ish HDD or you could do a 2U JBOD case with a 1U control head server which I think would cap at around 50 drive slots. All of this would run several thousand dollars. But if you had a group of storage horder friends you could make a very nice ZFS box that backs up everything for cheaper than cloud storage.

You could even add cloud like functionality with owncloud.

(perhaps you can tell I have thought about this a bit before, hehe)
Click to expand...

That sounds like the most cost-effective solution. My question on security would be more digital than physical--how you would sync with the system. For security sake, you'd have to set up a FTP or something. For example, I just use network shares (Samba) at home, but that wouldn't be very secure over the internet.

I'm sure there are plenty of solutions, probably even rsync can do it via FTP.
 
Seems like Backblaze and Crashplan are the cheapest option ( I have personally used them both). I do believe with Crashplan though you can back up to say a friends machine with it for free. Pay the the hard drives or case for him and then you at least have an offsite backup for free minus the cost of the startup and maintenance.
 
idea said:
However, it seems what I want does not exist.
Click to expand...

Several people suggested that you colocate. That gives you FTP access. A short drive if you need to recover all of your data.

As most people will tell you: if you lose your data, it is worth the cost.
 
iroc409 said:
That sounds like the most cost-effective solution. My question on security would be more digital than physical--how you would sync with the system. For security sake, you'd have to set up a FTP or something. For example, I just use network shares (Samba) at home, but that wouldn't be very secure over the internet.

I'm sure there are plenty of solutions, probably even rsync can do it via FTP.
Click to expand...

You could either do SSHFS to mount it to your local drive. Do SFTP directly, or do an rsync over ssh.

If you did sshfs for example it allows you to mount it to your file system and you could drag n drop files, sshfs, besides inherent lag, behaves similarly to NFS share except the SSH portion provides encryption for the connection. SFTP is basically a way of copying files securly. Rsync can be done over SSH to provide security as well.
 
My $0.02 is I wouldn't trust critical backups to any company running a solution that is oversubscribing. That's going to be any company offering a flat fee for 'unlimited' or outrageous amounts of space. They're banking on their income outdoing their expenditures primarily through a large percentage of the customer base using less than they can provide at whatever the fee is.

When they can't, one of the following is going to occur:
- something unexpected (such as Backblaze's policy with regards to your computer not talking to them and them deleting your data) from a 'backup' company
- suddenly being told you must stop using their product due to some violation of their EULA (many such companies even have, somewhere hidden deep within their EULA, a way to claim you're in violation for 'overconsumption' or similar 'too many resources' words)
- going out of business, which is what happens to them when they can't compensate when just 1% too many users of their service use /way/ more than they predicted
- "accidents" where data disappears/is unrecoverable

This is also true for hosting providers. Really, it's true of almost anything where the flat rate offered for a service is less than the business' cost to provide it to anyone going over 'X' utilization. Especially in the computer industry, where /average/ consumption of resources over time on a per-user basis is the NORM, not the exception.
 
Depends on how much control you want... With only 5TB it wouldn't be that unreasonable to colo a 1U server with 4 4TB drives in raid 6 or 10. Can do Windows, Linux, whatever you want on it and have it in driving distance if a disaster ever occurs and you need the whole thing.

Getting really serious build one of those 4U backblaze like servers for 180TB before redundancy. :p (yes that has it's own risks of course)
http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/

Or trust someone else to do it right?
 
Last edited:
idea said:
Very good advice in this thread. If I wanted to archive the data to the point where it's a pain in the butt to restore with proprietary apps (but at least it's backed up), it seems I would use Crashplan.

However, it seems what I want does not exist. I have been spoiled by so many Cloud offerings for such low prices (sometimes free!). However, getting my Linux box an additional 5TB+ of SFTP/always-on storage simply does not exist for a price low enough to justify for personal use.
Click to expand...

Everything is possible, it depends on what you're willing to pay.
If a service like you wanted existed, what would you be willing to pay?
 
You must log in or register to reply here.
Back
Top