Any Linux backup solutions that do the following?

M

mpeg4v3

Gawd
Joined
Sep 14, 2001
Messages
644
I've got a file server running 3x 1tb hard drives, no raid or anything, just straight up plugged in. I'd like to start backing up various files on it. Obviously getting 3x more 1tb drives to do a straight 1:1 backup is out of the picture, as I don't even need to backup all 3tb in the first place. I'd rather just buy another 1tb drive and have it be a dedicated backup drive.

In pursuit of that, I was wondering if anyone knew of any Linux based backup solutions that are web-based (as in, I set it up on the Apache server on my file server), and that will allow me to navigate the file structure of all of the drives and check off or select which files or directories I want to backup, and will then proceed to do so. Sort of like a web based file system explorer/backup hybrid. I suppose I could always just write one myself, but I'd much rather use an existing solution if possible.
 
Backuppc http://backuppc.sourceforge.net/

Very slick little app. It not only will backup your local machine to harddisk but can be used to backup external windows, linux, or mac machines using various methods from simple samba shares, to rsync over ssh. One cool feature that it has is it pools files so that if you have the EXACT same file in multiple places it only backs up the file once. This helps save space.

Just to point out I think backuppc covers every need you listed and then some. I use it to backup my local linux machine and 2 separate windows machines.
 
Agreeing with m1abram

I've installed and tested a large number of automated network backup programs and backuppc was the least poorly designed one I could find (how's that for faint praise).

It has some issues (like not storing checksums of files in the server pool) that cause it to scale poorly if you are backing up several machines simultaneously with large files. That said you can tweak it to avoid some issues with large numbers of hosts, and the other backup programs I have tried have more serious issues.
 
BadBlock said:
Agreeing with m1abram

I've installed and tested a large number of automated network backup programs and backuppc was the least poorly designed one I could find (how's that for faint praise).

It has some issues (like not storing checksums of files in the server pool) that cause it to scale poorly if you are backing up several machines simultaneously with large files. That said you can tweak it to avoid some issues with large numbers of hosts, and the other backup programs I have tried have more serious issues.
Click to expand...

Curious as to how storing the checksums in the server pool would solve the problem of backing up several machines at the same time with large files. That would still be an issue because you would not know the checksum of each servers file until the backup is complete. However I do know that backuppc does go back and remove files that it detects later are duplicates this happens by default daily.
 
I guess ideally I wish it could store all of the rsync hash in a sqlite database on the client and for the server side store all rsync hashes for every file in the pool in its own database (mysql/postgres whatever). Along with storing the size and last modified date as well.

So at time of backup the client checks its own database and sees what the state of its filesystem was at the time of last backup, and just parses everything on the system looking at size and modified date. Anything that has changed date/size wise then gets a new rsync hash calculated and sent to the server and the server compares those hashes with the ones in its own database. So if the server has already gotten this file from another client, even if the filename is different, it doesn't have to do any hash calculations itself and doesn't need the file to be sent.

The hashes could also be calculated for say every 5MB of a file rather than an entire file which could help out with something like email clients that use the mbox storage format, so you could avoid forcing the server to do a hash calculation every time someone does a backup and their giant 2GB mbox file has changed by 5KB.
 
BadBlock said:
The hashes could also be calculated for say every 5MB of a file rather than an entire file which could help out with something like email clients that use the mbox storage format, so you could avoid forcing the server to do a hash calculation every time someone does a backup and their giant 2GB mbox file has changed by 5KB.
Click to expand...

Not sure if I am reading this right. Are you saying only do a hash of 5mb of a file. That would not work well. Particularly with mbox because the data is always appended, so you will never get a change in the hash even if new mail arrives. One of the many issues with mbox is what you mentioned, backups very inefficient for backups. Course mbox has so many other issues. Does exchange/outlook still use mbox format?
 
You must log in or register to reply here.
Back
Top