• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

A dual backup medium diff scenario

M

mbrownnyc

Weaksauce
Joined
Jul 2, 2008
Messages
108
Hello,

I have a slightly interesting problem.

The CEO of my company wishes to have a full backup of data at two locations, at work and at home.

The data that is changing is not file based. There are many files, where only a small portion of information is added. Therefore, a full backup solution can't be done.

He wishes to only move diffs of these files, and back and forth. Meaning, the "master" copy changes.

What is the best method to do this?


Thanks very much!

Matt Brown
 
Last edited:
Well you didn't mention what platform you are on but something like Unison, rsync, rdiff-backup or robocopy should work.
 
Last edited:
He'd be using Windows 7. Cool. I was thinking rsync, but I'm not sure how it would work with the "two master file set" issue.

Any ideas on how I can handle this? Could I just bring over the DB or something?


Thanks,

Matt

[edit]

One more thing to mention is that the two machines have no network connectivity.
 
Well as far as I can tell you have 1 master (the main copy) and then two backups. If the DB is really big just put a copy on a external drive and copy it to the two backup locations and then rsync the changes periodically (daily?). Just sync with each back up location separately from the main location.

Will people be changing your backups also, then you need something that can sync both ways like unison.

I've never used rsync with windows so I can't offer any advice there. good
 
s0rce said:
Well as far as I can tell you have 1 master (the main copy) and then two backups. If the DB is really big just put a copy on a external drive and copy it to the two backup locations and then rsync the changes periodically (daily?). Just sync with each back up location separately from the main location.

Will people be changing your backups also, then you need something that can sync both ways like unison.

I've never used rsync with windows so I can't offer any advice there. good
Click to expand...

Thanks for replying so quickly.

I guess I will have to test... but think of it this way:

1) Dude works on "Live file set 1" on his computer at work
2) Dude backs up files to RAID1 at his desk at work ("Master copy 1")
3) Dude brings backup home and puts them into "Live file set 2"
4) Dude backs up "live file set 2" to RAID1 at his home ("Master copy 2")
5) Dude modifies some files in "live file set 2"
6) Dude back up diffs from "live file set 2" to "master copy 2"
7) Dude wants to merge changes from "live file set 2" to "live file set 1"

How do I do that? Am I making this more complex than it needs to be?


Thanks,

Matt
 
i'm not sure rsync would work well in that case.
sounds like you need SVN/CVS
 
s0rce said:
Well as far as I can tell you have 1 master (the main copy) and then two backups. If the DB is really big just put a copy on a external drive and copy it to the two backup locations and then rsync the changes periodically (daily?). Just sync with each back up location separately from the main location.

Will people be changing your backups also, then you need something that can sync both ways like unison.

I've never used rsync with windows so I can't offer any advice there. good
Click to expand...

ripken204 said:
i'm not sure rsync would work well in that case.
sounds like you need SVN/CVS
Click to expand...

I see... any idea on a solution to handle binary files?
 
You could try unison. It would probably work for you if you don't need versioning.
 
Okay cool. I'll test this. Hopefully it generates a DB of checksums, then can perform the necessary diffs of the files... I guess, we'll have to end up forcing replacement of the DB from the "i hold the master!" set of the files.

Thanks for your input!

Matt

[edit]
Oh... I just had one of those big picture revelations...

There are many solutions that will perform diffing of files, but replicating deltas on a binary file seems to be the requirement and a difficult requirement at that.
 
Last edited:
I'm not sure unison can create the diff files I think it needs an active connection to sync.
 
Ahh, yes. With always on connections, I'm very partial to DFS-Replication, which is a feature of Windows Server.

It generates a checksum data of files, holds file versions, watches the file system journal for changes, when detected stages the files, compresses it and sends it over the line, stages it locally, compares the checksums again, and pushes the changes into the production file.

I can't think of a better way to do this. It utilizes an ESE database, which seems pretty efficient.
 
You must log in or register to reply here.
Back
Top