Rsync / SyncToy / CPS / etc...

Asgorath

[H]ard|Gawd
Joined
Jul 12, 2004
Messages
1,253
Here's the situation.... at work, I have a few servers that house very important files. Each server has hundreds of thousands of directories and millions of small ~15KB - 250KB files. A few thousand files get added or changed daily. I need to do a daily backup to tape. The way I was handling this was using Symantec Continuous protection server and keeping a copy of these files on my backup server. Then going from that server to tape. Straight backup from those servers to tape is MUCH too slow...even over gigabit and fast hard drives it takes close to two days!

Here's the problem. Continuous protection server, although very nice in the feature department, is a flaming pile of garbage for this huge file set. It bogs down, has memory leaks, and in my opinion just wasn't designed for this large of a file set. As such, I'm searching for an alternative solution. All boxes are Win2003.

My lead contender is using DeltaCopy (rSync wrapper for windows) to keep the files synced. I can use This Method to take a shadow copy of the files and sync to that...that way if any files are open they'll still get sync'd. What do you guys think of this?

A few other contenders would be Microsoft's SyncToy (which I have found to be pretty robust), robocopy (which I haven't used much but hear good things about, and one of many 'sync' software from 3rd party vendors.

Any words of advice or recommendations to help me through this?

Thanks.
 
You can check out Hobocopy it uses VSS and has a lot of the functionality in Robocopy.

I use robocopy personally, but it's for overnight backups when the data isn't being accessed.
 
Personally robocopy I've found to be the best thing. It's super simple to setup and it's very 'set it and forget it'.
 
You could try using DFS to replicate the data to your backup server and then backup to tape from there. I've setup and used DFS for this and it works pretty well, only replicates the changes once the initial sync is completed. You can set it up to replicate throughout the day or after hours. It is "free" in that it comes with Windows Server 2003. Only thing is they revamped it in R2 so it is highly recommended that both servers be running R2 if possible. I haven't tested with pre-R2 so I have no idea how it works other than what I read....
 
What I have taken to doing is using a esata drive then use SyncToy or robocopy to create a backup set then disconnect the drive in the AM and connect the esata cable to the tape server which is an old desktop with a tape drive and run the backup to tape from the esata drive.
With today's huge datasets backing up across a network is a pita and not worth the hassle anymore.

I have been looking into DFS to do the replication also.
 
You could try using DFS to replicate the data to your backup server and then backup to tape from there. I've setup and used DFS for this and it works pretty well, only replicates the changes once the initial sync is completed. You can set it up to replicate throughout the day or after hours. It is "free" in that it comes with Windows Server 2003. Only thing is they revamped it in R2 so it is highly recommended that both servers be running R2 if possible. I haven't tested with pre-R2 so I have no idea how it works other than what I read....


another DFS user here. I use it to replicate all my user shares between my 2 DC's. Shares are mapped to users via the namespace rather than a path. If either one of my servers go down everyone still has access to their files. I backup the shares on one server to an external drive, the other server backups over an openvpn connection via deltacopy to a PC at my house, which I am going to move to one of our other offices since I'm done testing it and it works great.
 
Back
Top