4TB Backup

T

Torrevez

n00b
Joined
Oct 11, 2010
Messages
3
Hi Guys, I need some advice on what program to use to do a large back up, I have about 4TB of data spread across various different drives on my server and basically I want to back it up to some empty hard drives I have.

I dont need anything special no split archives or anything. I just need it to do a full backup say every 3 months and to do incremental backups every week. Raid is not really an option as most of the drives are miss-match.

I have tried a few programs, Acornis true image and the backup software that come with windows. The problem I am having is that they are estimating about a 5 days to do a full backup, so I left it for 5 days and it was still only at 60%. I was looking for something that would be able to do a full backup of my largest drive (2TB) over night. If I just copy the files in windows explorer it does take several hours but not days, several hours I can work with, unfortunately copy and paste isnt practical I need an automated backup.

Thanks
Daniel
 
WHS is brilliant, I have played with it before. Didnt really want to hassle of reinstalling my OS. My server is running Win XP at the moment.
 
Raid is not a backup, though I would recommend it anyway, if you have that much data. Hard drives are so cheap nowdays, see if you can save up to buy 4 2TB drives and make a 6TB raid with them. You could backup your existing drives to that array. A form of "powered off" backups is good to have too.
 
You can make up a couple, one for the full copy the other for the incremental copies, batch files that will run a series of xcopy commands.

Does a nice job.
 
Well if most of your 4TB of data is static, you can copy it once for all, and set up Backup & Restore only for the dynamic folders that change frequently: The static file copy will take only hours, same for the initial backup of the dynamic folders, and the daily incremental backups will then take only minutes.

If all your folders contain a mix of static and dynamic data, for instance a "Comedies" or "TV Series" folder to which you regularly add ripped movies or episodes, Backup & Restore is not appropriate.

I'll have the same problem soon when I upgrade from RAID 6 to a mirror backup. I was planning on creating a small Perl script to run windiff between the original and backup folders from the command line and analyze the windiff log to take different actions for the new, deleted, updated files.

More or less recreating the backup part of Backup & Restore, except that the backup folder is always an exact mirror of the latest original folder and is fully readable from Windows Explorer, not hidden in unreadable backup files. All file movements like deleted and updated files are stored in a separate location, so any file or folder can be restored to any previous state.

Damn, I should troll-patent this concept!
 
I understand 4TB data. Could you please also provide estimate

1. How many files? example x millions of files. No need to be exact, estimate will do.

I am trying to understand whether it is small size files impact or large size files impact.

Cheers
 
I am currently in the same but slightly different boat. If anyone wants to consider solutions from different platforms. I.E. I have a Windows machine that houses the TB's of space. I would like to automatically backup files from a Ubuntu machine as well another Win7 computer. Later, I may add a separate machine that runs ubuntu or FreeNas but don't have the option to run wire (My server is my htpc, which is in the living room and for me to offload movies from it, I would like to stream HD movies to it via gigabit ethernet).

To the OP, it seems everyone recommends WHS. Another option that I am exploring is some sort of rsync backup for Windows.
 
The 4TB is mainly backup's of my dvd's and bluray's the server is generally used as a media server so we are take about copying large files not lots of little ones.

I did a bit of googleing the other night and came across robocopy. I managed to put a script together which checks for changes, if it sees one change and it has been more than 60minutes it copies all the changes.

Code: 
robocopy D:\Video I:\Video /MIR /E /R:1 /MON:1 /MOT:60 /XD "D:\system volume information"  /XD D:\recycler

The script looks like that and seems to work well, but am not sure how many resources this is taking from the server running all the time like it is.
 
Robocopy is extremely resource-intensive during the actual copy operation. I use /mt (multithread): It's still CPU-intensive and brings all tasks to a crawl, but for a shorter time.

There are faster alternatives that may or may not preserve the ACL and file ownership information, but that's usually not a concern for media files at home.
I'd be interested to know what other command line alternatives people are using, as large media files and multiple terabyte disks or arrays are becoming more common.
 
You must log in or register to reply here.
Back
Top