Large backups to external media

T

the-one1

2[H]4U
Joined
Jan 16, 2003
Messages
2,982
I need to backup 500GB of data from a server onto an external media. Right now I am using 700GB USB drives, these are 3.5" full size drives. I calculated that it takes ~20GB per hour. So now the backup takes 25+ hours to finish and that is not even including the verification. The server has a RAID 5 w/ 5 drives and one spare.
Is there a faster way to get the data of the server an onto something external so that it can be taken off site?
I know I can do incrementals which will cut the daily backup time drastically. But I am using Symantec Exec System Recovery 8(in case you don't know what this is, think of GHOST) and it takes almost as long to restore as it does to backup and I would prefer to have the system restored in one pass as opposed to restoring the Full backup then restoring the last incremental.

I was thinking about either firewire drives or e-sata, but what is the sustained transfer of those in GB/Hour.

Is there a way you know of a that will make everything more efficient in backing up and restoring? All suggestions are welcome.
 
Part of the problem I've been trying to figure out as well.
My solution has thus far (still have yet to implement) to a storage server on the LAN. Then back that storage server up to external hard drive after X hours when the production box is done backing up.

I'm going to keep an eye on this one, see if anyone has anything better.

My problem is I can't drop insane amounts of money on T3 or something like that with decent speed to backup off-site (And you'd need one on the other end too....)
 
Well I actually have 2 customers with this problem. The other customer had 250+GB of data that gets backed up to a storage server. It takes 9+ hours to backup 110Gb per server to the storage server. Bandwidth is not an issue, network usage per server never gets above 40% . Then when the backup is done, a batch file copies the data from the storage server to an attached 500GB USB2 drive. But that also takes an awful long time. 4am til at least 3pm just for the copy.
 
the-one1 said:
Well I actually have 2 customers with this problem. The other customer had 250+GB of data that gets backed up to a storage server. It takes 9+ hours to backup 110Gb per server to the storage server. Bandwidth is not an issue, network usage per server never gets above 40% . Then when the backup is done, a batch file copies the data from the storage server to an attached 500GB USB2 drive. But that also takes an awful long time. 4am til at least 3pm just for the copy.
Click to expand...

Same problem I'm running into, exactly.

Shortly I'll have 3 production servers just on this LAN and I'm trying to figure out how to A) back up overnight and B) external media I can move to safe each day.

If it was just storing it on a server there would be no problem. Challenge comes when I want to fire/water/tornado protect my backups. Can't just move a server every day.
 
the-one1 said:
Well I actually have 2 customers with this problem. The other customer had 250+GB of data that gets backed up to a storage server. It takes 9+ hours to backup 110Gb per server to the storage server. Bandwidth is not an issue, network usage per server never gets above 40% . Then when the backup is done, a batch file copies the data from the storage server to an attached 500GB USB2 drive. But that also takes an awful long time. 4am til at least 3pm just for the copy.
Click to expand...

I have a backup solution that would be great for you, PM me, Blocks level synthetic incrementals.
 
k1pp3r said:
I have a backup solution that would be great for you, PM me, Blocks level synthetic incrementals.
Click to expand...


Can you put your solution here since apparently more than just me that need a solution to this "problem"? Thanks.
 
the-one1 said:
Can you put your solution here since apparently more than just me that need a solution to this "problem"? Thanks.
Click to expand...

Ditto.

Mine comes into problems when I need more than 1TB of storage PER SERVER. Not there yet but eventually I'll hit that mark and I need to be ready.
 
the-one1 said:
Can you put your solution here since apparently more than just me that need a solution to this "problem"? Thanks.
Click to expand...

Its a solution my company sales which is why i asked for the PM, its not a home solution which is what i would like to avoid getting requests for.

A little information is below.

Small to medium size business backup solution with offsite data storage.

The initial image for the backups is taken and transfered to a Colo offsite by hard drive.

After this, incremental backups run (as little as 15 minutes apart) Shadow copy backup once per day, at the end of the week, the incrementals collapse into a weekly synthetic incremental file. Then weeklys collapse into monthlys, this prevents the time consuming task of taking full backups. The incremental backups can be mounted to a logical drive for quick file recovery at any of the backup points.

All incremental backup files are encrypted on the backup server.

The data is encrypted again with a 128 bit AES encryption and transfered offsite to the colo over the internet, which is then replicated to a second colo on the other side of the US.

In the event of a server hardware crash, there are two options.

1) Virtualize the server off the backup images, (this can be done in as little as 30 minutes)
2) Do a bare metal restore to dis-similar hardware

Like i said, this is NOT a home solution, this is for SMB,

I hope i'm not breaking rules by posting this, the cost associated is very competative PM me if you would like more details.
 
Are you looking for an open source solution or solid commercial solution?

Open Source... bacula would best bet as you'll be able to have parallel data streams allowing you to fully utilize hardware / network resources. From my using of it, it gives similar performance as EMC Networker when setup properly and can be configured to work in pretty much every environment. It's not horribly hard to configure, just take your time a read the docs first.

Commercial ... NovaNET 11 would work really well for you with a combination of NovaNET-WEB to automate your offsite backups. You can get more information here
 
MrGuvernment said:
get off of USB and use eSATA ? or even FireWire?
Click to expand...
I don't think its the interface that is my problem. I think it's the actual drive inside the external drive that is limiting me. I get about 20-25 GB/hour. Anyone know how fast that is in MB/s?


I know in my original post I made it sound like I was looking for a personal solution. Well, This is actually for a couple of my customers.
When "synthetic incremental" was mentioned, I googled it and was surprised at what was available out there.
 
the-one1 said:
I don't think its the interface that is my problem. I think it's the actual drive inside the external drive that is limiting me. I get about 20-25 GB/hour. Anyone know how fast that is in MB/s?
Click to expand...

Here's a good question I think: Any solutions that can pack in multi-GB/s? USB 3.0 isn't quite here yet.

As for the MB/s... 20-25GB/hr is 20,000-25,000MB/hr. Divide that by 60 gives you minutes... 333-417 MB/min...... Divide that by 60 and that'd be seconds. 5.6-7.0 MB/s.

Also, some backup software can only work so fast. Some compress it, do integrity checks, etc... Might not be straight copy operations.
I know my TrueImage uses a triple buffer, which enables it to give me a sector by sector backup while Windows is running. With that overhead it won't be as fast as just copying your data files around.

All that said, with my backup solution (USB) it's got me at 53GB/hr. My problem is doing this to 3+ servers which I will be here shortly, AND still wanting to get all that onto a single drive for secure storage.
 
Well, here is more detail of what I do. i backup to a central server, then xcopy that backup image file onto a USB2.0 drive. That part takes 20-25GB/h. If I backup directly to a USB drive, it is then 50-55GB/h, but I can't do that as each server is a blade (not my choice on going this route).
 
We had the same issue we moved from USB to Esata cards in the server and now backup to 1.5 Tb drives.
 
[H]ump;1033395398 said:
We had the same issue we moved from USB to Esata cards in the server and now backup to 1.5 Tb drives.
Click to expand...

Good idea. But you've still got to do like our problems are and deal with the backup time to the central backup server.

OP: Question for you is when do you run the XCOPY?
 
USB 2 drive you may get a MAX if your dam lucky of 30MB/s the most i have ever seen myself on various drives is around 24-25MB/s , so yes USB would be limiting you vs the speed of modern hard drives easily hitting 60MB+
 
OP: Question for you is when do you run the XCOPY?[/QUOTE]

Backups run from 5pm til 3am. XCOPY is set to run from a batch file at 4am and when it finishes, it emails me that it's done and is ready for the drives to be swapped out.


[H]ump: Esata cards, I'm assuming it's hot swappable just like USBs, If so, I might look at that.
 
You must log in or register to reply here.
Back
Top