• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Virtual Removable drives

O

officeboy

n00b
Joined
Sep 28, 2011
Messages
38
I need to move a large amount of data from our production network to a virtual network, but they cannot be connected. I started off with a USB drive and on our existing physical computers it did a poor job (9MB/sec) but on the ESXi host it is even worse(6MB/sec).

So my plan is to have a throwaway guest* on the ESXi server that I can use to share a drive/VMDK on our production network, move data onto it. And then attach the VMDK to the servers that cannot have network access. This should give me speeds into the 30+MB/sec range. The problem I am having is cleanly detaching the drives without a reboot. Also the server copying data to the shared drives is generating errors about the connection being lost and that data may have been lost.

So my question is twofold, one; is there a faster way to transfer backup data from a physical server to an ESXi host, and two; is there a clean way to detach the drives.

Thanks
 
Last edited:
Why not just create an entire VM then copy whatever you need to it then do a transfer of the entire VM with FastSCP or copy the VM over somehow??
 
I'm not sure I'm reading you right, but are the 'no access servers' virtual? If so, can you convince management they will have virtual nics that are not connected to any physical nics? If so, you could copy the data from the production network to a network share on the transfer VM, go to the vsphere management interface, change the vswitch for the xfer host from the one connected to the real network, and select the special internal network that has no actual NICs attached to it, the move the data to the virtual hosts.
 
danswartz said:
I'm not sure I'm reading you right, but are the 'no access servers' virtual? YES If so, can you convince management they will have virtual nics that are not connected to any physical nics? They Do If so, you could copy the data from the production network to a network share on the transfer VM, go to the vsphere management interface, change the vswitch for the xfer host from the one connected to the real network, and select the special internal network that has no actual NICs attached to it, the move the data to the virtual hosts.
Click to expand...

I had considered this, but then I'm stuck with two network transfers. As the throwaway guest will reside on the same datastore as the "no connect" servers, it's just an on disk transfer at that point, and should be very fast.

Also I am about done with a 40gig test file and I'm still only getting 12.7mb/sec.. I duno if it is NTbackup, (not my usual tool) the network connection, or both together that is slowing me down.
 
Wait, so are you trying my idea or something else? I'm not sure I'm reading you right...
 
Your idea is what I was trying, except with just removing the VMDK from the live network guest, and then attaching it to the offline servers in need of the data.

I think I will still run into issues where the source server will complain that the network location it was writing to has disappeared and data may have been lost.
 
I am also open to suggestions on the fastest way to move 200+gb of data from a physical server to a virtual server. (When they can't be networked together)

Vmware p2v converter for the data drive only?
 
officeboy said:
I am also open to suggestions on the fastest way to move 200+gb of data from a physical server to a virtual server. (When they can't be networked together)
Click to expand...

Edit just re read the first post and USB is not what you want.

But are you setting up USB the correct way with the pass through? would is it so slow?

Also even with those speeds you have for 200GB it is not that much, you would have been done by now if you just waited?
 
AMD_Gamer said:
External USB HDD then attach it to the ESXi host and pass it through to the VM?
Click to expand...

officeboy said:
I started off with a USB drive and on our existing physical computers it did a poor job (9MB/sec) but on the ESXi host it is even worse(6MB/sec).
Click to expand...

I need to more of less shutdown things here, so the job will need to be done over a weekend or after hours, at 200gb that's about 6 hours for the backup and 8 hours for the restore.

But there is a fair amount of before and after work. And if could get it done on only a few hours that would be very nice.

(The data is users redirected folders. Maybe I can do an incremental backup and like you say wait for the main lump of data, and then hope that there are a lot less changed bits.)
 
Last edited:
I am confused. With my suggestion there is no downtime. I know you are copying the data twice, but that shouldn't be that big a deal, is it?
 
I need to move this data from one system to another and then the whole offline vs online servers changes vice versa. So once I start moving data no one can be allowed to access or modify files until the restore is done on the new system. So downtime will be equal to the time the whole process takes. This and my not wanting to spend the weekend working on this is why i'm looking for a faster way to do this.
 
Ok, P2V is about 3x as fast as anything else. Plus I may only need to do it once (that's 6x faster). I'll just P2V the drive, move it into the new server folder, add it and bam, my user data is brought over.

Only question, Is there any problem with having multiple VMDK's make up drives/partitions in a VM?

This is a my first serious VM project, so I'm a little fuzzy on what best practice is. But I had been planning on making one giant virtual drive and then partitioning that. But maybe a VMDK per planned partition would be a good idea. Then I could grow/shrink as needed, and could focus VM backup efforts on the boot drive, and let native backup take care of the other data.
 
Also of note, jumbo frames on the source makes P2V conversion VERY slow. And it doesn't appear to make any real difference in file copy speed or NTbackup speed. (Backup exec likes it though)
 
officeboy said:
Also of note, jumbo frames on the source makes P2V conversion VERY slow. And it doesn't appear to make any real difference in file copy speed or NTbackup speed. (Backup exec likes it though)
Click to expand...

did you set jumbo frames on your switch and the destination?
 
Yeah, jumbo frame test pings had been good. I was surprised but it's not the first time Jumbo frames have caused problems.
 
You must log in or register to reply here.
Back
Top