copy large folder with many files between two Vms

[keltik85]

Hi,

I have this VmWare vSphere Version:

[root@VMWareSvR2:~] vmware -vl

VMware ESXi 6.0.0 build-3825889

VMware ESXi 6.0.0 Update 2

I have 2 Virtual Machines running on it:

System A: Windows Server 2008 (not R2, System properties show it as Windows Server Standard FE)

System B: Windows 10 Enterprise

What I want to do is:

- Copy a big folder (150GB) with 2 Million files from the first VM (MS 2008 Standard) to the second VM (Win 10 Enterprise).

What I tried is:

Approach 1: Share a Network drive on System B and Copy&PAste the folder to it from System A.

Problems: Its slow 415 kb/s and takes 6 days.

I dont understand why the heck its so slow, because both VMS are on the same ESXI server?

Approach 2: Plug an external USB HDD into the ESXI-Server. Under System-A's VM-settings-> Hardware->Add USB device. Copy from System A to the external USB HDD. Then copy from the external HDD to System B.

Problems: Its also slow around 1 MB/s.

For Approach 2, I ended up checking wether USB 3.0 is enabled on the Esxi Server in general:

[root@VMWareSvR2:~] lsusb

Bus 001 Device 005: ID 13fd:1840 Initio Corporation INIC-1608 SATA bridge

Bus 002 Device 002: ID 1a5a:0005 Tandberg Data

Bus 001 Device 004: ID 046b:ff10 American Megatrends, Inc. Virtual Keyboard and Mouse

Bus 001 Device 003: ID 0480:b001 Toshiba America Info. Systems, Inc.

Bus 001 Device 002: ID 04d9:1603 Holtek Semiconductor, Inc. Keyboard

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

[root@VMWareSvR2:~] esxcli system module list | grep xhci

xhci                                true        true

It seems like USB 3.0 is enabled, but still both external USB HDD drives I tried out (in the example its Bus 001 Device 003: ID 0480:b001 Toshiba America Info. Systems, Inc.), always get connected to Bus 001, which is the USB 2.0 Controller.

How can I force it to be connected to the USB 3.0 Controller?

Is there a better way to copy that folder from System A to System B?

Help appreciated, nice evening.

[daphnissov]

Here's another option. Create a new, separate virtual disk on the source VM. Format it as, let's say, drive G. Perform a copy of these 2M files to this new drive G. Once complete, detach the disk from the VM and remove it from its configuration. Now attach that same disk to the destination VM and bring the file system online. You should have access to all those files without looping through the network or using a USB device.

[keltik85]

Thanks for that suggestion, I did mount a new virtual drive on that System A in the fashion you explained. After an hour or so the transfer rate drops to 1 MB/s. Thats still pretty slow. I will leave it for tonight, but I hope there are faster ways to transfer the stuff.

[continuum]

Did you consider to zip that directory first and then copy the zip-archive via network ?

[cnagrekar]

Why don't you attach one more disk to source server and once copied data locally to new drive just detach that vDisk from source server and attach it to destination server ?

[mskupin]

Hi. Regarding network copy. Are those VMs in the same VLAN (portgrouop)? If not, then network traffic will go outside of ESXi host. But as mentioned before, I would go with new virtual disk option.