Copying VM Clone to remote host?

[Horpel]

We currently have a Vcenter setup limited to 3 hosts.  I am trying to create a system for disaster recovery assuming our main Center were to fail.

Thus, I need to schedule a clone of the VM to take place every week - but then I need to copy that clone to a remote host that is not within the same datacenter although it IS on the same network.

Is there a way to accomplish this aside manually removing a host from Vcenter and adding the (4th) remote host every time I want to make the copy?

If I copy the clone to an external disk, I have to copy it again to the remote host.  That's 2 copy instances over the network and that can take hours to complete.

I would like to reduce it to a single copy and perhaps even automate it with a script, but I'm not all that familiar with the CLI in ESXi.

Any ideas are appreciated.

[tomtom901]

You could use something like Veaam to achieve this.

[Horpel]

Thanks. Probably a swell product, but I'm trying to avoid adding any more into the fold. 

I thought there might be a command within ESXi that I could run like Rsync to accomplish this task.

...copying on directory to a remote directory over the network. 

[tomtom901]

Problem is that you are dealing with locks. Or is the VM available

to go down during this process?

[Horpel]

I don't understand what you mean by "Locks."

I've cloned the machine live, then copied the clone while the original VM was running.

Once I had the clone on the remote host, I powered down the original and booted the clone with success.

I can do this process repeatedly, but I want it automated. 

Cloning can be scheduled.  Now I need to automate the copy across our network using one step instead of two (the latter of which requires an external drive).

I don't want this process to rely on manual labor, but to process without human interaction.

[tomtom901]

Understandable, I thought you just wanted to copy over VMDK's but a clone first takes a snapshot of the VM. That removes the lock on the base VMDK. ESXi can run rsync, however it is unsupported. That might be a bit over your head since you have no Linux experience.

http://www.virtuallyghetto.com/2011/02/how-to-compile-statically-linked-rsync.html

What you could do however is make sure that the source and destination host have access to the same storage. Then schedule a clone of the VM, but leave it powered off at all times. When in DR, on the other host, go to the datastore, select the VM's folder, rightclick the .vmx file and say add to Inventory. You can then boot it up on the other host. Or does the DR host only have local storage?

[Linjo]

Sounds like vSphere Replication could be a good fit for your needs, see more here:

http://www.vmware.com/se/products/vsphere/features/replication.html

[tomtom901]

Hosts are not in the same vCenter. Can vSphere Replication then still work?

[Linjo]

I believe it does:

http://www.vmware.com/files/images/vsphere_imgs/vmw-dgrm-vsphere-replication-image3-lg.jpg

[admin]

Cloning a virtual machine disk without delta files or snapshots

To clone a virtual machine disk using the ESX/ESXi host terminal:

1. Log into the terminal of the ESX/ESXi host.
   
   
2. Navigate to the virtual machine's directory using the cd command. It is located at:
   
   /vmfs/volumes/source_datastore/virtual_machine/
   
   
3. Confirm the destination directory where the clone will be copied to. Create this directory, if required.
   
   For example, if this destination directory does not exist:
   
   /vmfs/volumes/destination_datastore/virtual_machine/
   
   Create the directory using this command:
   
   mkdir /vmfs/volumes/destination_datastore/virtual_machine/
   
   
4. Clone the virtual hard disk using the vmkfstools -i command after shutting down the virtual machine:
   
   # vmkfstools -i "/vmfs/volumes/Storage1 (3)/examplevm/examplevm.vmdk" /vmfs/volumes/Storage2/examplevm_clone.vmdk
   
   Notes:
   The output appears similar to:
   
   Destination disk format: VMFS thick
   Cloning disk '/vmfs/volumes/Storage1 (3)/examplevm/examplevm.vmdk'...
   Clone: 100% done.
   • Encapsulate objects with quotes where appropriate to ensure spaces and other special characters are interpreted correctly.
   • By default, the vmkfstools -i command will create a destination thick provisioned disk. If you would like the destination disk to be thin provisioned, add -d thin to the end of the above command.

Cloning a virtual machine disk with delta files or snapshots

To clone a virtual machine disk using the ESX/ESXi host terminal:

1. Log into the ESX/ESXi host's terminal.
   
   
2. Navigate to the virtual machine's directory using the cd command. It is located at:
   
   /vmfs/volumes/source_datastore/virtual_machine/
   
   
3. Confirm the destination directory where the clone will be copied to. Create this directory, if required.
   
   For example, if this destination directory does not exist:
   
   /vmfs/volumes/destination_datastore/virtual_machine/
   
   Create the directory using this command:
   
   mkdir /vmfs/volumes/destination_datastore/virtual_machine/
   
   
4. Clone the virtual hard disk from its current snapshot delta point using the vmkfstools -i command.
   
   # vmkfstools -i /vmfs/volumes/Storage1 (3)/examplevm/examplevm-000003.vmdk /vmfs/volumes/Storage2/examplevm_clone.vmdk
   
   The output appears similar to:
   
   Destination disk format: VMFS thick
   Cloning disk '/vmfs/volumes/Storage1 (3)/examplevm/examplevm-000003.vmdk'...
   Clone: 100% done.
   
   Note: If the process fails, try selecting the next snapshot delta point in the snapshot tree. The selected point or one of its parents may be corrupt.

Message was edited by: Wil van Antwerpen The original text can be found here: http://kb.vmware.com/kb/1027876

[Horpel]

Thanks Tom and Linjo.

I will read up on the replication tool ASAP.  So far it sounds like a great solution.

Tom, I have experience using RSYNC on Netgear devices via SSL from a command line.  It copies from the Netgear device to a remote location in another sate.

So you could be correct if it's any more complex than that.

I'll also look over the instructions from Fillips.

One thing of note is that the remote host is not part of a Vcenter instance.  It is an independent ESXi installation managed only through the Vsphere client.

[tomtom901]

To run rsync on VMware ESXi you have to manually compile the binary yourself. So that might be a bit over the top. I'm not quite sure how vSphere Replication works in this kind of enviroment (one vcenter vs single host) but I'll look into that and let you know.

[Horpel]

It is my understanding, after reviewing some of the requirements, that Replication must be deployed on both the sending and receiving hosts.

If that is so, :smileysad:  Here we have our answer...

It does not say here that it requires VCenter, but why note it if not necessary? 

Thus, I think it is implicit:

Operational Limits of vSphere Replication

vSphere Replication has certain operational limits.

To ensure successful virtual machine replication, you must verify that your virtual infrastructure respects certain limits before you start the replication.

■	You can only deploy one vSphere Replication appliance on a vCenter Server instance. Deploying more than one vSphere Replication appliance is not prohibited, but might lead to unexpected results.

■	Each vSphere Replication management server can manage a maximum of 500 replicated virtual machines.

[Horpel]

I'll also note that I'm running Vsphere 5.1 and I don't see Replication anywhere - although documentation claims to have included the software within 5.1...

I'm downloading the component now to test.

[Horpel]

Update:

1. I have confirmed that Replication requires Vcenter.  So this solution is not going to work.

2. I am starting to look into this from Tom:

What you could do however is make sure that the source and destination host have access to the same storage. Then schedule a clone of the VM, but leave it powered off at all times. When in DR, on the other host, go to the datastore, select the VM's folder, rightclick the .vmx file and say add to Inventory. You can then boot it up on the other host. Or does the DR host only have local storage?

My question about the above brings me back to what I'd originally wanted to do.  I thought that it might be possible to mount network storage to ESXi in some fashion.  I have yet to see this described anywhere, but it would make this very simple to be able to schedule a clone to mounted network storage.  I say this because you are correct, they do not have access to the same datastore - not at the moment.  The main machine is within a blade system but the independent host is just a desktop computer running ESXi.  SO, that is my question: Can I mount a network storage device (Like Netgear NAS)?

3. And finally, regarding Fillips' suggestion...

I'm wondering the same thing.  When running those commands, can "destination datastore" be referred to as a network path like "/10.10.X.X/Datastore_name"...?

[tomtom901]

Yes, you can mount a NAS storage if it can use something like iSCSI (or with most NAS systems) NFS. Mount this as a datastore in VMware and configure a scheduled task to clone the VM to the other datastore. If the main environment goes down, you can add the VM to the other host by registering the VMX. NFS is the easier way (I guess) but iSCSI would be the better choice as you can use VMFS on the datastore.

Also, regarding Fillips suggestion, that would work, but you still need to access the data somehow in case of a DR. So the best, and most simple solution in my point of view would be to configure shared storage (even if it is a low budget NAS storage) between the 2 hosts and let vCenter clone the VM(s) via a scheduled task.

[Horpel]

Thanks Tom.

Great info.  I'll be working on that today.

[tomtom901]

Great, let me know if you have some more questions. I'll be around :smileyhappy:

[Horpel]

First, let me say thank you, Tom.  This has been one of the most rewarding "Community" experiences I've had.  I'm actually getting something done.

Next, I should mention that I'm not asking a question about every problem I encounter.  I had problems for the last 2 hours getting an NFS store created and mounted on each Host. But alas, it was Successful!

With that, I researched my next roadblock and did not find anything regarding management of the storage size of the NFS share.

The capacity is unknown and the free space is 0.00 B.  I can tell you that it's simply a shared NFS directory on my Win 2008 Backup server which hosts many other shared folders.  Quite simply, when scheduling the clone, there is "Insufficient disk space on datastore 'DataStoreX'.

Any idea what to do with that?

[tomtom901]

Hm, I've never used NFS on Windows 2008. But I've fired up my lab for you and tried this, without any weird issues. I've followed this guide:

http://www.vmwarearena.com/2012/07/create-nfs-datastore-for-esx-in-windows.html

How big is the partition on which you have shared the folder for NFS to use. Is there perhaps a firewall between the host and the NFS share? I've opened up TCP and UDP 111 and 2049 in my firewall. That worked without any problems.

[Horpel]

Heh, the author of that page coincidentally blocked the view of total/available storage.

My NFS services were already setup, so I only needed to do the second half of what is described in that link - but it did it exactly as he/she did.

The share exists on a Dell MD1000 with around 3TB of storage (Approx 1.8TB available).

No Firewall inbetween.

[tomtom901]

Allright, it works without any problems in my lab. If possible, could you configure a seperate partition for this NFS folder (in example 100 GB) and see if that works? Although VMware says they do not put a limit on the maximum size, I think the 3TB is too big. Are you running 2008 or 2008 R2 as source for the NFS?

[Horpel]

I think I found a hardware solution. 

My Vcenter machine is 2008 R2 and happened to have 2 TB of unused storage sitting beside it.  I hooked it up via SAS with a PERC 5/E and partitioned 2TB into 3 different volumes and left one drive untouched for a spare.

Once I configured NFS for the server, I added it to each host and it recognized 500GB from each without a problem.

I am cloning my VM to the storage as I type this!

I will let you know the results once I've had a chance to fire it up.

I believe this is the resolution I was looking for and I appreciate all of the help!

[tomtom901]

Should work without any problems, I think it was still hitting the TB limit on your last attempt. If you could mark answers as helpfull or correct, that should help others in the future, and please, do let us know.

[Horpel]

If you could mark answers as helpfull or correct,

Already done.

:smileywink:

[Horpel]

I've run into a problem here :smileycry:

I've accomplished what I needed to do except for a tiny piece of the puzzle.

I was able to clone the VM to the shared NFS storage.  No problem.

I was able to boot the VM from the independent host and it was useable, but understandably slow as molasses while running over the network.

I continued with the expected procedure of copying the cloned directory to the Independent host's Local storage using the "Move a file" function while browsing the shared datastore.  However, this is where I run into some issues.

First, it appears that the folder was indeed moved.  But some files remained on the shared storage (huh?).

After the move was complete (which required 6 hours), when I created the new VM using the previously configured disk, 2 things happened:

1. An additional directory was created on the same datastore and it contains a .vmx file, nvram file, a log file, and a few others...

2. The VM will not boot

I have recreated the VM several times and I keep getting the same results.

Why is the additional directory being created?

Am I missing something?

I have created VM's with an existing disk before and not had such problems...

[Horpel]

bump :smileysad:

[tomtom901]

Perhaps the move didn't go OK.. The extra directory is being created because you added a new VM, and did not import the one from the existing VM. Try that next time, right click the clone's .vmx file and choose Add to Inventory:

[Horpel]

No Kiddin?  Didn't know that was possible.  I always create a new VM with an existing disk.

I will try your suggestion.  Thanks again!