Hi all
I have several Esxi 5 hosts (free) which all have their own local Datastores. A few months ago, I got myself a Qnap NAS (459URP++ 8TB RAID 6). I used the NAS to provide secondary Datastores (iSCSI Targets) to each one of my Esxi hosts. Using the VCBghetto backup script, I'm making weekly backups of all my VMs - storing up to 4 versions of each VM. This has been working very nice.
However, being a paranoid SysAdmin, I remained restless because I wanted to have a secondary backup of those iSCSI Targets in case my NAS would somehow become inaccessible or damaged (I usually store 3 backups of any user file or server in different locations).
As far as I know, there are only 3 ways to backup an iSCSI Target:
1) Make a backup of the whole LUN
2) Make a backup of the individual files inside the LUN
3) Make a backup of the partition residing inside the LUN
I tried 1) which kinda worked but is awfully slow on a QNAP NAS. Even though I'm using two aggregated GBit NICs, The NAS won't backup LUN's faster than 20Mbytes/sec over Samba or 2Mbytes/sec over NFS. It would need well over 10 hours to backup a 1TB Lun. This wasn't acceptable so I gave up on that and moved to 2)... well this was much faster on NTFS and EXT4 LUNs (around 35-40MBytes/sec on the first run) but was very complicated with VMFS datastores as neither Windows or Linux boxes can read the filesystem. I tried using vmfs-tools (fuse) but even with the latest release, it would mix up some VMs and place them in awkward directories, not to mention that some files were unreadable.... I gave up on this option too.
Having only option 3) to work on, I researched for a tool that would backup an entire partition under Linux regardless of its filesystem type. dd came up immediately in my mind but it is rather inefficient as it reads the entire filesystem space and handling sparse files isn't exactly its strong side. So I researched further until I found partclone... version 0.2.44 is not only able to read EXT4, NTFS and a bunch of other filesystems but also VMFS! I thought: great! I'm going to use this and it does in fact work very well backing up VMFS partitions (I get around 60MBytes/sec over a Gbit connection).The best part is that it only stores the used parts of the partition saving quite a lot of time during backups.
Now that I had images of all my VMFS Datastores, it was time to test a restoration... which worked on the partclone side (giving no errors) but failed shamelessly in being recognized by my ESXi Hosts... what a waste of time.
Not wanting to give up, I started to check what partclone was really doing and soon enough I found out that it would restore the "contents" of a partition but not recreate the partition itself. That was a major problem because, apparently, there is no way to create a VMFS partition and a GPT disk at the same time. Linux fdisk supports fb type partitions but not GPT. Parted supports GPT but not fb partitions. So back to square one...
After another hour of research I found an utility called gdisk which is able to read GPT disks but, just as parted, it doesn't recognize VMFS partitions. Fortunately, in the latest version of the program, you can provide your own GUID (partition type Identifier) even if the program doesn't recognize it.
I mounted an existing Datastore and got the GUID from it: AA31E02A-400F-11DB-9590-000C2911D1B8
Now all I had to do was to create a new iSCSI Target with the same size as one of my backed up Datastores, create a GPT record, create a partition with the same size as the original and write the GUID I found. After that it only was a matter of restoring the image to this new LUN and mount it on a ESXi host.... it worked! :smileyhappy:
Now the short version for anybody who cares:
-- BACKUP --
1) Mount the iSCSI Target (which has your Datastore) as read-only (we don't want to corrupt the LUN with two RW initiators) on a Linux Box
2) use partclone.vmfs -O /backup/datastore-backup.img -c -s /dev/sdXX (this is tricky when using in a backup script because the letter can change on each reboot and disk-by-UUID doesn't work properly with VMFS partitions... use LOCATION=$(blkid | grep UUID_SUB_of_datastore_you_want|cut -d: -f1) and then use $LOCATION in a bash script to give the proper path to partclone. It should be quite fast even though it depends on your connection to the NAS
-- RESTORE --
1) Create a new iSCSI Target with a attached LUN on your NAS (same size as the original) or reuse the corrupt one you're trying to restore (I prefer to create a new one)
2) Mount the iSCSI Target as RW on your linux box
3) enter gdisk /dev/whatever_your_new_LUN and create a GPT header
4) create a new partition with the same size or bigger than your original had
5) hit option t and instead of the available code types, put in: AA31E02A-400F-11DB-9590-000C2911D1B8
6) exit gdisk
7) use partclone.restore -O /dev/whatever_your_new_LUN_1 -s /backup/datastore-backup.img
8) go get a coffee although its quite fast (mine goes at 2.88GBytes/min)
9) unmount the iscsi Target on the Linux Box and rescan your storage adapter on your ESXI host. The new iSCSI Target should appear. Attach it.
10) add new storage, choose the new LUN and ESXi will ask if you want to keep your current UUID or generate a new one (if your original Datastore is corrupt or destroyed, you can use the current UUiD).
Thats it! :smileyhappy:
Hope this helps someone out there
Regards
Tony