Snapshot removal takes extremely long

[virtualdutch]

When I create a backup with Veeam Backup & Replication it takes extremely long time to perform the standard VMware remove snapshot task, it stucks several hours at 95%. When i use the snapmon script i see delta files round 5GB.

I've read a article about NFS in combination with CBT http://communities.vmware.com/message/1812898#1812898 , but that's solved with a ESXi patch http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031106.

How is it possible that it take so much time to remove the delta files?

[titaniumlegs]

We've found that a very busy VM with lots of writes can outrun the snapshot delete process.  Is your VM/app doing lots of writes?  Can you quiesce it?

Do you get the same results doing a simple VMware snapshot without a Veeam backup task?

[virtualdutch]

The server itself is a busy SAP database server, but when i look at IOPS utilyzation it goes about 2000 ~ 3000 IOPS for the complete enviroment on a HP Lefthand P4500G2 with 15k SAS storage.

If i do a manual create snapshot + removal it takes about the same time. In Veeam are all tasks completed with the backup in 10 minutes, only the removal takes enormous time.

[nalfonso14]

the extend has a slight loss of performance .. must consider the RAID disk and performance...

important is the amount of VM's in a datastore and its role (DB, AD, backup server)

[virtualdutch]

This server has four VMFS datastores dedicated, the RAID of the storage LUN is Network RAID-10 (2 Way Mirror).

[nalfonso14]

How many VM are in each datastore ?

[virtualdutch]

In this Lefthand cluster every server has dedicated VMFS datastores in total there are four servers (but the one with snapshot removal problems is the only active).

[nalfonso14]

that server is in a datastore that has free space? and how are the snapshots?, that is very important as I said before

[virtualdutch]

The server has the following VMFS datastores;

VMFS01

Size: 250GB

Free: 95GB

Server config

Virtual Disk 40GB

Virtual Disk 20GB

Virtual Disk 60GB

Snapshot delta files (when snapshotting)

VMFS02

Size: 450GB  

Free: 49GB

Virtual Disk 400GB

VMFS03

Size: 1,6TB

Free: 101GB

Virtual Disk 1,5TB

VMFS04

Size: 1,99TB

Free: 91GB

Virtual Disk 1,9TB

[nalfonso14]

Hi!, whats your esx version? and build?.. delay is normal, according to the size of the VM and the number of snapshots .. One very important thing to consider is storage space available if it contains more than a snapshot. good practice is to have 20% free space in the datastore.

  Any question tell me

King Regard

[virtualdutch]

When i want to use the 20% rule for every virtual i've to create an extended VMFS datastore because the virtual disk is now 1,9TB.

Are there performance issues with extended datastores?

In our enviroment we're running ESXi 4.1 with the latest patches.