Deleting VMDK fails using Datastore Browser for NFS Datastore

[J1mbo]

- NFS datastore provided by Debian 5

- NFS share is a single SATA drive (test environement).

- Export options (rw,no_root_squash,async,no_subtree_check)

- Mount options (for XFS) rw,async,noatime,nodiratime,logbufs=8

- NFS datastore seems fine for everything except deleting a VMDK using the vSphere client Datastore Browser.

Client shows "in progress" for a while, then returns "error caused by file...".

Looking at the directory from the Debian box, the ***-flat.vmdk file is deleted OK but the ***.vmdk file itself is left, occupying a 0.5KB. Refreshing the browser and deleting this file then succeeds. Deleting other small files (like VMX for example) works fine.

Same result with ext3 or XFS as underlying file system.

Many thanks for any suggestions.

[rogard]

chmod 777 ?

Have you check to see if it is a permissions issue?

[J1mbo]

Thanks very much for the suggestion. I applied 777 permissions using chmod -R then re-tested, but still it fails.

Actually it only seems to fail for VMDK's larger than about 8GB. Perhpas there is some timeout issue due to the slow SATA drive? But that said, deleting an 8GB file is almost immediate.

Many thanks

[J1mbo]

Update - it seems to be something off happening with file over about 8GB.

Deleting a 14GB VMDK provides about a minute of disk thrashing and then the error is produced - iostat -xk 2 gives this, disk IO stopping immediately that the error is produced:

avg-cpu: %user %nice %system %iowait %steal %idle

0.00 0.00 5.45 0.00 0.00 94.55

Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util

sda 0.00 0.00 1.50 79.00 6.00 5025.00 124.99 7.12 89.71 12.37 99.60

So it seems there is something odd happening in the Debian box. Deleteing the same file from a Windows NFS share works normally.

[rogard]

Try having a look at this:

Its best practises for NFS:

http://vmware.com/files/pdf/techpaper/VMware-NFS-BestPractices-WP-EN.pdf

Has some good tweaks for NFS datastores

[RParker]

Deleteing the same file from a Windows NFS share works normally.

Same 1 SATA disk scenario and files larger than 8GB?

[J1mbo]

Many thanks for those helpful replies.

Actually the Windows NFS is running from hardware RAID-10 (SATA drives), so I tested a Debian server against that config, which works OK at least up to 16GB (as large as I tested). The delete does take a while though (maybe 10 seconds) with 100% disk utilisation in the process. Perhaps the underlying issue is a 30s timeout, based on the values given in the PDF.

Anyway I resolved the issue on the simple PC-with-SATA-drive server by reformatting with reiser-fs. Not 100% confident about this (it seems there is some undercurrent against reiser) but, the delete happens almost immediately and random IOPS measured from a VM running from it outperform both ext3 and xfs and by some margin.

Many thanks

[J1mbo]

Update - ReiserFS has improved matters, but still it is too slow for the files I'm handling (about 450GB). Deleting a 450GB VMDK takes about 4 minutes, with the client erroring after one minute. The NFS server at least remains responsive as disk load is lower for some reason.

I find conflicting reports - some suggesting that XFS should provide much faster delete and others contradicting that (per my experience) - any other experiences with this?

Can the ESX NFS client be modified to provide a longer timeout for file deletes?

http://blog.peacon.co.uk

Please award points to any useful answer.