Unmount datastore: device is busy

[mdrechsler]

Hello,

I'm trying to unmount a datastore, but it's always saying that the device is busy. There are no VMs running on this datastore, not even stored. I think the problem is that this is the only storage available to the ESXi and the ESXi OS itself is installed on an USB stick. The ESXi host seems to write the logging on the datastore which I'm trying to unmount and I have no idea at the moment, how I can disable the logging so I can unmount the datastore.

Can anyone help on this, please. If you need more information just let me know.

[MKguy]

Did you double check there is no ISO or floppy from that datastore being mounted in some VM?

The log location is set with the ScratchConfig.ConfiguredScratchLocation advanced setting. Run this command to check the current scratch location:

# vim-cmd hostsvc/advopt/view ScratchConfig.ConfiguredScratchLocation

If the value is empty, no log scratch has been defined, which is the default for USB drive installs.

Run the following command to list currently open files on VMFS datastores:

# lsof | grep '/vmfs/volumes/'

Note that most files will only be listed with the VMFS volume ID path and not the readable datastore display name (that is just symlink'd). Use this command to see how datastore names and VMFS UUIDs correlate:

# esxcli storage filesystem list

You can then investigate individual file locks further with vmkfstools -D and vmfsfilelockinfo commands.

[mdrechsler]

Logging is obviously enabled, besides that I have no clue why.

~ # vim-cmd hostsvc/advopt/view ScratchConfig.ConfiguredScratchLocation

(vim.option.OptionValue) [

   (vim.option.OptionValue) {

      dynamicType = <unset>,

      key = "ScratchConfig.ConfiguredScratchLocation",

      value = "/vmfs/volumes/56ba86bf-58db69c8-5cca-0cc47a6af506/.locker",

   }

]

Is there any way to temporarily disable the scratch partition? I can't find any information on that. Unfortunately this is the only datastore available to the ESXi host, so I can't change it to some other location.

[MKguy]

Is there any way to temporarily disable the scratch partition?

You should be able to disable it by just resetting the advanced host setting back to the default blank value. See this article and just "reverse" it:

VMware KB: Creating a persistent scratch location for ESXi 4.x/5.x/6.0

You may also need to restart the management agents after the change to make sure the logging to the scratch disk stops. If the device is still busy after that, check file access with lsof as mentioned above.

[mdrechsler]

I'm sorry to say this, but I can't set the value of ScratchConfig.ConfiguredScratchLocation to blank. It accepts a blank value, but it resets it back to the old value. Even after restarting the agents or the whole ESXi host, the old value still exists. There must be something else which has configured this. I knew that the scratch settings are disabled by default when you install ESXi on a USB stick. I have not manually configured this, so I'm still wondering why it actually is configured.

[MKguy]

Ok, a blank value is invalid to reset it to default behavior (even though the value is really empty by default), but the default logging location when no scratch is configured should be /tmp/scratch (local ramdisk only), try to manually set that.

It appears you still actually have to reboot the host for the new value to take effect and be reflected in the ScratchConfig.CurrentScratchLocation parameter though, but if you're lucky a management agent restart could be sufficient:

I just confirmed this myself after trying to change the scratch location:

> Get-VMhost myesxi | Get-AdvancedSetting | Where {$_.Name -like '*scratch*'} | Select Name, Value | FT -AutoSize

Name                                                   Value

----                                                   -----

ScratchConfig.ConfiguredScratchLocation

ScratchConfig.CurrentScratchLocation                  /tmp/scratch

> Get-VMhost myesxi | Get-AdvancedSetting -Name "ScratchConfig.ConfiguredScratchLocation" | Set-AdvancedSetting -Value "/vmfs/volumes/[VMFS-ID]/.locker-myesxi"

> Get-VMhost myesxi | Get-AdvancedSetting | Where {$_.Name -like '*scratch*'} | Select Name, Value

Name                                                           Value

----                                                           -----

ScratchConfig.ConfiguredScratchLocation                      /vmfs/volumes/[VMFS-ID]/.locker-myesxi

ScratchConfig.CurrentScratchLocation                         /tmp/scratch

[mdrechsler]

First of all I have to say I really appreciate your help - thanks alot for this. Unfortunately when I try to change the scratch location to /tmp/scratch it says that the value is invalid.

Just to let you know what I'm trying to accomplish by unmounting the datastore. I want to change the SSD option on our datastore (VMware KB: Enabling the SSD option on SSD based disks/LUNs that are not detected as SSD by default), but on unclaiming the device it says:

Unable to perform unclaim. Error message was : Unable to unclaim all requested paths. Some paths were busy or were the last path to an in use device.  See VMkernel logs for more information.

The command "lsof | grep '/vmfs/volumes/'" is showing no result - so the scratch logging must be the cause.

Do you have any further ideas on this?

[hairychris]

Sorry to necro but I was struggling with this earlier on:

This error is probably because you manually deleted something, see: Manually setting a scratch location for ESXi in the vSphere Client appears to succeed but the setting is not honored (20…

You might also need to recreate the /vmfs/volumes/<your_volume_here>/.locker directory.

If this is still not working then you should make sure the /tmp/scratch folder exists (e.g. SSH in and mkdir /tmp/scratch).

You should then be able to follow the instructions to set the directory to /tmp/scratch or whatever. You may need to reboot between steps.

This still wasn't working fully for me so I just setup scratch to be /tmp rather than /tmp/scratch which finally did the trick for me.

Once all that is done you should be able to unmount the datastore.

If none of that works you can hack round it like this: ESXi 6.0 – Switching from persistent scratch to transient scratch | YAVB - Rich Dowling

[impranayk]

Make sure that datastore doesn't have any VM, iso, or file exists. Also ensure to remove scratch log partition if configured on this.datastore.

[roman79]

Hi there,

I would suggest checking if the datastore was enabled for the local swaping.

In case it is on, you should see a file named sysSwap-ds-<UUID>.swp on that datastore. You can disable this functionality from that Host Client and then reboot your host.