I don't think these two VMs are configured correctly for disk sharing

[JDMils_Interact]

I have 2 VMs which my customer says are sharing disks for clustering however when I looked at the configurations of each VM, I cannot figger out how this can be working. Here's what i can tell you:

• The 2 VMs are BEMS02 & BEMS03. and both run Windows Server 2012.
• BEMS02 is on Host03 which is running VMware ESXi, 7.0.3, 21930508, ESXi 7.0 Update 3n.
• BEMS03 is on Host06 which is running VMware ESXi, 7.0.3, 20328353, ESXi 7.0 Update 3g.
• Host10 & Host07 were patched to  VMware ESXi, 7.0.3, 23794027, VMware ESXi 7.0 Update 3q.

BEMS02 & BEMS03 were powered off and migrated to new hosts:

• BEMS02 was migrated to Host10
• BEMS03 was migrated to Host07

When I tried to power on the VMs, I got this error:

For BEMS02:
Cannot use non-thick disks with clustering enabled (sharedBus='virtual'). The disk for 'scsi1:0' is of the type 'zeroed thick'. Failed to start the virtual machine. Module DevicePowerOn power on failed. Unable to create virtual SCSI device for scsi1:0, '/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03_1.vmdk' Powering on the new Virtual Machine System 3 ms 08/05/2024, 10:14:36 PM 08/05/2024, 10:14:39 PM 2 s 

For BEMS03:
Cannot use non-thick disks with clustering enabled (sharedBus='virtual'). The disk for 'scsi1:0' is of the type 'zeroed thick'. Failed to start the virtual machine. Module DevicePowerOn power on failed. Unable to create virtual SCSI device for scsi1:0, '/vmfs/volumes/########-a3c9e8cb/BEMS03_1/BEMS03_1.vmdk' 

I don't understand what's going on here. I then moved the VMs back to their original hosts and they powered up OK?!?

Here's the disk setup for BEMS02:

Hard disk 1	60GB	Thick Provision Eager Zeroed	No Sharing	[MyStorage_sl_0002] BEMS02/BEMS03_1.vmdk	SCSI(1:0) Hard Disk 1
Hard disk 2	80GB	Thick Provision Eager Zeroed 	No Sharing	[MyStorage_sl_0002] BEMS02/BEMS02.vmdk	SCSI(0:0) Hard Disk 2
Hard disk 3	1GB	Thick Provision Eager Zeroed 	No Sharing	[MyStorage_sl_0002] BEMS02/BEMS03_2.vmdk	SCSI(2:0) Hard Disk 3

Host:
Host03	 VMware ESXi, 7.0.3, 21930508	ESXi 7.0 Update 3n

SCSI Controllers:
SCSI controller 0	LSI Logic SAS	SCSI Bus Sharing = None
SCSI controller 1	LSI Logic SAS	SCSI Bus Sharing = Virtual
SCSI controller 2	LSI Logic SAS	SCSI Bus Sharing = Virtual

Here's the disk setup for BEMS03:

Hard disk 1	80GB	Thick Provision Eager Zeroed	No Sharing	[MyStorage_sl_0002] BEMS03_1/BEMS03.vmdk	SCSI(0:0) Hard Disk 1
Hard disk 2	60GB	Thick Provision Eager Zeroed 	No Sharing	[MyStorage_sl_0002] BEMS03_1/BEMS03_1.vmdk	SCSI(1:0) Hard Disk 2
Hard disk 3	1GB	Thick Provision Eager Zeroed 	No Sharing	[MyStorage_sl_0002] BEMS03_1/BEMS03_2.vmdk	SCSI(2:0) Hard Disk 3

Host:
tfasl1ivmhp06	 VMware ESXi, 7.0.3, 20328353	ESXi 7.0 Update 3g

SCSI Controllers:
SCSI controller 0	LSI Logic SAS	SCSI Bus Sharing = None
SCSI controller 1	VMware Paravirtual	SCSI Bus Sharing = Virtual
SCSI controller 2	VMware Paravirtual	SCSI Bus Sharing = Virtual

Please correct me if I am wrong, but both VMs are using different disks, none shared, BEMS02 is using the wrong controller types and the disks are setup for "No Sharing"? Is this a configuration issue with the VMs or a bug in ESXi 7.0u3q?

Here are excerpts from the logs:

Host10:

hostd.log:
2024-08-05T12:14:39.121Z verbose hostd[2099895] [Originator@6876 sub=Vigor.Vmsvc.vm:/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS02.vmx opID=lym79tev-1629243-auto-yx4s-h5:70373278-10-01-01-01-01-a1-09ce] Power On message: Cannot use non-thick disks with clustering enabled (sharedBus='virtual'). The disk for 'scsi1:0' is of the type 'zeroed thick'.
--> Unable to create virtual SCSI device for 'scsi1:0' (/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03_1.vmdk).
--> Module 'DevicePowerOn' power on failed.
--> Failed to start the virtual machine.
-->

vmware.log
2024-08-05T12:13:46.428Z In(05)+ vmx - Power on failure messages: Cannot use non-thick disks with clustering enabled (sharedBus='virtual'). The disk for 'scsi1:0' is of the type 'zeroed thick'.
2024-08-05T12:13:46.428Z In(05)+ vmx - Unable to create virtual SCSI device for 'scsi1:0' (/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03_1.vmdk).
2024-08-05T12:13:46.428Z In(05)+ vmx - Module 'DevicePowerOn' power on failed.
2024-08-05T12:13:46.428Z In(05)+ vmx - Failed to start the virtual machine.
2024-08-05T12:13:46.428Z In(05)+ vmx -

[Richie55]

I think your boot drive harddisk 1 should be on scsi0:0 not scsi1:0

------------------------------
Richard Osborne
Senior Professional System Architect
DXC
------------------------------

[JDMils_Interact]

For BEMS02 the boot drive is Hard Disk 2 - both VMs have 80GB system drives.

[JDMils_Interact]

**** DELETED ****

[Steven Van Hauwermeiren]

Hi,

Can you check this Article?

https://knowledge.broadcom.com/external/article/311900/cannot-poweron-a-vm-with-the-multiwriter.html

Regards,

Steven

[JDMils_Interact]

Yes I have checked that article and it did not work for me:

[root@MyHost03:/vmfs/volumes/########-a3c9e8cb/BEMS02] vmkfstools -D /vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03-flat.vmdk
Could not get the dump information for '/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03-flat.vmdk' (rv -1)
Could not dump metadata for '/vmfs/volumes/########-a3c9e8cb/BEMS02/BEMS03-flat.vmdk': Inappropriate ioctl for device
Error: Inappropriate ioctl for device

[Steven Van Hauwermeiren]

I suspect a host is locking your vmdk file.

To mee it looks like host03 is locking a file, but the VM is running on Host10(?).

https://www.linkedin.com/pulse/release-locked-vmdk-files-without-pid-charles-masters/

 

[JDMils_Interact]

I doubt it. Both VMs are powered off then vMotioned to newer patch-level hosts then powered on. The power on fails reporting the disk format issue. Locks don't exist on powered off VMs and would not allow the vMotions. The VMs do not reference each others disks so that's not an issue.

I think I may know the reason for this behaviour.It seems the customer is using Windows Clustering in association with SQL Availability Groups. This involves each VM has its OWN disks, and Windows/SQL clustering replicates the data at the OS layer between the servers. I suspect that since the VM's SCSI controllers are set to Virtual Bus Sharing, once moved to the new hosts, this causes an issue where the disks are not really shared. My guess is that by setting all the SCSI controllers Bus Sharing to None should fix this issue.

I need to log a support ticket with Broadcom however my account has expired even though it was recently renewed.

[Steven Van Hauwermeiren]

Hi,

In the past, we had a locking problem and the article gave us the solution. Also VMware supported us perfectly, but it was not Broadcom then :-)

Indeed, SQL Always On uses his own disks, every SQL data is on every server and in Sync.

We als have an Always On SQL Cluster running on VMware.

Not optimal, but this is the config:

1/ VM01: 2 disks

disk1 OS: Thin Provision - No Sharing - Dependent - SCSI(0:0)

disk2 DATA: Thin Provision - No Sharing - Dependent - SCSI(0:1)

SCSI Controller 0: LSI Logic SAS - Sharing: None

2/ VM02: 2 disks

disk1 OS: Thin Provision - No Sharing - Dependent - SCSI(0:0)

disk2 DATA: Thin Provision - No Sharing - Dependent - SCSI(0:1)

SCSI Controller 0: LSI Logic SAS - Sharing: None

We also have Windows Failover Clusters on VMware configured for SQL.

But that is a more complex config, because of the shared disks...

[JDMils_Interact]

I too have had the same issues with hosts locking VM disks and have solved these in the past. But the situations were where a host had failed, locking the disks although the VM's were HA'd to another host, but these were running VMs.

[Steven Van Hauwermeiren]

Setting the VM's SCSI controllers to None instead of Virtual Bus Sharing, should solve your issue.

[JDMils_Interact]

I'm thinking the same. Thanks for spending time to help me out.