How to disable VMDirectPath and remove controllers

[sdeshpande]

Hello,

I  successfully enabled VMDirectPath for USB controller and unfortunately  also for SATA AHCI controller. As SATA AHCI is also enabled, I can not  see anymore my locally attached disk and so the datastores.

1. Even if I try to disble it and try to reboot after that, it comes again "Enabled", and no matter what do.

2.  /etc/vmware/exs.conf,  I changed passthru devices to vmkernel, but  after reboot, it again comes again as passthru. No change at all.

I do not see VMkernel.Boot.noIOMMU option in Software -> Advanced Settings -> VMkernel.

Host: Dell PowerEdge T110 II, ESXi 5.

Please suggest.

Many thanks in advance.

Sameer

[VTsukanov]

Hi

try to edit file /etc/vmware/esx.conf from console

In esx.conf should be a string of the form "/device/000:31.2/owner = "passthru",

1) delete this string

2) after rescan storage adapters from VI client, you should see your storage adapter,

3) after "Rescan All" on tab storages you should see all your storages.

Now you will need store the new configurations to bootbank partitions (exec "/sbin/auto-backup.sh" from console) and copy state.tgz to your old bootbank partitions (should be mounted after successfull step 3 to /vmfs/volumes/....)

[sdeshpande]

Thanks for the reply. After deleting the string (complete line itself with passthru) from /etc/vmware/esx.conf file, devices still appear in passthru.

1.

Original file contents:

/advUserOptions/options[0018]/type = "int"
/device/000:000:26.0/device = "1c2d"
/device/000:000:26.0/owner = "passthru"
/device/000:000:26.0/vendor = "8086"
/device/000:000:29.0/device = "1c26"
/device/000:000:29.0/owner = "passthru"
/device/000:000:29.0/vendor = "8086"
/device/000:000:31.2/device = "1c02"
/device/000:000:31.2/owner = "passthru"
/device/000:000:31.2/vendor = "8086"
/device/000:000:31.2/vmkname = "vmhba0"
/device/000:001:00.0/vmkname = "vmnic0"
/device/000:001:00.1/vmkname = "vmnic1"
/device/000:002:00.0/vmkname = "vmnic2"

Modifed esx.conf

/advUserOptions/options[0018]/type = "int"
/device/000:000:26.0/device = "1c2d"
/device/000:000:26.0/vendor = "8086"
/device/000:000:29.0/device = "1c26"
/device/000:000:29.0/vendor = "8086"
/device/000:000:31.2/device = "1c02"
/device/000:000:31.2/vendor = "8086"
/device/000:000:31.2/vmkname = "vmhba0"
/device/000:001:00.0/vmkname = "vmnic0"
/device/000:001:00.1/vmkname = "vmnic1"
/device/000:002:00.0/vmkname = "vmnic2"

2. Rescan of adapters.

/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ----------------------
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
/etc/vmware #
/etc/vmware #
/etc/vmware # esxcli storage core  adapter rescan --all
/etc/vmware #
/etc/vmware #
/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ----------------------
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
/etc/vmware #

3. Rescan all from the storage tab (ESX client) does not show either :-(

[VTsukanov]

1) try to remove

/device/000:000:31.2/device = "1c02"
/device/000:000:31.2/vendor = "8086"
/device/000:000:31.2/vmkname = "vmhba0"

2) Check "vmkload_mod -l" that you have achi module loaded (something like "vmkload_mod -l | grep achi"), if it not loaded load it by command "vmkload_mod ahci"

3) Try rescan storage adapters

....

[sdeshpande]

1. Removed the lines as mentioned.

2. Loaded the AHCI module which was not loaded before.

/etc/vmware # vmkload_mod -l | grep ahci
/etc/vmware #
/etc/vmware # vmkload_mod ahci
Module ahci loaded successfully
/etc/vmware #

3. Rescan did not change

/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ----------------------
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
/etc/vmware #
/etc/vmware #
/etc/vmware # esxcli storage core  adapter rescan --all
/etc/vmware #
/etc/vmware #
/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ----------------------
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter

4.  Devices still appear as passthru

/etc/vmware # vmkchdev -l

000:000:00.0 8086:0108 1028:04de vmkernel
000:000:26.0 8086:1c2d 1028:04de passthru
000:000:28.0 8086:1c10 0000:0000 vmkernel PCIe RP[000:000:28.0]
000:000:28.4 8086:1c18 0000:0000 vmkernel PCIe RP[000:000:28.4]
000:000:29.0 8086:1c26 1028:04de passthru
000:000:30.0 8086:244e 0000:0000 vmkernel
000:000:31.0 8086:1c52 1028:04de vmkernel
000:000:31.2 8086:1c02 1028:04de passthru vmhba0
000:000:31.3 8086:1c22 1028:04de vmkernel
000:001:00.0 8086:10c9 8086:a03c vmkernel vmnic0
000:001:00.1 8086:10c9 8086:a03c vmkernel vmnic1
000:002:00.0 14e4:165a 1028:04de vmkernel vmnic2
000:003:03.0 102b:0532 1028:04de vmkernel

5. Assigned the devices to vmkernel

/etc/vmware #
/etc/vmware # vmkchdev -v 000:000:26.0
/etc/vmware # vmkchdev -v 000:000:29.0
/etc/vmware # vmkchdev -v 000:000:31.2

6. List again

/etc/vmware #
/etc/vmware # vmkchdev -l
000:000:00.0 8086:0108 1028:04de vmkernel
000:000:26.0 8086:1c2d 1028:04de vmkernel
000:000:28.0 8086:1c10 0000:0000 vmkernel PCIe RP[000:000:28.0]
000:000:28.4 8086:1c18 0000:0000 vmkernel PCIe RP[000:000:28.4]
000:000:29.0 8086:1c26 1028:04de vmkernel
000:000:30.0 8086:244e 0000:0000 vmkernel
000:000:31.0 8086:1c52 1028:04de vmkernel
000:000:31.2 8086:1c02 1028:04de vmkernel vmhba0
000:000:31.3 8086:1c22 1028:04de vmkernel
000:001:00.0 8086:10c9 8086:a03c vmkernel vmnic0
000:001:00.1 8086:10c9 8086:a03c vmkernel vmnic1
000:002:00.0 14e4:165a 1028:04de vmkernel vmnic2
000:003:03.0 102b:0532 1028:04de vmkernel
/etc/vmware #

7. Adaper list without rescan, All adapter are available again.

/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ---------------------------------------------------------------------
vmhba38   ahci       link-n/a    sata.vmhba38   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba0    ahci       link-n/a    sata.vmhba0    (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
vmhba34   ahci       link-n/a    sata.vmhba34   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba35   ahci       link-n/a    sata.vmhba35   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba36   ahci       link-n/a    sata.vmhba36   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba37   ahci       link-n/a    sata.vmhba37   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller

8. Rescan the adapter list

/etc/vmware # esxcli storage core  adapter rescan --all
/etc/vmware #

9. All the adapters are show as in no. 6

/etc/vmware # esxcli storage core  adapter list
HBA Name  Driver     Link State  UID            Description
--------  ---------  ----------  -------------  ---------------------------------------------------------------------
vmhba38   ahci       link-n/a    sata.vmhba38   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba0    ahci       link-n/a    sata.vmhba0    (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba33   iscsi_vmk  online      iscsi.vmhba33  iSCSI Software Adapter
vmhba34   ahci       link-n/a    sata.vmhba34   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba35   ahci       link-n/a    sata.vmhba35   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba36   ahci       link-n/a    sata.vmhba36   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller
vmhba37   ahci       link-n/a    sata.vmhba37   (0:0:31.2) Intel Corporation Cougar Point 6 port SATA AHCI Controller

10. All the adapers and VM are shown.

But as I know, vmkchdev -v is not persistent and it will go back to passthru once rebooted. How shall I make persistent so that it will show me all the adaperts and not in passthru again?

Thanks

[VTsukanov]

You need create new state.tgz (/sbin/autobackup.sh should be create it, take a look at "cat /sbin/auto-backup.sh") and place it to /bootbank & /altbootbank (should be mouted in /vmfs/volume/... you can determine their by files boot.cfg, state.tgz ).

[sdeshpande]

I created a new state.tgz using /sbin/autobackup.sh as mentioned. The state.tgz file was old in /albootbank and copied over it with the new one. Rebooted the host and all went fine. Thanks for the immediate help. All the devices removed from passthru and finally disabled VMDirectPath :-) I could recover all my VMs.

Many thanks

Sameer

[VTsukanov]

No problem, could you please mark the thread as answered and consider to mark correct and/or helpfull answers.

[sdeshpande]

I already marked answered with points :-)

[VTsukanov]

Thanks :smileyhappy:

[kikistan]

I have tried everything you explain.

Success till .... state.tgz

I copied from sbin the auto-backup.sh to /vmfs/volume/XXX

I don't have a clue where is the bootbank and altbootbank

When I reboot ... confi is lost

I need some help.

Thnaks

[VTsukanov]

You can identify bootbank & altbootbank partitions by size (~ 250Mb in output command "df -h").

[Drumboy444]

I too have the Dell T110 II with ESXI (5.5 to be exact but have the same problem with 5.0).  No matter what I do in configuration I have the error that says Host does not support passthrough configuration.  Can you tell me what BIOS version you have on your machine for that might be my problem.  I have virtualization enabled on the BIOS but ESXI still says on the summary tab "The hardware of the host does not support DirectPath I/O.  My CPU CPU is an Intel Core i3-3220 3.3ghz.  What did you need to do special just to pass the USB ports through.  I want to use an external USB drive array and pass it to my virualized file server but for the moment have to keep using one of the internal SATA connections.  I want to use all of the internal SATA connections for raiding my ESXI build and have all other drives be external.

Any help would be appreciated.

Thanks,

Stephen Bailey

[sdeshpande]

If I am not wrong, do you want to connect USB device (external hard drive) to VM? If yes, you do not need to pass USB ports through. You can plugin external USB device to main server, i.e PowerEdge and add USB controller to VM (VM settings and so on) and then external device will appear to VM.

If my understanding is correct, follow the link - http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.vm_admin.doc_50%2FGUID-3E6E58AA-2AD4-4370-9955-07E5FBDE800D.html

HTH

Sameer

[Drumboy444]

No this does not work, according to that link you provided and the links on that page these must be passthrough USB so I believe these ports need to show on Configuration and advanced settings.  I did just try adding all available USB adapters to my VM but it never recognized my drive plugged into the host.