VMware {code}

 View Only

Reproducible VMDirectPath I/O failure with populated xHCI USB controllers – IOMMU/DMAR fault during VM power-on

  • 1.  Reproducible VMDirectPath I/O failure with populated xHCI USB controllers – IOMMU/DMAR fault during VM power-on

    Posted 12 days ago

    Hello Broadcom / VMware Engineering,

    I would like to report a reproducible VMDirectPath I/O issue involving PCI passthrough of an entire USB xHCI controller.

    ENVIRONMENT

    Host:
    - Dell PowerEdge T360
    - Intel Xeon E-2488
    - VMware ESXi 8.0 Update 3e

    Guest:
    - Windows Server 2025

    Primary USB controller:
    - Intel Raptor Lake USB 3.2 Gen 2x2 xHCI Host Controller
    - PCI address: 0000:00:14.0
    - Vendor / Device ID: 8086:7a60

    ESXi reports for this device:

    Configured Owner: VM Passthru
    Current Owner: VM Passthru
    Passthru Capable: true
    Reset Method: Power state reset
    FPT Sharable: true

    The passthru.map configuration contains:

    8086  7a60  d3d0  false


    PROBLEM

    PCI passthrough of the complete xHCI controller is reproducibly affected by whether USB devices are attached to the controller before the virtual machine is powered on.

    When affected USB ports are empty, VMDirectPath assignment succeeds and the Windows Server 2025 VM starts normally.

    When a USB peripheral is attached to an affected port before VM power-on, VMDirectPath initialization can fail and the VM does not start.


    FAILURE SIGNATURE

    During a failed VM power-on, vmkernel.log shows the xHCI passthrough device being allocated, followed by an Intel VT-d / IOMMU / DMAR fault involving the same device:

    PCIPassthru: pcipDevInfo allocated for 0000:00:14.0
    ...
    WARNING: VTD IOMMU Fault ... Device 0000:00:14.0 Addr = 0x40000000000
    WARNING: DMAR Fault ... Device 0000:00:14.0 Addr = 0x40000000000
    IOMMU Fault detected. IOaddr 0x40000000000

    ESXi subsequently removes the passthrough binding/domain and VM power-on fails.


    REPRODUCIBLE BEHAVIOR

    Working case:
    1. Boot ESXi.
    2. Keep the affected USB ports empty.
    3. Power on the VM with 0000:00:14.0 assigned by VMDirectPath I/O.
    4. Passthrough succeeds and the VM starts.

    Failing case:
    1. Boot ESXi.
    2. Attach a USB peripheral to an affected USB port before VM power-on.
    3. Power on the same VM with the same passthrough configuration.
    4. The passthrough handoff fails and an IOMMU/DMAR fault is logged.


    IMPORTANT TOPOLOGY OBSERVATION

    The behavior depends on USB topology.

    On this Dell PowerEdge T360, USB devices connected through the three rear SuperSpeed ports located behind the onboard USB hub topology can remain attached during VM power-on and the xHCI passthrough succeeds.

    USB devices attached to certain other/direct USB ports can cause the passthrough power-on failure.

    Placing an external USB hub on an affected direct port does not solve the problem. If a USB device is present behind that external hub during VM power-on, passthrough can still fail.

    ESXi can enumerate and use USB devices connected through the working rear SuperSpeed ports before the controller is passed through. Therefore, the issue does not appear to be simply that ESXi must never enumerate a USB device before passthrough.


    SECOND CONTROLLER OBSERVATION

    A similar behavior was observed with a separate PCIe USB controller:

    - PCIe USB controller with no USB device attached at VM power-on -> passthrough succeeds.
    - Same PCIe USB controller with a USB device attached at VM power-on -> passthrough fails.

    This suggests that the problem may be related to reset, DMA/IOMMU state, or handoff of an active/populated xHCI controller rather than to one specific physical USB connector on the motherboard.


    RESET INFORMATION

    For Intel 8086:7a60, ESXi reports:

    Reset Method: Power state reset

    The passthru.map entry explicitly uses:

    d3d0

    The PCI capability list of this integrated xHCI function does not advertise a standard PCI Express Function Level Reset capability, so the controller is being handed over using a power-state reset rather than FLR.


    EXPECTED BEHAVIOR

    A PCI device reported by ESXi as passthrough-capable should be safely reset and assigned to the VM regardless of whether USB endpoints are already attached, or ESXi should document this limitation if populated xHCI controllers are unsupported during VMDirectPath handoff.


    ACTUAL BEHAVIOR

    With certain USB devices/ports active before VM power-on, the xHCI controller produces an IOMMU/DMAR fault during VMDirectPath initialization and the VM fails to power on.


    CURRENT WORKAROUND

    All permanently connected USB peripherals are kept on the three rear SuperSpeed ports whose onboard hub topology does not trigger the problem.

    Other affected USB ports must remain empty during VM power-on.


    REQUEST

    Please confirm whether this is a known VMDirectPath/xHCI reset or IOMMU issue.

    If known, please advise:
    - whether it is fixed in a later ESXi 8.x build,
    - whether there is an official VMware/Broadcom workaround,
    - and whether there are any recommended reset or passthrough settings for populated xHCI controllers.

    I can provide full vmkernel logs, PCI configuration-space dumps, esxcli hardware pci output, and USB topology information if required.

    Best regards,

    Igor Bornay
    GynVi s.r.o.