vSphere Hypervisor

 View Only

X540-T2 & VT-d DirectPath Physical Port assignment

  • 1.  X540-T2 & VT-d DirectPath Physical Port assignment

    Posted Oct 15, 2013 04:33 PM

    Ok, my first time posting in the VMWare Community.  I've yet to see any reference to the issue I am experiencing with my two Intel X540-T2 adapters.

    Some background. I have two test environments:

    First is:

    • Asus Z9PA-U8 w/3930K & 64GB RAM
    • ESXi v5.1u1 & v5.5
    • X540-T2 w/in-box driver, followed by 3.18.7 driver
    • LSI 9260-8i w/Intel SAS Expander and 12 disks RAID10

    Second is (about to be trashed):

    • SuperMicro X8SAX w/i7-950 & 24GB RAM
    • ESXi v5.1u1
    • X540-T2 w/in-box driver
    • LSI 9260-8i w/6 disks RAID10

    The good:

    • The adapters work as host NICs connected to vSwitch, just fine, but performance is currently abysmal - even with more buffers & jumbo frames.
    • I have temporarily walked through creating VFs for the NIC and have been able to assign and use each VF without issue - again, ignoring the abysmal performance.

    The Goal (why I am here):

         I want to assign a single port from the X540-T2 to my main Windows 2008R2 guest so I can reduce my performance tuning steps to one OS, and because my connection via that particular port is x-over from my workstation, so I don't WANT that port running through a vSwitch anyway. 

    Here's what I did (before messing with VFs, so max_vfs was 0 on all for this):

    1. Set one port to be available with DirectPath, rebooted the host
    2. Added that PCI Device to the guest (Win2008R2)
    3. Booted the guest and it identified the device as a NIC in need of drivers
    4. Ran the Intel ProWinX64 package and the guest hard crashed, i.e. powered off
    5. Booted the guest again and the device shows as an X540-T2 CNA, but states that it couldn't be started because Windows couldn't load the driver (the infamously useless Error Code 31)
    6. Tried moving the NIC to other >=8-lane-electrical slots, followed by a redo of steps 1 & 2, w/hardcrash & step 5 always resulting

    The same exact symptom appears in both test boxes, while other NICs and PCI-E devices get assigned without issue.

    Any help figuring this out would be enormously appreciated.