VMware vSphere

 View Only

  • 1.  support rdma with broadcom BCM57414

    Posted Apr 11, 2023 08:57 AM

    Hi all, we purchase new dell servers with BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller Broadcom.

    They are defined as network card with support with RDMA but in the Networking option of vmware hosts i don't see any tab for rdma support. Does anyone have any suggestions on how to fix this? Thanks, regards.



  • 2.  RE: support rdma with broadcom BCM57414

    Posted Apr 11, 2023 12:35 PM

    Hello ,

    If you connect to the ESXi and run:

    esxcli rdma device list

    What is the output?



  • 3.  RE: support rdma with broadcom BCM57414
    Best Answer

    Posted Apr 11, 2023 12:43 PM

    esxcli rdma device list is empty!

    if i run esxcli hardware pci list :

    0000:cb:00.1
    Address: 0000:cb:00.1
    Segment: 0x0000
    Bus: 0xcb
    Slot: 0x00
    Function: 0x1
    VMkernel Name: vmnic5
    Vendor Name: Broadcom
    Device Name: BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller
    Configured Owner: VMkernel
    Current Owner: VMkernel
    Vendor ID: 0x14e4
    Device ID: 0x16d7
    SubVendor ID: 0x14e4
    SubDevice ID: 0x4141
    Device Class: 0x0200
    Device Class Name: Ethernet controller
    Programming Interface: 0x00
    Revision ID: 0x01
    Interrupt Line: 0xff
    IRQ: 255
    Interrupt Vector: 0x00
    PCI Pin: 0x01
    Spawned Bus: 0x00
    Flags: 0x3001
    Module ID: 55
    Module Name: bnxtnet
    Chassis: 0
    Physical Slot: 8
    Slot Description: PCIe Slot 8
    Device Layer Bus Address: s00000008.01
    Passthru Capable: true
    Parent Device: PCI 0:201:4:0
    Dependent Device: PCI 0:203:0:1
    Reset Method: Function reset
    FPT Sharable: true
    NUMA Node: 1
    Extended Device ID: 0
    Extended Device Name:



  • 4.  RE: support rdma with broadcom BCM57414

    Posted Apr 11, 2023 01:11 PM

    It could be possible that the RoCE driver has not been installed, could you please run:

    esxcli software vib list



  • 5.  RE: support rdma with broadcom BCM57414

    Posted Apr 11, 2023 01:25 PM

    Thanks for helping!

    [root@:~] esxcli software vib list | grep -i roce
    bnxtroce 222.0.155.0-1OEM.700.1.0.15843807 BCM VMwareCertified 2023-01-10


    [root@:~] esxcli software vib list | grep -i rdma
    irdman 1.4.1.0-1OEM.700.1.0.15843807 INT VMwareCertified 2023-01-10
    nmlx5-rdma 4.21.71.101-1OEM.702.0.0.17630552 MEL VMwareCertified 2023-01-10
    nmlx4-rdma 3.19.16.8-2vmw.703.0.20.19193900 VMW VMwareCertified 2023-01-10
    nvmerdma 1.0.3.5-1vmw.703.0.20.19193900 VMW VMwareCertified 2023-01-10

     

    The installed version is not the last, as link there:

    https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=51988&vcl=true

    https://customerconnect.vmware.com/en/downloads/details?downloadGroup=DT-ESXI70-BROADCOM-BNXT-NET-ROCE-22211660&productId=974

    in the last link there is a mismatch with the version. Maybe I should try to upgrade to the last...



  • 6.  RE: support rdma with broadcom BCM57414

    Posted Apr 11, 2023 01:35 PM

     

    Give it a try to upgrade to the last version, also make sure to be on the same firmware version.