ESXi

 View Only
  • 1.  Infiniband not working..

    Posted Dec 30, 2010 08:01 PM
    vmnic0    0000:01:00.00 nx_nic      Down 0Mbps     Half   00:25:b3:00:a2:98 1500   NetXen HP NC522SFP Dual Port 10GbE Server Adapter
    vmnic1    0000:01:00.01 nx_nic      Down 0Mbps     Half   00:25:b3:00:a2:9c 1500   NetXen HP NC522SFP Dual Port 10GbE Server Adapter
    vmnic2    0000:08:00.00 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:01 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
    vmnic3    0000:08:00.01 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:00 9000   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
    vmnic3.p1 0000:03:00.00 ib_ipoib    Down 0Mbps     Half   00:02:c9:21:11:ad 1500   Mellanox Technologies MT25208 InfiniHost III Ex
    vmnic3.p2 0000:03:00.00 ib_ipoib    Down 0Mbps     Half   00:02:c9:21:11:ae 1500   Mellanox Technologies MT25208 InfiniHost III Ex
    vmnic4    0000:09:00.00 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:03 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
    vmnic6    0000:09:00.01 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:02 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
    vmnic7    0000:0b:00.00 igb         Down 0Mbps     Half   00:30:48:cc:f5:aa 1500   Intel Corporation 82575EB Gigabit Network Connection
    vmnic8    0000:0b:00.01 igb         Down 0Mbps     Half   00:30:48:cc:f5:ab 1500   Intel Corporation 82575EB Gigabit Network Connection


    I've added two Infiniband cards to each of my ESXi 4.1 servers but have been unable to get even a link light to come on although the cards show in ESXi. For the Mellanox cards I had to install some beta drivers I got from the company and that seemed to work without a problem. For the NetXen cards they seemed to be found by ESXi out of the box.


    If I connect my iSCSI device to my switch by the same cables the link lights come on and everything seems to work but if I plug into the ESXi machines there is nothing. The only way I've yet got a link light out of them is by plugging the same cable into the two ports on the same card.


    Is there something else I need to do?

    Also, is there any way to cause the pnics to be listed in a more logical order and to stop moving around when I apply system patches? Something similar to Linux where you can assign a specific device MAC to an interface?



  • 2.  RE: Infiniband not working..

    Posted May 12, 2011 05:36 PM

    FYI for anyone interested. You absolutely must have a subnet manager running on the network before Infiniband will show a physical connection. We decided Infiniband was a pain and switched to plain 10G ethernet which has been working great.