ESXi 5, CNA Qlogic 8262 and Cisco Nexus 5548

[dmesg]

Hello everyone,

I have strange situation.

Small cluster based on 10 Dell R720 servers which have mentioned CNA cards and are connected to Cisco Nexus 5548 switches.

After vMotioning virtual machine from one ESXi server to another, in strange way this VM loses communication to virtual machines which was left on source ESXi host in the same vlan.

We can see that:

1. MAC addresses are travelling along with VM on Nexus switch

2. VM's left on source ESXi host can "hear" arp-requests from migrated VM

3. Migrated VM no longer "hears" arp-replies from VM's located on source ESXi host

4. After changing MAC address on migrated VM everything runs smooth

5. There's no problem at all with storage access, FCoE works like a charm.

Switches don't have any port security enabled. This situation is independent of load-balancing mechanism on ESXi, it's seen on both vSwitches and dvS (Nexus 1000V).

I presume that this can be problem related to drivers or firmware of Dell/Qlogic cards. Did you ever see such kind of behavior?

Best regards,

Peter.

[vmroyale]

Hello and welcome to the communties.

Note: Discussion successfully moved from VMware ESXi 5 to VMware vSphere™ vNetwork

[dmesg]

Hi again,

I'll respond to my question by myself.

There was a problem (maybe not a problem but inconvenience) with qlcnic driver for Qlogic cards. It appears ONLY when we're using partitioned CNAs.

Kernel doesn't update its MAC address table location index (be careful, I don't mean ARP cache!) and it thoutht that MAC of migrated VM still existed on local interface...

Problem has been resolved using kernel module parameter:

esxcli system module parameters set -m qlcnic -p "rx_mac_check=1"

and reboot.

VMware adviced me to upgrade qlcnic driver to new one which was published on 08 June 2012. As I can see, this driver has this option enabled by default :smileyhappy:

Best regards to all,

Piotr.