MAC address Flapping

[thorqa]

Hi,

A client of ours is getting these errors on a switch connected to 1 of 5 esxi 5 hosts.

Dec 17 00:00:10 GMT+10: %SW_MATM-4-MACFLAP_NOTIF:  Host 02bf.0a95.600c in vlan 96 is flapping between port Gi1/0/26 and  port Gi1/0/3

I have checked host MAC and MAC of vm's that sit on that vlan but none are matching.

Client is persisting that message is coming from that host.

What are some trouble shooting methods I could use to convince the client.

Thanks

[memaad]

HI,

From virtual machine, you will see MAC address starting from only two different series

00:50:56:3F:FF:FF

00:0C:29:0e:51:95

Other than these, mac is not from VM.

Here is KB which explain about mac generation for VM

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=219&sliceId=2&docTypeID=DT_KB_1_1&dialogID=562804726&stateId=1%200%20562826305

Regards

Mohammed

[a_p_]

The first step is to find out to which physical NIC the MAC address belongs. Once you have this information check the vSwitch and Port Group settings to ensure it's not a policy related issue (e.g. "IP-Hash" without "EtherChannel", ...)

André

[thorqa]

Hi Andre,

The MAC address does not match physical NIC on host.

Thanks

[a_p_]

I took another look at the MAC address and it looks like this is a "locally administered" MAC address (http://en.wikipedia.org/wiki/MAC_address). So far I only saw XenServer generating such MAC addresses for it's VMs!?

André

[Josh26]

I agree this doesn't look like VMware.

Regardless of what's generating it, the usual reason here is setting up a network team of some description on the server, and failing to setup the switch to reflect that.

[thorqa]

Could Windows NLB be a possible cause?

Thanks

[MKguy]

That's what I thought seeing this mac too. 02BF is a Windows Unicast-NLB MAC, the other 4 bytes represent the virtual NLB IP address. So that makes 10.149.96.12 (0a95600c). Check which NLB cluster this is on your network.

For Unicast-NLB (seriously, consider to switch to multicast NLB, it can make life so much easier), you have to keep all NLB-cluster members on the same physical host and consider a few other points as explained in:

http://kb.vmware.com/kb/1006580

http://kb.vmware.com/kb/1556

Otherwise you might run into issues like this because of how Unicast-NLB assign the same MAC to all member interfaces.

[thorqa]

Hi MKguy

Will switching to Multicast stop the flap messages from the switch?

Thanks

[Josh26]

thorqa wrote:

Hi MKguy

Will switching to Multicast stop the flap messages from the switch?

Thanks

Multicast NLB will still need to be setup in the switch, with static ARP entries, so that the switch knows the MAC address is supposed to flap.

Implementing multicast and setting up the switch appropriately would be my strong recommendation.

Edit: Although it won't look like flapping and will resolve your error log. But setup those entries anyway, you will need them.

[MKguy]

Source-MAC to switch port learning (the issue at hand here causing the flapping) and ARP resolution are two distinctly different things. The static ARP needs to be set on the gateway router (or switches handling the routing in this subnet) because most routers refuse to accept ARP replies containing MACs with the mutlicast bit set. Static ARPs are an easy thing to configure though.

With Multicast NLB, the nodes won't actually send Ethernet frames with the same virtual MAC as source address anymore, so this should stop the flapping behavior you're experiencing.

[Josh26]

MKguy wrote:

Source-MAC to switch port learning (the issue at hand here causing the flapping) and ARP resolution are two distinctly different things. The static ARP needs to be set on the gateway router (or switches handling the routing in this subnet) because most routers refuse to accept ARP replies containing MACs with the mutlicast bit set. Static ARPs are an easy thing to configure though.

However, you will still, in this configuration, need to configure static MAC address to switch port mappings, in addition to the static ARP on the router. Refer to this Cisco guide here:

http://www.cisco.com/en/US/products/hw/switches/ps708/products_configuration_example09186a0080a07203.shtml

It WILL work without it - however - the switch will behave as a hub and flood all ports. Not what you want.