NSX MTU over WAN and do not fragment

[scotta1white]

Hi,

I'm setting up an Cross Vcenter NSX implementation, and have determined the WAN connection is allowing 9000 MTU packet size, but it appears to be  fragementing the packets.

# vmkping  -I vmk3  -s 8972  <remote_ip_address>  works, but

# vmkping -I vmk3 -s 8972 -d <remote_ip_address> does not work

The difference in the ping times for the packets averages is about 1.5 milliseconds more when the size is greater than 1500 bytes.

The local area  network is fine.  I can ping locally with the "-d" (do not fragment) option with size 8972 so apparently the LAN is not fragmenting packets.

My question is will the WAN connection for Cross vCenter NSX work if it's fragmenting the packets? I guess the question boils down to does NSX add the "do not fragment" bit to the packets?

Regards,

SW

[Luke Hoffer]

NSX VXLAN traffic will have the df bit set so will not work if it sends a frame larger than the MTU.  In your case it sounds like there's either a device somewhere in the path with an interface that isn't enabled for a 9000 byte MTU or traffic is getting some sort of additional encapsulation across the WAN that's causing it to exceed 9000 bytes.  That being the case, you might want to start by adjusting your ping sizes to see if something slightly smaller will work (pointing to additional encapsulation overhead being the issue, in which case you'd just need to set the vDS MTU to slightly smaller than 9000 so that it keeps the VXLAN frames small enough to get through without fragmentation), otherwise, you'll need to start checking interfaces in the path to see where the MTU is smaller and/or if it's your WAN provider.

[tanurkov]

HI

I suggest to use a packet capturing on ESXi host to see the packet leaving the host with -d option inside.

Regards Dmitri