NSx-T Edge Node Status Degraded

[stgepopp]

Hi VCI's,

just started to build my own (physical) VCF-Environment. Was so far with some learning succesful. During preparing for vRSLCM and other topics with an edge cluster i ran into this:

 

This setup is very similar to the VCF M&O (and others) but physical.

I'm able to ping all Edge TEP IP adresses and to ping Host TEPs as well 

Any idea what's wrong

Thanks and best regards

 

Erich

 

 

[TronAr]

Just shooting something: Do you have edges running on enabled ESXis ? This could be the edge not able to talk to its "holding" ESXi because TEP/vlan clash.

[TronAr]

Nope, that can not be it, you have two TEPs on each Edge on the same subnet. Is this running on one ESXi ?

May be physical MTU  not high enough ? Thus inter NIC traffic is not making it (but intra NIC is ok)

[stgepopp]

Hi TronAr,

both edges are running on two different ESXi Hosts (DRS rule -> will disable this soon) to try.

My lab is almost identical to the VCF M&O lab, but different names and IPs. Delivering this week this class and compared alle settings, but still have no idea.

I guess it's not uplink related but who knows. From my understanding are both edge nodes not able establish all tunnels. 

regards

Erich

 

 

 

[stgepopp]

Hi,

just disabled the DRS rule und bound both edges to the same host:

 

tunnels are up, but why? 

Erich

[TronAr]

Hard to tell w/o the complete topology. You have 2 ESXis with 2 NICs ea ? How are those connected ?

Find the rule about which tunnel works and which does not

[stgepopp]

Hi TronAr,

yes, each host does have 2x 10GE (some VLANs from pSwitch per TRUNK) and as mentioned before MTU set to 9000. So far this worked on other scenarios. 

Maybe i should decrease the MTU on dvSwitch side to see if the pSwitch is the problem

Erich

[TronAr]

Devil is in the details. pswitch may have per vlan MTU ? Go back to the non working setup and verify that ping with large size works in the path that the tunnel does not. If that works, may be the pswitch is eating/filtering BFD ? Port mirroring/analyzer may help

Again, hard to tell w/o detailed topology and switch config, but it seems to me that it IS uplink/pswitch related.

[stgepopp]

Hi TronAr,

checked more than one time connectivity with size 8940 from edge on vdr 0 (tunnel) to other edge and hosts

physical MTU is 9000 and VLAN is also tagged on all switch ports

Erich