Upgrading to NSX 4.2.3.3: Any gotchas with the newly added "Known Issue 3626240" for Edge TEPs?

[Serhii]

Hi everyone,

I'm currently reviewing the release notes for an upcoming upgrade to NSX 4.2.3.3 and noticed a strange recent addition to the "Known Issues" section.

According to the document revision history, Broadcom just added the following known issue on March 5th:

"Issue 3626240: Edge tunnels to ESXi hosts are down when sharing the same VLAN for TEP traffic.
Tunnels between the edge node and prepared ESXi host will be down if the edge node's TEP interface uses a vSphere-created distributed virtual port group (DVPG) and shares the same TEP VLAN as the host. It is recommended to avoid using this particular DVPG and TEP VLAN configuration.
For information about tunnel alarms, see KB article 368269."



Link to 4.2.3.3 Release Notes: https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-2/release-notes/vmware-nsx-4233-release-notes.html

My question to the community is: Why was this suddenly converted into a tracked known issue for 4.2.3.3? And in my view, it would be more appropriate to reference KB 312645 rather than KB 368269.

As many of us know, this has been a long-standing architectural limitation in NSX-T dating back several major versions. But seeing it abruptly listed as a "Known Issue" in this specific release makes me wonder if there is more to the story.

• Is this simply Broadcom finally formalizing an age-old limitation with an official issue ID?
• Or is it not that simple - did something actually change under the hood in 4.2.3.3 that makes the old workarounds fail, making it unsafe to upgrade?

Has anyone pulled the trigger and upgraded to 4.2.3.3 yet? Did everything go smoothly, or have you noticed any unexpected TEP tunnel drops?

Thanks in advance for any insights!

-------------------------------------------

[Charlie Silverman]

Per the TID, it's only an issue if the TEPs are on the same network as the host management, something that is antithetical to best practice anyway.   Is that a consideration in your environment?

-------------------------------------------

[Serhii]

Hi Charlie,

Thanks for chiming in! I think there might be a slight mix-up regarding the networks involved in the release note.

The issue Broadcom listed (and the related KB 312645) actually isn't about the TEPs sharing a network with Host Management. It is specifically about the Edge VM TEPs sharing the exact same VLAN as the ESXi Host TEPs on the same vSphere Distributed Switch (vDS). When an Edge VM and its host use standard distributed port groups (DVPGs) on the same VLAN, the vSwitch drops the Geneve traffic due to hairpinning.

To answer your question: yes, this absolutely is a consideration in our environment. Our Edge TEP interfaces currently share the same TEP VLAN as our ESXi hosts' TEP interfaces.

To make this work and avoid the tunnel drops, we currently utilize Workaround 1 from KB 312645. Instead of connecting the Edge TEPs to a standard vSphere distributed port group, we have them connected to a trunked VLAN-backed NSX Segment (configured with a VLAN range 0-4094). This allows the traffic to flow without being stripped or dropped by the vDS.

My main concern with this newly added "Known Issue 3626240" is whether 4.2.3.3 introduces a change that breaks this trunked segment workaround, or if Broadcom is simply formalizing and documenting this age-old vDS behavior for the masses.

If anyone is currently running 4.2.3.3 utilizing this shared TEP VLAN + trunked segment workaround and can confirm your tunnels are still stable, it would be a huge peace of mind!

Best regards, Serhii

-------------------------------------------

[Brandon Stricker]

Hi Serhii,

We had this issue several versions ago and ultimately moved to separate VLANs for the edge TEPs from the host TEPs. I wasn't aware of the workaround you were using but hopefully Support will be able to answer that for you for sure. Sorry I couldn't be of more help!

-------------------------------------------

[Serhii]

Hi Brandon,

Thank you for sharing your experience!

The funniest part is that in the VMWARE NSX ® REFERENCE DESIGN GUIDE Software Version 4.2  we can find the following quotes: 

"Starting with NSX version 3.1, edge and host TEPs can reside on the same VLAN because the
host now can process Geneve traffic internal to the host itself. We must transport edge VM
overlay traffic over an NSX Segment in this case. If the edge TEPs are connected to a vCenter
managed dvpg, tunnels between the host and the edge will not come up"

"In most cases, the single TEP design is preferred for its simplicity".







 

-------------------------------------------

[Serhii]

Hi everyone,

I wanted to provide a final update on this for anyone else who might be planning a 4.2.3.3 upgrade and gets spooked by this release note.

I opened a support ticket to get official clarification from Broadcom VCF Support regarding "Known Issue 3626240" and whether it broke the established workarounds. The verdict: It is a documentation error and the upgrade is safe.

Here is the official Broadcom Support response and resolution:

"I'm working with the documentation team to have the release notes updated, it is yet to be decided which product release will this actually be mentioned in.

In respect of your query, it will definitely be removed from 4.2.3.3 Release Notes. 

I will wait with the update in the communities thread until the change is in place (and I will update it once it's all clarified), but in respect of this case, you're good to go ahead with the upgrade. 

As this exhausts the scope of this case, I will now place the case in a resolved status. The case will auto close in 7 calendar days unless you respond with an update. To ensure clarity on the resolution of your issue, below is a summary of what we worked on:

Product:  VMware NSX

• Question:

• We are upgrading NSX from 4.2.3 to 4.2.3.3. Are we in risk of hitting the "Issue 3626240: Edge tunnels to ESXi hosts are down when sharing the same VLAN for TEP traffic." discussed in NSX 4.2.3.3 Release Notes (https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-2/release-notes/vmware-nsx-4233-release-notes.html)?

• Answer:

• This issue has been present in 4.2.x in general, and if the Edge nodes are currently operational, it's unlikely that you will hit this issue post upgrade to 4.2.3.3.
• I would suggest to review KB article "NSX-T Edge tunnels are down to ESXi hosts when sharing the same VLAN for TEP traffic" - https://knowledge.broadcom.com/external/article?articleNumber=312645 , specifically the Resolution section, to make sure the current design is preventing occurrence of the issue.

                  Essentially, the Edge overlay interface(s) should:

                             Be connected to a VLAN based segment (created in NSX Manager), or

                             Use a VLAN that is other than the VLAN used for host TEP interfaces, or

                             Connect through different standard/distributed switch. 

• 4.2.3.3 Release Notes to be reviewed and mention of "Issue 3626240" is to be removed." 

Upon review, I noticed that "Issue 3626240" has recently been removed from the Release Notes. 

So, mystery solved! It was just a poorly executed documentation update. You are safe to proceed with your upgrades.

Thanks to everyone who chimed in!

-------------------------------------------

[Brandon Stricker]

This is great to know. Thank you for posting a follow up!

-------------------------------------------