Promiscuous mode on an NSX-T Segment

[CyberNils]

How can I enable promiscuous mode on an NSX-T Segment? "Mac Learning" is enabled, but the application which requires promiscuous mode doesn't work. I can set promiscuous mode with the command:

nsxdp-cli vswitch l2sec set

But this has to be done on each ESXi host in the cluster and I would like to avoid having to do that.

[Rutger Blom]

Hi,

Promiscuous mode doesn't exist within NSX-T. Use MAC learning or/and port mirroring instead.

[CyberNils]

Here is a blog about promiscuous mode in NSX-T, so looks like it exists, but it needs to be configured on each host manually:

Nesting vSphere vDS on NSX-T N-VDS – doOdzZZ'sNotes

MAC learning doesn't work in this scenario. Do you have any more details on how we can use port mirroring to replace promiscuous mode? Thanks.

[Rutger Blom]

Port mirroring replaces promiscuous mode in the sense that you can mirror network traffic of segment ports, segments, and virtual machines to a L2 or L3 destination (like a VM or a physical/virtualized network monitoring application).

[CyberNils]

We have two VMs using VRRP (Virtual Router Redundancy Protocol) on NIC2 connected to a dedicated Distributerd Port Group on a vDS. When we move NIC2 from the vDS to a Segment on an N-VDS, the virtual IP keeps flapping back and forth between the VMs. Enabling MAC learning on the Segment didn't resolve this. Promiscuous mode is enabled on the vDS.

[Rutger Blom]

VRRP (often) uses multicast. Are you sure you aren't blocking multicast traffic somewhere like in the DFW?

Could you tell me which VRRP implementation the VMs are using? Is this keepalived or something else?

[CyberNils]

The DFW is not configured yet, only the default Any - Any - Allow rule. I can double check tomorrow though.

The VMs are running Aruba Mobility Master.

Cheers.

[Rutger Blom]

Have you tried to enable "MAC Change" on a MAC Discovery segment profile attached to the segments?

[CyberNils]

Yes, we enabled "MAC Change", "MAC Learning" and "Unknown Unicast Flooding".

[billho]

Would like to know too, having the same problem here

[CyberNils]

Hi,

We have still not been able to use promiscuous mode on NSX-T. Rumors say it will be a new feature in the next NSX-T release coming soon. Please let me know if you figure out how to do it :-)

[Rutger Blom]

Once again, there is no promiscuous mode for NSX-T N-VDS based segments the way there is for VDS based port groups. At least not as of version 2.5.

This is the correct answer even though you might not like it :-)

[vforde]

Hi,

For running Nested environments on NSX-T Backed Segments on VDS7 it is a requirement set.

nsxdp-cli vswitch l2sec set --dvport <dvportgroup id> -dvs Global-NVDS --mac-change --forge-src --promisc

Source

http://notes.doodzzz.net/2019/10/27/nesting-vsphere-vds-on-nsx-t-n-vds/

Thanks

Victor

[vforde]

Hi,

I stand corrected... MAC Learning on the NSX-T Segment at physical layer addresses this now.

https://www.virtuallyghetto.com/2019/11/running-nested-esxi-nsx-v-or-nsx-t-on-top-of-nsx-t.html

Thanks,

Victor

[CyberNils]

Thanks for your input, but this does not address the issue in my original post

 

 

 

[GaelA]

Hello,

I encountered the same issue after a V2T migration.

To resolve the issue on the MAC, I created a specific MAC Discovery Profile with MAC Learning enabled and attached it on Mobility Master Segment.

Then I followed the recommendation of  . I created a new service with a service entry with type = IP and Additional Properties = VRRP

I created a new rule with src/dst = Mobility Master group and the service previously created.

And it works. No more flapping.

Thanks  to have create this topic and  for your tips.

[CyberNils]

Thanks for the update and happy it works for you.

Not sure why it didn't work for my customer since they only had one DFW rule Any-Any-Allow.

It was on NSX-T 2.4 or 2.5, so could be something has changed in later versions.

[GaelA]

I tried the rule Any-Any-Allow but it didn't work. the VRRP rule is mandatory.

You could also try to add Mobility Master VMs in exclusion list.

[hschoenf]

Ran into the same issue last week. Is anyone aware if this has been solved on NSX-T 2.5 yet?

[Rutger Blom]

Hi,

Promiscuous mode like we know it on VDS port groups is not implemented in 2.5 or 3.0.

Keep in mind that this is not an NSX-T issue, but rather a functionality not implemented (yet).

[hschoenf]

Hi,

thanks for your quick reply! I'm not quite sure if Promiscuous mode is even an issue for us...

We too have two Aruba Mobility Master VMs with non-working VRRP as soon as they are migrated onto a N-VDS. Tried every option NSX-T has to offer.

[Rutger Blom]

Yes, I think it's an issue for you. It's bad design by Aruba to require promiscuous mode, but that's not something you can change :-)

So you need to stick to VDS-based port groups for those VMs. With NSX-T <3.0 this means dedicated pNICS. From NSX-T 3.0 you can at least leverage VDS 7.0 and have everything on the same pNICS without having to collapse the vmkernel adapters into NSX-T (N-VDS).

[CyberNils]

I don't think so. Heard some rumors about this being implemented in NSX-T 3.0, but haven't had time to confirm it yet. Can't find it in the release notes though. My customer is still running a vDS occupying two extra NICs in each host just because of this.

[Alexander Ries]

I had that issue already seen in NSX-V. Two VRRP instances didn't worked with implicit allow.

The solution for NSX-V was to add an additional Service (L3_others, Protocol Number 112).

The solution for NSX-T could be to add an additional firewall rule with

Create > Group with both VMs

Create > A new service (IP > Additional Properties VRRP)

Create > Firewall rule under Application /Src Group / Destination Group / Service VRRP / applied to Group / allow