Using vlans in a project

[Rob Simons]

Hi,

When creating a project there is only a single transport zone available which is the default transport zone for the project. That transport zone needs to be an overlay transport zone. This is understandable.

However I have a customer which I want to add to a separate project space in NSX (a tenant). This customer has VMs mixed with physical equipment on the same vlan's which are extended into NSX through vlan segments in their dedicated NSX environment. We need to migrate this setup to a tenant in our enviroment and want to put this customer in a project space.  Since there are also bare metal servers that are connected to the same vlan/IP segment, I'd like to use a vlan segment.

This is however not possible in a project space. 

In the default  space it is possible to use vlan transport zones and vlan segments but we prefer not to put the customer in the default space.

In both the default space as well as a project space you can give an overlay segment a vlan ID. In the default space you can also set an edge bridge and vlan on an overlay segment. In the project space there is no option to assign a edge bride and vlan to a segment. 

When I assign a vlan ID to a (overlay) segment in a project I see in the topology overview I see that 'Traffic Type' is changed from "Overlay" to "VLAN". There is however no communication possible from and to the vlan.

How should we configure a project overlay segment so it behaves as a regular vlan?  Is a tier-1 gateway necessary when using the overlay as a vlan?  I can imagine that we assign the tier-1 gateway some unused IP in the same segment so as to not be in the way of traffic to the physical gateway.

I know that this is not quite how it's supposed to be used, but it's a lot better as an in-between solution than placing everyting in the default space. 

Thanks,

Rob.

[Steffen Richter]

Hi Rob,

Did you actually check in the Docs what features are supported with the (rather new) Projects feature of NSX?

search for "NSX Projects" -> https://docs.vmware.com/en/VMware-NSX/4.2/administration/GUID-52180BC5-A1AB-4BC2-B1CE-666292505317.html

my eyes directly see a link there called "Features Available for Consumption Under NSX Projects" -> https://docs.vmware.com/en/VMware-NSX/4.2/administration/GUID-65FF7290-B739-4A0B-9C08-F3CDADF4D7A9.html

Overlay Segments	Yes
VLAN Segments	No

L2 Bridge

No

So - no VLANs supported for your workloads in Projects, in no way. Maybe sometime in the future (I have absolutely no clue in that regard), but IMHO VLANs are contradictory to the concept of Multitenancy resp. Projects, besides the general idea of NSX and Overlay Networking.

BR
Steffen

[Rob Simons]

HI Steffen,

I agree and already stated that it makes perfect sense to use overlay network only within the projects, however I also think that one of the reasons for using an on premise cloud solution is to be more flexible as to what is or isn't possible with 'cloud' technologies. In our case we'e severely limited due to an already existing single use (all in default) implementation of NSX which we need to move into our (new) multi tenant NSX environment.  Only vlans sengemts, DFW and  so used on local managers, we therefore need to copy the config throug scripts. Groups have dynamic memberships based on VM tags which will be lost when moving the VM to the new NSX environment. We solved most of the issues now with some scripts but are forced to move this customer into the default space which we don't really want to so.  Converting everything to overlay segments introduces additional complexity and risk so we really don't want to do that right now.  When the entire customer is placed in a project (tenant) and we control all VMs and segments and routers it will be far easier to do a segment conversion for those segments where the tenant need the control of their own segments.

Thanks for the links, and yes I did read loads of documentation but as you already stated it's quite new and I wanted to make sure I didn't miss anything since it does sound as something that should/might be possible in an on premise setup.

Kind Regards,

Rob.

[Rob Simons]

The following picture shows my initial confusion on this topic.  This is a project segment (and therefore an overlay segment).  At the settings page I can add a vlan ID at the additional settings section.  When I do that I see that something has changed in the topology overview (see picture).  It now states 'Traffic Type VLAN' where it first stated Traffic Type Overlay.  This is why my initial thought went to some functionality within the projects that I don't understand.  However, the sharing feature Luca mentioned suits us perfect.

[Steffen Richter]

Hi Rob,

I see your point and confusion. That VLAN on an Overlay Segment is not the best documented resp. used functionality. AFAIK, its for Guest Inter-VLAN Routing, more can be found here https://docs.vmware.com/en/VMware-NSX/4.2/administration/GUID-08930EDF-C0BE-435C-A9EC-CA1303A9AA30.html or here https://vskeeball.com/2022/02/28/nsx-t-802-1q-over-geneve/.

I wasnt aware either of the Sharing Feature within NSX Projects, thanks Luca ;). 

When typing my first answer to you and checking the docs, my thoughts also went to Service Interfaces, but when I saw that nothing in this regard seemed to be supported, that was it for me ;). But with the Sharing feature it makes sense now, though there is a small glitch in that regard in the list of supported features. But as we know, its new, so docs take time :).

Beware with Service Interfaces though - if I am not totally confused this morning, they do not support egress routing, meaning your customers physical equipment needs to use the Service Interface as its gateway. 

One of the TPMs created a nice doc for Service Inerfaces (which are only supported on A/S gateways): https://community.broadcom.com/viewdocument/nsx-t-service-interfaces?CommunityKey=b76535ef-c5a2-474d-8270-3e83685f020e&tab=librarydocuments

Happy reading & implementing!

BR
Steffen

[Luca Camarda]

Hi Rob,

the way we designed the NSX muti-tenancy feature assumed that project admins do not have control over the physical infrastructure, which is why the lifecycle of VLAN networks is not part of the capabilities allowed within a project or a VPC. That said your use case is valid, a tenant ( mapped to an NSX project ) may have virtual and physical workloads. The way we thought to cover this use case is via the sharing feature:
https://docs.vmware.com/en/VMware-NSX/4.1/administration/GUID-F17B20A0-12B8-483A-AE54-75AC580F57F1.html#GUID-F17B20A0-12B8-483A-AE54-75AC580F57F1

The enterprise admin can create VLAN segments and share them with a project, the project admin can then interconnect that VLAN to a Tier-1 gateway service interface so that the T1  is the gateway for the server. This way the project admin has a way to control the routing and gateway firewall for the physical server without the risk impacting the physical fabric that should remain under the control of the provider.

[Steffen Richter]

Hi Luca,

Does the documentation then need an update? If Service Interfaces, connecting into VLANs, are supported with NSX Projects, the Feature Matrix should list it as such, doesn't it?

(besides architectural questions why and if such a thing makes sense, depends on circumstances I guess)

Thanks, BR

Steffen 

[Rob Simons]

Hi Luca, 

This sounds exactly like what I'm looking for I'll check out the full solution tomorrow!

Thanks,

Rob.

[Rob Simons]

Hi Luca, 

I've read the document and played with it a bit but in the document it states that:

"When a segment is shared with a project, it does not mean that the VMs, ports, and gateway interfaces on this segment are exposed to the project. In fact, the project does not have visibility into the segment ports, interfaces, and workload VMs of the shared segment. Therefore, project users cannot configure distributed firewall policies on the workload VMs that are connected to the shared segment."

I can view the shared segment in the segments overview from within the project and it's associated port groups.  I cannot add a VM or portgroup to a security group which is correct according to the document.

The vlan which is shared with the project is reachable through a service interface, which means that it's behind a L3 hop from a network perspective.  This means I cannot share VMs and bare metal servers on the same segment which was my original intent. Since it's (from project perspective) behind a L3 hop it makes sense that DFW cannot be applied either. However I do need a mix of DFW and bare metal in the same segment since that's what the source environment uses.

Back to the drawing board it seems. 

Regards,
Rob.

[Luca Camarda]

Hi Rob,

based on your requirements you need bridging between the VLAN and the Project Overlay segments. You have 2 main options:

1) Extend VLAN to project overlay via a L2VPN (supported starting in NSX 4.2.0)

2) Do bridging via the management plane API. This is not what we would like to see from a product perspective (The idea is that management API are deprecated and will be removed at some point), but if I put my Architect hat on, this is probably what I would do. An example is here:

https://blogs.vmware.com/networkvirtualization/2023/09/nsx-v2t-layer-2-bridging-with-nsx-t-projects.html/

I will be happy to discuss your specific customer project needs and challenges on a call. If interested, ping me on my email: [e-mail].