VxLAN to VLAN mapping

[SunshineM]

I am a newbie to VMware NSX. Started learning and went through many videos of NSX VxLAN, but still not able to understand the mapping of VLAN with VxLAN.

VLAN > 12 bits > 4096 addresses

VxLAN > 24 bits > 16 M addresses

Taking an E.g.:

(1) 2 Clusters

> Cluster 1 - Edge & Management: Running NSX Manager, NSX Controllers,vCenter Server(vCSA), Distributed Router VM, NSX Edge VM

   ESX_1

   ESX_2

> Cluster 2 - Compute_1: Running VM_App, VM_Web, VM_DB

   ESX_3

   ESX_4

(2) 4 Logical Switches

> LS_App having VNI 5001

   Running VM_App

> LS_Web having VNI 5002

   Running VM_Web

> LS_DB having VNI 5003

   Running VM_DB

> LS_Transit having VNI 5004

> This will create 4 dvPortGroups on dvSwitch

(3) For 3 VMs I want to give IP as below so that I can access it from Outside (Not Internet, but within Company domain, by RDP)

  VM_App <> assign IP as 10.10.10.1 (from VLAN10 which is configured on Physical switch to which uplink is connected)

  VM_Web <> assign IP as 10.10.20.1 (from VLAN20 which is configured on Physical switch to which uplink is connected)

  VM_DB  <> assign IP as 10.10.30.1 (from VLAN30 which is configured on Physical switch to which uplink is connected)

Obviously communication between these VMs will be through DLR

Question 1:

When we prepare host for VxLAN by selecting "configure" option, do we need to enter any VLAN ID, say suppose we want to use above IPs or leave it blank ?

Question 2: (With relation to above question 1)

If we were to use Private IP address, then its ok, as then VMs will be accessed from console.

But I am not getting the point that how VLAN/VXLAN will be configured if we were to use IPs from those above listed VLANs (i.e. any IPs that can be RDP, or ssh etc). And if VLAN <> VXLAN is 1:1 then what's the importance of VXLAN with 16 M addresses.

Question 3:

Here if VNI to VLAN mapping is 1:1, i.e. 5001 <> VLAN10, 5002 <> VLAN20 and so on, then still the maximum VNI that can be used is 4096 in total only which is equal to total VLAN ID 4096. I did not get this point

Thanks,

[Sree]

Question 1:

When we prepare host for VxLAN by selecting "configure" option, do we need to enter any VLAN ID, say suppose we want to use above IPs or leave it blank ?

When we prepare the compute nodes , we need to enter a VLAN-ID which will be VXLAN Transport VLAN. It doesn't mean we can't send a untagged VXLAN packet, if needed you can mention VLAN=0 as well.

Question 2: (With relation to above question 1)

If we were to use Private IP address, then its ok, as then VMs will be accessed from console.

But I am not getting the point that how VLAN/VXLAN will be configured if we were to use IPs from those above listed VLANs (i.e. any IPs that can be RDP, or ssh etc). And if VLAN <> VXLAN is 1:1 then what's the importance of VXLAN with 16 M addresses.

I'm not fully sure about your ask on IP's . To clarify a bit based on my understanding, VTEP-Pool is a unique private ip Pool which we need to consider if there is a VXLAN use case, ideally tagged with a VLAN ID as well. We are not mapping 1 VXLAN network(VNI) to unique VLAN , rather all VXLAN networks are mapped to one VLAN . 802.1q being 12 bit  for a single L2 domain we are always limited with 4096 ,however VXLAN being 24 bit which is exactly the double , we can have 16 million address .  You can check

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/NSX%20for%20vSphere%20Recommended%20Configuration%20Maximums_63.pd…  to understand config maximum numbers or in NSX U.I under system scale all supported values are mentioned .

Question 3:

Here if VNI to VLAN mapping is 1:1, i.e. 5001 <> VLAN10, 5002 <> VLAN20 and so on, then still the maximum VNI that can be used is 4096 in total only which is equal to total VLAN ID 4096. I did not get this point

I hope above explanation answers this question as well

[SunshineM]

I understood for the Question 1 > That VLAN ID in VxLAN configuration property should be the VLAN used (if using any) for VTEP IPs.

Now if VLAN used for VTEP IPs is say VLAN 40

So in Cluster 2 > hosts will have VETPs as;

ESX_3:

> ESX_3_VTEP_1_IP: 10.10.40.1

> ESX_3_VTEP_2_IP: 10.10.40.2

ESX_4:

> ESX_4_VTEP_1_IP: 10.10.40.3

> ESX_4_VTEP_2_IP: 10.10.40.4

Logical Switches have been created like;

> LS_App having VNI 5001

> LS_Web having VNI 5002

> LS_DB having VNI 5003

> LS_Transit having VNI 5004

> This will create 4 dvPortGroups on dvSwitch

I have created 3 VMs, each on different VLAN with below IPs;

  VM1_App <> assigned IP as 10.10.10.1 (this is VLAN10 on physical switch)

  VM2_Web <> assigned IP as 10.10.20.1 (this is VLAN20 on phyiscal switch)

  VM3_DB  <> assigned IP as 10.10.30.1 (this is VLAN30 on physical switch)

Associated VMs with Logical switch;

LS_App <> VNI 5001 <> VM1_App

LS_Web <> VNI 5002 <> VM2_Web

LS_DB <> VNI 5003 <> VM3_DB

Q1) Is this configuration correct, so that I can access this 3 VMs via RDP ? If not, then please explain how can this be acheived ?

Q2) Here VLAN10 <> VNI 5001, VLAN20 <> VNI 5002, VLAN30 <>5003 >> This is 1:1 mapping between VLAN <> VxLAN. Then how come 16M VxLAN becomes possible when VLAN has 4096 limitation only.

[Sree]

I have created 3 VMs, each on different VLAN with below IPs;

  VM1_App <> assigned IP as 10.10.10.1 (this is VLAN10 on physical switch)

  VM2_Web <> assigned IP as 10.10.20.1 (this is VLAN20 on phyiscal switch)

  VM3_DB  <> assigned IP as 10.10.30.1 (this is VLAN30 on physical switch)

I can understand were you are confused . You don't need to define VLAN anymore for compute workloads .That defeats the purpose of VXLAN ,just scope the subnets for workload VM's , ESG&DLR. Below mentioned are the only VLAN that is ideally required.

• ESXi Management 
• VXLAN Transport
• vMotion
• IP Storage
• Transit Network for Tenant Mapping

LS_App <> VNI 5001 <> VM1_App

LS_Web <> VNI 5002 <> VM2_Web

LS_DB <> VNI 5003 <> VM3_DB

Q1) Is this configuration correct, so that I can access this 3 VMs via RDP ? If not, then please explain how can this be acheived ?

Yes, RDP will work as long required ports are opened and NAT/Routing is configured based on the design.

Q2) Here VLAN10 <> VNI 5001, VLAN20 <> VNI 5002, VLAN30 <>5003 >> This is 1:1 mapping between VLAN <> VxLAN. Then how come 16M VxLAN becomes possible when VLAN has 4096 limitation only.

Correction, you can send all VXLAN network (Tunneled traffic) via single VLAN.

For Eg:  VNI 5001,  VNI 5002, VNI 5003  going via VXLAN Transport VLAN 10

[SunshineM]

",just scope the subnets for workload VM's , " >> So here you mean, that we just need to decide the scope of subnets that we will be using for workload VMs, ESG.

For e.g. decided to use;

> 192.168.0.1 - 192.168.0.50 for workload VMs > and assign these IPs to any workload VMs created

> 192.168.1.1 - 192.168.1.5 for ESG

Is this what you mean ?

[Sree]

Yes, that's all you need .  For eg : Have a look at below topology

3 Tiers of Logical Route instances are Peered with Transit Edge via OSPF/BGP . Respective logical switches connected to DLR will be mapped to VNI which we have defined in VNI scope. All encapsulated traffic for those VNI's will flow via common transit VTEP VLAN . DLR's and Edges are also directly connected via Transit Logical Switch. However upstream routes from Edge to External network will be via Single/Multiple VLAN's .

[bayupw]

Question 1:

When we prepare host for VxLAN by selecting "configure" option, do we need to enter any VLAN ID, say suppose we want to use above IPs or leave it blank ?

This is a dedicated VLAN ID to be used for VTEP.

Here's a diagram that may be helpful to explain this

Each ESXi host will have one or more VMkernel IP address for VTEP (one IP in single VTEP configuration or more than one in multi-VTEP config)

This requires a dedicated VLAN, just like any other VMkernel in ESXi i.e. vMotion, Management

This VLAN configuration is per cluster so you can have different VLANs for different clusters as long they are routable with each other.

The VXLAN VNI will be encapsulated and use that configured VLAN, if you create logical switches later you would be able to see that all of the PortGroup will share a common VLAN

Question 2: (With relation to above question 1)

If we were to use Private IP address, then its ok, as then VMs will be accessed from console.

But I am not getting the point that how VLAN/VXLAN will be configured if we were to use IPs from those above listed VLANs (i.e. any IPs that can be RDP, or ssh etc). And if VLAN <> VXLAN is 1:1 then what's the importance of VXLAN with 16 M addresses.

VLAN is not 1:1 to VXLAN, you can have 1 VLAN for the VTEP and create thousands of VXLAN logical switches that isolated from each other but sharing the same one common VLAN

Question 3:

Here if VNI to VLAN mapping is 1:1, i.e. 5001 <> VLAN10, 5002 <> VLAN20 and so on, then still the maximum VNI that can be used is 4096 in total only which is equal to total VLAN ID 4096. I did not get this point

vCenter can create up to 10,000 dvPortGroup so you can create up to 10,000 logical switches per vCenter but only requires one VLAN for the VTEP