VLAN Trunking

[iamkinghenry]

I can't get VLAN trunking to work and I'm not sure if I have VMware setup correctly.

~ # esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks  
vSwitch0         128         5           128               1500    vmnic1   

  PortGroup Name        VLAN ID  Used Ports  Uplinks  
  VM Network            0        1           vmnic1   
  Management Network    0        1           vmnic1   

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks  
vSwitch1         128         7           128               1500    vmnic2   

  PortGroup Name        VLAN ID  Used Ports  Uplinks  
  VM                    0        1           vmnic2   
  VLAN 20 - NIC         20       1           vmnic2   
  VLAN 10 - NIC         10       1           vmnic2   
  VM - NIC              0        1           vmnic2

~ # esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description                  
vmnic1  0000:03:00.00 r8168       Up   100Mbps   Full   80:ee:73:73:51:7e 1500   Realtek Realtek 8168 Gigabit Ethernet
vmnic2  0000:05:00.00 r8168       Down 0Mbps     Full   80:ee:73:73:51:7f 1500   Realtek Realtek 8168 Gigabit Ethernet

~ # esxcfg-vmknic -l
Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type               
vmk0       Management Network  IPv4      192.168.1.117                           255.255.255.0   192.168.1.255   80:ee:73:73:51:7f 1500    65535     true    STATIC             
vmk0       Management Network  IPv6      fe80::82ee:73ff:fe73:517f               64                              80:ee:73:73:51:7f 1500    65535     true    STATIC, PREFERRED  
vmk1       VM - NIC            IPv4      192.168.6.1                             255.255.255.0   192.168.6.255   00:50:56:60:ca:6a 1500    65535     true    STATIC             
vmk1       VM - NIC            IPv6      fe80::250:56ff:fe60:ca6a                64                              00:50:56:60:ca:6a 1500    65535     true    STATIC, PREFERRED  
vmk2       VLAN 10 - NIC       IPv4      192.168.110.1                           255.255.255.0   192.168.110.255 00:50:56:6d:d4:b6 1500    65535     true    STATIC             
vmk2       VLAN 10 - NIC       IPv6      fe80::250:56ff:fe6d:d4b6                64                              00:50:56:6d:d4:b6 1500    65535     true    STATIC, PREFERRED  
vmk3       VLAN 20 - NIC       IPv4      192.168.120.1                           255.255.255.0   192.168.120.255 00:50:56:67:40:fc 1500    65535     true    STATIC             
vmk3       VLAN 20 - NIC       IPv6      fe80::250:56ff:fe67:40fc                64                              00:50:56:67:40:fc 1500    65535     true    STATIC, PREFERRED  

I think this is setup correctly.  The non-VLAN is working fine but the VLANs won't give IPs.

I did create a thread in the Debian forum because I think it's something wrong with how my DHCP server is setup.

http://forums.debian.net/viewtopic.php?f=5&t=109503

[tomtom901]

Could you show a screenshot of configuration -> networking in the GUI? Might give a quicker overview of the configuration.

[a_p_]

How does the configuration of the physical switch ports look like? With the settings you posted the ports have to be configured for 802.1Q (trunk/tagged).

André

[iamkinghenry]

Here is a picture.

[iamkinghenry]

Do I need create a Virtual Machine and VMkernel for each VLAN?

I did add Ethernet Adapters for each VLAN for the server that is the router, is this necessary?

When I look at the "vMotion and IP Storage Port" under "NIC Teaming" the active adapter only shows 192.168.6.100-103 but I think it should also show the Virtual Machines I created for the VLANs, should the VLAN IPs be shown?

[vJJSosa]

Hi Iamkinghenry, in the esxcfg-nics the vmic2 appear as down... this is the actual status for this nic? Because this is a link error, port error, or nic error, but we can't talk about trunk error yet

Please verify this status

Regards

[tomtom901]

Hi Henry,

No, you do not need to configure a VMKernel port for each VLAN. In VMware networking, you can configure 2 types of port groups:

1) VM Port group, which will switch VM traffic

2) VMkernel port which can be given an IP address on which management traffic, fault tolerance logging or vMotion can be enabled. It can also be used to connect to IP storage, either iSCSI or NFS.

In your example, you did configure the VLAN part correctly, but you chose VMkernel ports instead of Virtual Machine traffic. So, if you configure the same thing but with Virtual Machine traffic port groups, assuming the physical switch configuration is correct, you should be good to go. If the Debian VM functions as a router, it should have a vNIC (virtual nic) and an IP address for each of the VLANs.

Your config should look like this then:

Hope this helps,

[ucusan]

Except for the native vlan, you cannot have both tagged and untagged traffic on the same physical port (the port is either in access mode or in trunk mode). If you say your untagged traffic is OK on these ports, it means they are configured in access mode, so your tagged traffic will not work as well.

You don't have to create vmks for each vlan you want to use. VMKernel ports are used for specific host-to-host, host-to-storage, or host-to-vCenter purposes.

[de2rfg]

ucusan wrote:

Except for the native vlan, you cannot have both tagged and untagged traffic on the same physical port (the port is either in access mode or in trunk mode). If you say your untagged traffic is OK on these ports, it means they are configured in access mode, so your tagged traffic will not work as well.

I've tried for a couple of days now to configure vSwitch portgroups with a vlan id that is configured as native and tagged vlan on the physical nexus switch. I need the native vlan for pxeboot but after that I would like to use the same vlan id (vmware mgmt) as tagged vlan, so that all vmk and portgroups are configured in the same way. I can setup and use a vmk or portgroup without vlan id (as native vlan). But as soon as the vlan is configured as native and tagged vlan and I add a portgroup with this vlan id I see no packets on this portgroup.  As soon as the native vlan id is removed from the trunk I see packages coming in. Support told me that this is a limit of 802q and a  vlan id can only be used as native or tagged one.

[iamkinghenry]

vmnic2 is up (not sure why it was down when I posted that before)

~ # esxcfg-nics -l

Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description                

vmnic1  0000:03:00.00 r8168       Up   100Mbps   Full   80:ee:73:73:51:7e 1500   Realtek Realtek 8168 Gigabit Ethernet

vmnic2  0000:05:00.00 r8168       Up   1000Mbps  Full   80:ee:73:73:51:7f 1500   Realtek Realtek 8168 Gigabit Ethernet

I configured vmnic2 with VM Port Groups for the VLANs.

Thanks for explaining the use of VMKernal port and VM Port group.

ucusan wrote:

Except for the native vlan, you cannot have both tagged and untagged traffic on the same physical port (the port is either in access mode or in trunk mode). If you say your untagged traffic is OK on these ports, it means they are configured in access mode, so your tagged traffic will not work as well.

This worries me a bit.  I do have traffic that is not tagged that I need to served as well as VLAN traffic.  The router (EdgeMax) I'm currently will allow for both types of traffic.  How do I set the port to be in trunk mode?

Edit:

For a quick test I removed the VM Port Group for non-vlan traffic and still the VLANs don't get IPs.  The non-vlan of course does not work when I do this.

I think my Debian settings are right:

ifconfig | less

eth1	Link encap:Ethernet  HWaddr 00:0c:29:2e:49:2d
	inet addr:192.168.6.1  Bcast:192.168.6.255  Mask:255.255.255.0
	inet6 addr: fe80::20c:29ff:fe2e:492d/64 Scope:Link
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:45459 errors:0 dropped:0 overruns:0 frame:0
	TX packets:33608 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:1000
	RX bytes:3723766 (3.5 MiB)  TX bytes:3490837 (3.3 MiB)

eth1.10   Link encap:Ethernet  HWaddr 00:0c:29:2e:49:2d 

	inet addr:192.168.110.1  Bcast:192.168.110.255  Mask:255.255.255.0
	inet6 addr: fe80::20c:29ff:fe2e:492d/64 Scope:Link
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:0 errors:0 dropped:0 overruns:0 frame:0
	TX packets:35 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:0
	RX bytes:0 (0.0 B)  TX bytes:5727 (5.5 KiB)

eth1.20   Link encap:Ethernet  HWaddr 00:0c:29:2e:49:2d 

	inet addr:192.168.120.1  Bcast:192.168.120.255  Mask:255.255.255.0
	inet6 addr: fe80::20c:29ff:fe2e:492d/64 Scope:Link
	UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
	RX packets:0 errors:0 dropped:0 overruns:0 frame:0
	TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
	collisions:0 txqueuelen:0
	RX bytes:0 (0.0 B)  TX bytes:5418 (5.2 KiB)

route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0

169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth1

192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

192.168.6.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1

192.168.110.0   0.0.0.0         255.255.255.0   U     0      0        0 eth1.10

192.168.120.0   0.0.0.0         255.255.255.0   U     0      0        0 eth1.20

root@debian:~#

ps ax | grep dhcpd

2882 ?        Ss     0:00 /usr/sbin/dhcpd -q -cf /etc/dhcp/dhcpd.conf -pf /var/run/dhcpd.pid eth1 eth1.10 eth1.20

3121 pts/0    S+     0:00 grep dhcpd

/etc/dhcp/dhcpd.conf

option domain-name "mydebian";

# Use Google public DNS server (or use faster values that your internet provider gave you!):

option domain-name-servers 8.8.8.8, 8.8.4.4;

# Set up our desired subnet:

subnet 192.168.6.0 netmask 255.255.255.0 {

    range 192.168.6.101 192.168.6.254;

    option subnet-mask 255.255.255.0;

    option broadcast-address 192.168.6.255;

    option routers 192.168.6.1;

    authoritative;

}

subnet 192.168.110.0 netmask 255.255.255.0 {

    range 192.168.110.101 192.168.110.254;

    option subnet-mask 255.255.255.0;

    option broadcast-address 192.168.110.255;

    option routers 192.168.120.1;

    authoritative;

}

subnet 192.168.120.0 netmask 255.255.255.0 {

    range 192.168.120.101 192.168.120.254;

    option subnet-mask 255.255.255.0;

    option broadcast-address 192.168.120.255;

    option routers 192.168.120.1;

    authoritative;

}

default-lease-time 600;

max-lease-time 7200;

[iamkinghenry]

Finally, I got it to work.  Thanks for all the help.

So I had to create 1 Virtual Machine under the vSwitch1 with "All (4095)" for VLAN ID.  The non-VLAN traffic is also getting IPs.

[tomtom901]

That would indicate that the Debian VM will do VGT (VLAN Guest tagging) - meaning it will tag VLAN traffic and not let vSphere handle this.This is what you want?

[ucusan]

This is OK if you want to do the tagging at VM level. Anyway, unless you configure the physical port in trunk mode you will not be able to establish external traffic (at least on a standard enterprise network infrastructure). But I'm starting to think that all your traffic is internal, right?

[iamkinghenry]

I have APs (UniFi) that tag the traffic and a switch so the server will be a DHCP server with PacketFence.  I'm using a computer on a VLAN that I registered through PacketFence for network access/Internet and it's working.

If my APs were better then I wouldn't be doing this inline enforcement setup but my APs won't work with PacketFence VLAN enforcement.

I just trying to get better control of my network with tenants on it.  I have groups of VLANs currently with QoS & firewall rules on my router (EdgeMAX) to keep the network secure and from getting overloaded.  I have the DNS set to opendns.org and have the router blocking p-2-p for some protection. I'm just experimenting with PacketFence features to make my network better.

I really appreciate the comments and help.