new to vmware, vsphere and vswitch networking

[kohszejie]

Hi all,

I am new to virtualization but have decided to try out since my company is going full fledged on ESXI5.5.  Therefore I hope some kind gurus here can point me in the right direction with regards to some doubts that I have... and do forgive me if i did ask some ridiculous questions...

==========

So far,  I have know that VMs, through their vNIC connect to vSwitch which is in turn connected to the physical machine or ESXI host's network adapter (vmnic). The vmnic is then connected to an actual L2/L3 switch.

[VM's vnic] <---> [vswitch] <---> [vmnic] <---> [physical switch]

However, what i do not understand is how does the vmnic actually works to propagate traffic "inwards" to the vswitch ? I can't really come up with a single question to explain my doubt but below are doubts that I have been asking myself ->

• How is it able to assign IP off to the internal VMs ?
• When a external packet from the physical switch is being send to the vmnic, how does it pass it internally to the internal VMS ?
• How does it even know that the designated IP belong to a VM inside and will pass it "inwards" to the vswitch ?
• Is the vmnic using the IP or MAC address to decide whether to pass the packet inwards to the switch ?
• When a arp request for a VM's IP is send out from external, will the vmnic reply with its mac address ?
• When a arp request is send out from an internal VM, does the vmnic keep track of the mac address of the VM so that next time when it saw a reply to that MAC, it know to "forward" inside to the vSwitch
• Is it working something like a "Bridge mode" bridging the internal vswitch to the external physical switch ? Is there a mac address table belonging to the vmnic ?
• Whenever VM's IP is being assigned, will the IP be "bind" to the vmnic ?
• How do we list all the IPs or MACs (if there are) that are associated with a vmnic ?

Or rather simply put, i am wondering how does the physical nic on the host machine (esxi) , know that it must forward a frame "inwards" to the vswitch ? is there the vm's IP bind to the interface ? or nic did keep a mac address table or something ... (but mac address table suppose to belong to vswitch ?)

Thank you.

Regards,

Noob

[kohszejie]

Anyone ;()

[SRoland]

Hi,

Welcome to the community!

I would start with a doc. I recommend you to read our VMware Virtual Networking Concepts guide which I hope will help you to better understand the basic details of the virtual switching within VMware even though it is a bit old.

On your last question though:

> Or rather simply put, i am wondering how does the physical nic on the host machine (esxi) , know that it must forward a frame "inwards" to the vswitch ? is there the vm's IP bind to the interface ? or nic did keep a mac address table or something ... (but mac address table suppose to belong to vswitch ?)

The vminc (i.e. the physical adapter) on the ESXi host does not know that there is a virtual switch behind it. It forwards the packets to the driver then the upper layer software managing it. There is no magic about that. In some cases there are some techniques added (for example netqueue) but this basic remains the same. The vmnic (i.e. the physical NIC adapter) does not keep VM MAC address information about the VMs, but the vSwitch does. But even the vSwitch does not keep IP address details.

A virtual switch as such does not have the same kind of mac address table like a physical switch. If a VM powered on then the vSwitch stores the virtual port-number <-> MAC  address combination and keeps it that way (so far that is not really different than a physical switch). A big difference is that though that the vSwitch does not learn MAC addresses from the external network.. If there is a frame from outside with unicast MAC address for which there is no port associated then the packet gets dropped. And if the VM sends traffic to another MAC which does not exist within the virtual switch then it will be forwarded out through the uplink(s) (if exists).

The MAC address of the vmnic does not play here (except if one of the vmkernel interface gets its MAC from an uplink adapter).

I hope this helps. Let me know if you still have questions.

//Roland

[kohszejie]

Hi SRoland,

Thanks for replying to my post!.

In your explanation,

q1) does it means that somehow the VMnic / physical adapter is in promiscuous mode accepting all frames being send to it ? and propagating up to the higher level stack (e.g. vswitch to see if the mac address belong to one of the VMs) ; else will be drop accordingly  - am i right ?

q2) also you mentioned that vSwitch does not learn mac address from external network, meaning if there is a external mac address coming in, it will not be recorded as all non-VMs mac addresses will be forwarded to the uplink interface  <-> meaning vswitch only contain mac address for its VMs - am i right ?

q3) I am running ESXI 5.5 and connecting to the ESXI via vsphere client. I have 2 physical NICs on my Host machine.  vswitch0 is connected to vmnic1. 

vmnic0 is not physically connected and is down. at the moment.

However, i am not able to create any VLAN in the vswitch0 nor am I able to create another vswitch.  I need to create 3 different network for my VM (1 for production, 1 for storage access and 1 for private interconnect)

Is it because i am using a free hypervisor version ?

Regards,

Noob

[kohszejie]

Dear Roland,

You still around ?

Regards,

Noob

[SRoland]

q1) Yes, that is the situation, but the thing is that the physical switch will not even send these kind of packets to the host because the physical switch will only send for which it learnt the traffic through that port where the NIC connects.

 

q2) Yes.

q3) you should be able to create another vSwitch, even without uplink. Also configuring VLAN on a port-group should work. Waht do you mean on "free version"? Are you using this in the 60 day trial period?

[kohszejie]

Hi Roland,

I got my VMware vSphere Hypervisor 5.5 from VMware as a free product. There is no trial period.

When i go to networking properties and click on Add, I can only choose Virtual Machine and VMKernel

If i choose Virtual Machine, I am able to create another VLAN in vSwitch0.

But how do i create another vSwitch ?

Regards,

Noob

[SRoland]

Ok, thanks it is clear now the version.

On the vSwitch creation whatever option you chose there (UPDATE: I mean in the previous step on your screenshot) it should come up with this ( actually something similar )  as a next step:

That's where you can select the existing vSwitch or create the new. Otherwise I do not think that this free version has any limitation different in regards networking compared to a licensed standalone host.

//Roland

[kohszejie]

Hi Roland,

I think somehow, i make a mistake.. if I go to the option below, i am not able to create another vswitch,  I am just able to create another vlan or vmkernel.

I have to go to the option here below to add another vswitch :smileywink:

===============================================================================================

Roland, I have got a last favour/question to ask you..

Q1) In the environment above, it is setup at my home. I do not have any L2/L3 physical switch. The NICs are connected to my home router with only 1 routed port which is used for connecting to the Internet.

Given my current setup, If I need to setup 2 different network (VLAN) in a vswitch but the VMs in the VLAN must be to intercommunicate, there is no way for me to achieve that right  ?

Q2) If 1 of my VM is setup with 2 vNic, am i able to still do teaming/etherchannel/lacp with vSwitch ? or people don't normally do LACP between VM and Vswitch. ?

Q3) How many layers of teaming(lacp) should be done ?

Can we do LACP between the vSwitch and the physical adapters then ?

Do we also do teaming on the ESXI host level <---> to the physical L2/L3 switch ?

Regards,

Noob

[SRoland]

OK, I just assumed you clicked on "Add Networking" in the earlier question :smileyhappy:

On the questions:

Q1) No. You will something which routes between VLANs and handles the trunking (802.1q frames) on the physical side.

Q2) There is no such thing as LACP between the VM and the vSwitch. LACP can only be exist between the host and the physical switch and only on the distributed switch which is not available if you are not using vCenter (see below).

Q3)

> How many layers of teaming(lacp) should be done ?

Not clear what's the question here. On LACP see the answers below.

> Can we do LACP between the vSwitch and the physical adapters then ?

Yes, but as you see above only with vDS. On the other thing there is a common mis-conception here. You may do ether-channel, or "link aggregation" without the use of LACP on the vSwitch without problem,. But to use the LACP protpocol for channeling you will need the distributed switch. See the following KB about this:

Sample configuration of EtherChannel / Link Aggregation Control Protocol (LACP) with ESXi/ESX and Cisco/HP switches (1004048)

> Do we also do teaming on the ESXI host level <---> to the physical L2/L3 switch ?

If you want to do so, yes. You can go without it as well. Check the "teaming and failover" options for the port-groups, or (again) here is a KB in detail:

NIC teaming in ESXi and ESX (1004088)

HTH

//Roland

[kohszejie]

Hi Roland,

Thanks for replying.

I am actually abit confuse about the following 2 types of ethechannel/teaming below

1) between external switch and vmnic

2) between vmnic and vswitch

especially 2.

=========================================

For 1)

Lets say i have team up vmnic0 and vmnic1 with physical switch fe0/1 and fe0/2 respectively.

Now traffic can flow through both the links.

For 2)

What will happen when if i have or have not setup etherchannel/bundling between the vmnic and vswitch ?

Regards,

Noob

[SRoland]

I think there is a confusion here. There is no etherchannel possibility between the vSwitch and vmnic. Any kind of channeling (if any) is between the external switch and the vSwitch.

The vSwitch is just using the vmnic to send/receive traffic and that's it. All the settings are on the vSwitch. No config/setup/setting should happen on the vmnic, other than link speed/duplex....

Treat the vmnic as a port/pipe where the vSwitch just communicates with the external world.

[kohszejie]

Hi Roland,

In that case can you elaborate further on the 2 points below

> Can we do LACP between the vSwitch and the physical adapters then ?

Yes, but as you see above only with vDS. On the other thing there is a common mis-conception here. You may do ether-channel, or "link aggregation" without the use of LACP on the vSwitch without problem,. But to use the LACP protpocol for channeling you will need the distributed switch. See the following KB about this:

Sample configuration of EtherChannel / Link Aggregation Control Protocol (LACP) with ESXi/ESX and Cisco/HP switches (1004048)

> Do we also do teaming on the ESXI host level <---> to the physical L2/L3 switch ?

If you want to do so, yes. You can go without it as well. Check the "teaming and failover" options for the port-groups, or (again) here is a KB in detail:

NIC teaming in ESXi and ESX (1004088)

Isn't NIC teaming on the ESXI host level <--->physical switch  = Ether-channel ?

Regards,

Noob

[kohszejie]

Hi Roland, you still around ?

Regards,

Noob

[SRoland]

> Isn't NIC teaming on the ESXI host level <--->physical switch  = Ether-channel ?

Yes...something still not clear?

[kohszejie]

Hi Roland,

> Can we do LACP between the vSwitch and the physical adapters then ?

Yes, but as you see above only with vDS. On the other thing there is a common mis-conception here. You may do ether-channel, or "link aggregation" without the use of LACP on the vSwitch without problem,. But to use the LACP protpocol for channeling you will need the distributed switch. See the following KB about this:

Sample configuration of EtherChannel / Link Aggregation Control Protocol (LACP) with ESXi/ESX and Cisco/HP switches (1004048)

q1) I think i got confuse when you replied yes to the question on whether we can do LACP between vSwitch and physical adapters. I think you meant physical switch right ?

==============================

q2) In the event i would like to have nic teaming on my VM level to vSwitch, it means i am not able to do so ? What if i want redundancy for my NIC at VM level ?

Regards,

Noob

[SRoland]

> q1) I think i got confuse when you replied yes to the question on whether we can do LACP between vSwitch and physical adapters. I think you meant physical switch right ?

I answered this question:

> Isn't NIC teaming on the ESXI host level <--->physical switch  = Ether-channel ?

And this I really meant yes. The teaming is only possible between the host and the physical switch. Of course when I say "host" in this interpretation I mean the virtual switch on the host. As I said forget about the physical adapters in this case, they are just the 'middle-men'.

So, you can only do LACP between the virtual switch and the physical switch.  Nowhere else in the path between the VM and physical world.

> q2) In the event i would like to have nic teaming on my VM level to vSwitch, it means i am not able to do so ? What if i want redundancy for my NIC at VM level ?

No you cannot team within the VM with any external entity. That would be like trying a teaming with a switch across another switch. On the other hand why would you do so if the virtual switch can provide uplink connection redundancy? You can have additional interface if you want in the VM but no channelling/bonding/LACP (whatever we call it) is possible.

I suggest to read through the docs I sent you about virtual switching.

Regards,

//Roland

[kohszejie]

Hi Roland,

Thanks for replying.

On my physical setup, i will always team my 2 nic ports on the host to the switch. On the switch, it might have another etherchannel between itself and another switch.

So i am thinking in sense of the VM will be team to the Vswitch, while the Vswitch will be uplink/teamed to another physical switch.

What is 1 of my vmnic's vnic got down, at least i still have 1 vnic talking to the vswitch. Isn't it ?

Regards,

Noob