Need Help to Understand the vSphere Networking Environment

[yvv47]

Hi,

I need your help in understanding the network environment in vSphere. Please look the below diagram and text:

NIC#1 – Management Port (Access it from vsphere client, vCenter server)

NIC#2 – VMkernal Port (For iSCSI, vMotion etc)

NIC#3 - ?

NIC#4 – ?

Q1 How do I allow the external users to access the services over ESX hosts? (e.g., IIS, FTP, Exchange, NFS)

Q2 What will be the role of remaining two NICs? Will it be connected to the physical switch?

* Request you all to please help me by sharing your knowledge / experience on the networking part you’ve done in your environments like how many NICs, what to do with that. *

I really need to undertand the networking concepts in vSphere, hope to receive your help!

Regards: Yash

[iw123]

Hi

Normally you would dedicate the remaining NICS for virtual machine guest traffic, though it vsphere there are many ways in which to set up your host networking, depending on your environment.

The best place to get started is the vsphere networking guide - see link below:

http://www.vmware.com/support/pubs/vsphere-esxi-vcenter-server-pubs.html?rct=j&q=&esrc=s&source=web&cd=2&ved=0CFsQFjAB&url=http://www.vmware.com/go/support-pubs-vsphere&ei=0LjVT-PvLeem0AXTidGSBA&usg=AFQjCNGXtIiO0Z1fajLgAxEYdJ3VXGQBcA

This will help you to understand what network options you have for your hosts.

Hope this helps

Thanks

[yvv47]

Hi Ian (iw123), thanks for your help and the link of the VMware resources. It is really very helpful.

thanks!

[logiboy123]

Please check out my vSphere host networking designs, it may help you visualize what you are trying to do and will also provide some good links to expand your understanding of vSphere networking concepts;

http://vrif.blogspot.com/2011/10/vmware-vsphere-5-host-network-designs.html

Regards,

Paul

[yvv47]

Thanks Paul for sharing!

[rickardnobel]

Do you only have 4 NICs in the host? If you should run both Management, vMotion, iSCSI and VM network it could be a bit too much.

Typically you will not want to only have 1 physical nic port for each function (i.e. iSCSI, vMotion, Management) but at least two, connected to two different physical switches, to avoid single point of failures.

[yvv47]

Sorry I could not get you completely as I am a newbie at networking side. (can understand from diagrams)

are you asking me to do like this? -

Blue: VM Network Port | Red: VMKernal Port

[rickardnobel]

The way on your picture is a good way in general to setup the networking, this provides a good redundancy.

You must then configure how to attach your physical NIC ports (called vmnics) to your vSwitches and how to group the different functions (VM traffic, Management, vMotion, iSCSI).

Do you know if you are using all of these?

[yvv47]

Hi Rickard, as of now we are not using esx(i). I want to try the activity at our office lab :smileyhappy:  for future implementation.

We have two hosts, each having 4 NICs, SAS storage, switch. What I want to learn is to make the basic vSphere environment in a proper way. I have tried and even got succeded in creating an environment with shared storage, but I know that I was just doing it without any proper steps. (lack of experience, exposure :smileysad: )

I am having problems in networking part, what to do with 4 NICs properly. It would be a privilege to me if you can suggest or guide me how we can achieve this task for a small implementation.

Thanks for your inputs :smileyhappy:

[rickardnobel]

yvv47 wrote:

Hi Rickard, as of now we are not using esx(i).

---

I am having problems in networking part, what to do with 4 NICs properly. It would be a privilege to me if you can suggest or guide me how we can achieve this task for a small implementation.

Do you have the possible to add one more physical network card in your host? One more card with 2 or 4 ethernet ports would make your setup much better.

[Josh26]

Hi,

Whilst I'm typically against the trend of adding seemingly endless NICs to an environment to physically separate out every possible service until your servers run out of PCI slots...

You should double each NIC. Therefore your management/vmotion NICs should be NIC1 and NIC2.

Then you will need VM traffic, 3 and 4.

The thing is if you are using iSCSI, you are going to want that separated, that means two more NICs, and more importantly, at least one more switch, preferably two.

[rickardnobel]

Josh26 wrote:

Whilst I'm typically against the trend of adding seemingly endless NICs to an environment to physically separate out every possible service until your servers run out of PCI slots...

..

The thing is if you are using iSCSI, you are going to want that separated, that means two more NICs, and more importantly, at least one more switch, preferably two.

I am on the other hand somewhat against the trend of recommending completely separate physical switches for iSCSI. :smileyhappy: This means not that it is not often necessary, but before we must know what kind of physical switch/switches that are already in place. If the switch is of good quality and could guarantee a backplane-throughput enough for all ports simultaneously then there is in my opinion enough to separate iSCSI into its own VLAN.

[Josh26]

Rickard Nobel wrote:

I am on the other hand somewhat against the trend of recommending completely separate physical switches for iSCSI. :smileyhappy: This means not that it is not often necessary, but before we must know what kind of physical switch/switches that are already in place. If the switch is of good quality and could guarantee a backplane-throughput enough for all ports simultaneously then there is in my opinion enough to separate iSCSI into its own VLAN.

Certainly when someone has a Cisco Nexus or other high end switch I'm quite happy to suggest sharing it.

But when I see diagrams like this, two servers, and a technician asking for basic information, it's usually the sort of environment I see an eight port Netgear providing iSCSI and VM connectivity. Everyone thinks they have a "good" switch until priced on something appropriate for sharing iSCSI.

[rickardnobel]

I totally agree with you. So, yvv47, what kind of physical switch (or switches) do you have for this setup?

[yvv47]

I have following products: PowerConnect 2824 switch, two R710 Server, Powervault 6100 for implementing the small vSphere environment

[rickardnobel]

yvv47 wrote:

I have following products: PowerConnect 2824 switch, two R710 Server, Powervault 6100 for implementing the small vSphere environment

A quick check on the specifications for your switch says it should be ok for this. I have not worked with this switch type myself, but this document from Dell claims:

http://www.dell.com/us/enterprise/p/powerconnect-2824/pd

"The PowerConnect 2824 LAN switch is a high-performance Gigabit Ethernet solution delivering full wire-speed switching across all ports"

If the switch really is non-blocking and allows all ports to run at full wirespeed at once then this should be good in your setup. As noted earlier a very important best practice in ESXi setups is to have two physical switches. This gives you a lot more fail over capacity and allows one of the switches to break / reboot / lose power and all connections will stay up.

As mentioned earlier, if you could add another network card with 2 or more ports to your hosts it would be very good.

[yvv47]

Thanks Rikard for your comments. It really looks great to have a redundant network.

I think we can add an additional NIC to the host. :smileyhappy:

[rickardnobel]

With an additional NIC with 2 ports a general setup could look something like this:

vSwitch0 - vmnic0, vmnic1 (each connected to different physical switches)

VMkernel Management network

VMkernel vMotion network (own IP network, own VLAN)

Best if these are setup as active/standby.

vSwitch1 - vmnic2, vmnic3 (each connected to different physical switches)

VM portgroups

vSwitch2 - vmnic4, vmnic5 (each connected to different physical switches)

VMkernel iSCSI (own IP network, own VLAN)

Best if this is added to the iSCSI Multipath component, which is quite easy in vSphere 5.

[yvv47]

thats great! :smileyhappy: thanks for your help!