Extra Nics in Standby or Active?

[mrizos]

I have 3 ESXi servers with 4 physical nics (3 nics for iSCSI and 1 for Management Traffic)

Each physical nic has 2 ports.

All the ports on each nic are connected to 2 Dell 6224's that are stacked via 10G.

Currently, I have 1 port on each nic active and one in standby (under nic teaming)

Do you guys think I should make all the nics active, some of them (like for iSCSI) or keep them as they are (1 port active and 1 port in standby)

As always, thanks guys!

[idle-jam]

For my case it would be it as active active with the right load balancing technics (depending on your switch configuration).

Check this out http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1004088&sliceId=1&docTypeID=DT_KB_1_1&dialogID=140441286&stateId=0%200%20140439823

iDLE-jAM | VCP 2, VCP 3 & VCP 4

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points.

[logiboy123]

So you effectively have 6 x 1GB network ports?

In this setup I would have 2 x 1GB NIC on a switch for the vmkernel (service console if you have ESX) and VMotion. Each using a different VLAN. Further I would specify an Active/Standby configuration for teaming as follows:

NIC1 - VMkernel Active / VMotion Standby

NIC2 - VMotion Active / VMkernel Standby

In the above configuration you get network redundancy for the vSwitch, VLAN segregation is used to keep traffic apart but even further load balancing is configured to keep traffic apart as well. This is a robust and secure setup.

For iSCSI I would probably use multipath IO in a round robin configuration, especially if you are using a P4000 or something.

So you would set up your Switch for iSCSI and create a VMKernel port for every NIC that you have assigned. Then you go onto the console (yes, even on ESXi this needs to be done) and bind each vmkernel on the iSCSI switch to the iSCSI software initiator. From memory the cmd line syntax would be:

VMware CLI - esxcli swiscsi nic add -n vmk1 -d vmhba33

VMware SSH / Telnet session - esxcfg swiscsi nic add -n vmk1 -d vmhba33

The above cmd line presumes that you are using the software initiator and that it is in position vmhba33. You can verify where the software storage adapter is by opening up the configuration storage adapters tab in vsphere. The above command needs to be done for each NIC on the switch. If you want Jumbo Frames then this also needs to be configured on the switch using cmd line. I can't remember exactly what the syntax is though.

Then on the switch configure each NIC in an Active / Unused configuration, where Active is a single NIC in the team and every other NIC is unused.

Now create your iscsi initiator connection and authentication if required.

Provision a LUN and when the LUN appears click "manage paths" and set the type of connection to "round robin" instead of the default. For each LUN you will need to manually specify the Round Robin configuration.

It is important to note that Host Profiles will not apply the above

configurations done on the command line. Not sure why, but it doesn't

work even on 4.1.

If all the above is a bit confusing then I recommend reading the following documents:

iSCSI SAN Configuration Guide 4.1

http://www.scribd.com/doc/24586958/Configuring-Left-Hand-ISCSI-and-VSPHERE-MPIO

[mrizos]

Wow! Thanks for that excellent post! I've never heard of the round robin stuff (using a AX4-5i SAN from Dell) and since I'm not a network guy I've never actually created a vlan (although it look's easy enough in the web based dell GUI). I'm going to digest this post tonight. Thanks again!

[logiboy123]

I consider VLAN segregation a must especially if your infrastructure is running on a shared network.

Having said that, even if I had a seperate and unconnected switch for iSCSI I would probably still use VLAN's because it adds another layer and security, but very little as an overhead as far as management goes.

I try to provision my iSCSI switches in such a way that I don't need to povision iSCSI connections to my VM guests. This removes the vulnerability of allowing VM's to see storage traffic. However in a network load balancing situation this isn't always possible, some applications or OS want to see RAW disks. In which case I just add a VM Network port group to the vSwitch and assign this to the VM.

Cheers,

logiboy123