vMotion in v8.x

[Bill Clark]

Currently migrating to v8.x and have questions regarding vMotion.  In the past, we only had a single physical port (1GB) dedicated to vMotion.  With new hosts we have allocated x3 1GB physical ports to vMotion.  We currently have a single VMKernel adapter that is tied to a standard vSwitch with those 3 ports.  I read in a now-absent article (here from the WayBack Machine:  Faster vMotion Makes Balancing Workloads Invisible | VMware)  that all VMKernel interfaces enabled for vMotion are checked out-of-the-box and that multi-nic setups for vMotion are a thing of the past.  So my underlying question is this:  Do I need to do anything beyond what I currently have setup, for vMotion to maximize throughput and use all 3 of the assigned physical ports?

[a_p_]

"This means that all VMkernel interfaces enabled for vMotion are checked ..."

A Multi-NIC vMotion setup is still necessary to use multiple interfaces as only vMotion enabled VMkernel port groups are considered.

André

[Bill Clark]

So go ahead and follow this KB regarding a multi-nic setup then?  Multiple-NIC vMotion in vSphere.  The article specifies 2 physical ports, what do I do for 3?  Do I just setup the two VMKernel portgroups in the article and have 1 active, 2 standby for each portgroup?  Thanks.

[a_p_]

Yes, one active vmnic per vMotion port group, and all other vmnics are standby.

André

[Bill Clark]

Sorry for the back and forth. But following the article I will create 3 port groups (instead of 2), where each physical port gets to be active for its own port group, correct?  Lastly, do I need to do anything on the core switch stack besides have them in the same vLan?  Any need for an etherchannel group that has the 3 physical ports as members? Thanks again.

[a_p_]

Yes, each vMotion port group will have its own active vmnic, and the other two set to standby.

There's nothing special that needs to be done on the physical switch. Just configure the ports as either tagged or untagged ports, depending on your preferences.

André

[Bill Clark]

Thanks for the assistance Andre!  I found another article that explains the process better than the Broadcom article and it verifies what you were saying.  Here's that new article:  ESXi vMotion Network Configuration: Best Practices .  I now have the first new host configured with 3 vMotion nics and initial testing works.  Will be adding the other new hosts with identical settings next week so I should be able to do some stress testing then.  Thanks again for your help!

[PaulB2]

Verify vmotion is using all 3 nics. Enable SSH on your host, ssh to the host console, run esxtop, press s and then 1 to set the refresh to 1 second, press n for network screen. Watch the 3 nics as you do a vmotion. You should see a burst of traffic on all three if setup correctly.

Press q to quit esxtop. Remember to stop the ssh service.

------------------------------
-pb
------------------------------

[Bill Clark]

Following your steps, our initial testing between two hosts with the multi-nic configuration shows only a single port being used for vMotion.  We tested a single VM migration, seeing no activity on the other ports, then tried 3 VMs at once and same results.  Ideas on where to start troubleshooting?  Just as an FYI, we do not have these vMotion ports in any type of portchannel or etherchannel group on our core stack of Cisco switches.