Nic teaming with ESXi and Cisco port-channel

[jackpod]

I've spent time searching this and haven't found either what I was looking for or an article that I can understand. I have a dell 1950 with ESXi 4.1, both nics are connected to a port-channel on a cisco sup IV. It works ok as I can create several virtual machine port groups and vm kernel ports to allow different hosts on different vlans, but it appears I am not utilizing the agregation of the port channel. The load balancing policy exception is set to "Route based on the originating virtual port ID". From the articles I have found it appears that the proper selection would be "Route based on IP hash" However when I select this option I get error messages on the console of the router module stating that the host is flapping. Also when I do a show ethernet summary on the Cisco it shows the 2 ports in that group are connected but in stand-alone mode

Any ideas?

[weinstein5]

Welcome to the Community - To use route based on IP hash you will have to enable Ether Channel on the ports being used by your ESX host -

[Rumple]

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004048

Basically you need Mode ON set for your port channel and you need to also configure a native vlan on the port channel that will never be used (like 666 or something).

then you can set route based on ip hash (might need to unplug one cable while getting setup to get it working)

These are not bad instructions above...

[rickardnobel]

Mark Hodges wrote:

you need to also configure a native vlan on the port channel that will never be used (like 666 or something).

This is as long as you actually have set a VLAN number on all portgroups. If there is any portgroup without a VLAN id then this traffic would fail, so just check the configuration the vSwitches before changing this on the physical switch.

[jackpod]

Ok, I played with this a little bit and still no go, The port channel is on native vlan 1, I tried changing it to an unused vlan ID, changed one port to channel-group XX mode on, changed the vmhost to routing by IP hash and and soon as I change the second port to Mode on, I lose connectivity. However when I change both ports to Mode ON, "show ethernet summary" shows the 2 ports as aggregated. Should I change the port channel to the vlan I have setup for management? When I get a minute I will post snipets of the config and a screenshot of the host. This isn't critical as it is working as is, but I have 16 guests on this host and all guests are stored on an Iscsi datastore so if I could get it to aggregate the 2 nics instead of just load balance/failover it would be nice

[Rumple]

I always leave only a single cable plugged in until its all configured. Once its all configured then I plug in the other cable.

PS – are you actually using vlans on your network or are you just using vlan 1 everywhere?

[jackpod]

I have 12 vlans the reason for wanting the trunk ports so I can have guests on different vlans. Also the iscsi is on an isolated vlan

Connected by DROID on Verizon Wireless

[Rumple]

Did you double check the src-ip piece on your cisco switch?

Usually as long as you configure the port channel, virtual ip hash, and start with a single cable plugged in, its somewhat simple to get going

[jackpod]

Ok, so I thought someone had posted the src-ip piece on this thread, but I guess not, maybe it was one of the other several pages I had open. I will need to find it to try it again. I will try it with only one nic plugged in once I find that command

I really apreciate all the helpful posts

[jackpod]

Ok, so I got it to work, well sort of.

By doing a show etherchannel load-balance, it shows the switch is configured for src-dst-ip

I brought the nics up one at a time, but after about 15 minutes I lose the management interface, but I have one guest running that lives on the Iscsi Vlan and has an address of the server vlan (different vlan from the iscsi. although I lose management I have a ping -t running that doesn't appear to be interrupted. On the management interface on the vmhost, I select vlan 2 and provide an ip that resides in the vlan 2 address space, also when I run the test from the console of the vm host, the gateway fails but it finds the dns servers which again live in a different vlan (server vlan). The only way I can get the management back is to shut down one of the ports in the port channel then I can re enable the second port

[Rumple]

And the native vlan is configured for a non functional vlan?

Here is another great resource to check

http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/

[jackpod]

I guess it just wasn't meant to be LOL Yes I created a non functional vlan. I setup a kernel for my iscsi vlan, setup the iscsi initiator, created another kernel and port for my server vlan, brought up a vm, RDP'd it with a ping t to a different vlan. Exactly 12 minutes later (I did this 4 consecutive times) the management interface goes away and I am not able to connect to the management interface without shutting down one of the ports, however the rdp'd vm that lives on the iscsi box is still up and pinging away.

Strange