unique :Network Uplink redundancy lost

[vDeepak]

One of our esx host is showing alarm for "Network Uplink redundancy lost ". we are using vsphere 4 and esx box is in a cluster.

I checked and found that all of our Vmnic and switches are working fine. and service console is already connected to 2 vmnic over virtual switch.

And when i run esxcfg-config -l command , i found that all vmnic and switches are in Up condition.

Any ideas?

[vDeepak]

Also i checked the KB article :kb.vmware.com/kb/1009542 , but this does not reolved my problem. As none of our vmnic is showing red mark or showing down when checking through command line.

Any sugestions ?

[a_p_]

Maybe there was a short outage, which triggered the alarm.

Once everything is working again, you will have to acknowledge or reset the alarm manually.

André

[vDeepak]

There was no such outage in the past.

Is there any alternative way , so that i can check it out and fix it.

Thanks

[dnetz]

Are you sure about that? You should check /var/log/vmkwarning (or if it's vmkernel) on the ESX host and see if it logged any interfaces going down, it should appear in the log as "vmnicX NIC Copper Link is Down" and "Lost uplink redundancy on virtual switch "===

If you find this information useful, please award points for "correct" or "helpful".

[vDeepak]

i found few vmkwarning files under/var/log and they are named as vmkwarning.1 , vmkwarning.2 etc..But wen i am opening them in VI editor its blank .

Please help.

[vDeepak]

oops..i got it..i found one error message of vmnicXX: Link is Down,.

But am eager to know here the cause for this so that i make it permanent fix . in the logs i didnt find the cause for it.

Ideas?

[dnetz]

The ESX host can't possibly know the cause of the outage, all it knows is that it lost the physical connection to the switch, maybe through faulty wiring, a faulty switch, someone tripping over a cable etc. You would need to monitor the switch to better tell what happened.

===

If you find this information useful, please award points for "correct" or "helpful".

[vDeepak]

ok, i understand!

Also is this something related to network teaming as well , i checked and found that my affected vmnic is teamed to another vmnic...and after searching articles ..few people are saying this is becuase of teaming..but no clear answer.

[dnetz]

If your vSwitch has more than one network adapter configured in a failover configuration then your host would continue without interruption, but it's farfetched to believe that the failover configuration is what caused the uplink port to fail in this way.

===

If you find this information useful, please award points for "correct" or "helpful".

[vDeepak]

Yes, i can understand that when i am using Network teaming then there is fault tolerance . But exactly am eager to know is the cause for this alert.

Altough i am using Vsphere 4.0 and if you check : http://kb.vmware.com/kb/1007041 , it says that this is because of Network teaming which is causing "network uplink redundancy lost", although this issue is fiexed in this release of 3.5.

I need to fix it up and need ideas here.

[dnetz]

That patch is almost two years old, I doubt it relates to ESX 4 in any way. As I said before, you should monitor your network infrastructure to see if anything else is affected, if you can't isolate the problem you're left to guessing.

===

If you find this information useful, please award points for "correct" or "helpful".

[ChrisClayton]

On what do you base your assumption that there is something faulty with ESX that needs to be fixed? Are you sure that this isnt a problem with your ethernet adapter, physical cable, switch, etc. What time did this occur, was anyone in your datacenter, in the rack, at this time?

[vDeepak]

This is so , because this error comes over some of the other ESX too sometimes . And i can assure that there was nothng wrong with physical cabling etc.

Thanks

[vanille]

I've got exactly the same problem.

Is there a solution somewhere?

vanille

[BradL]

I am having this issue as well

I just built a new host and patched it to current patch level

added it to the cluster and tested a vmotion - all good

put it to maintenance mode and built out the teaming on the Cisco switches (redundant 6509)

once that was completed - brought it out of maintenance mode and the error appeared

back to maintenance mode and a reboot of the host

still there when it came back up "network uplink redundancy lost"

I tested the host out and appears all links are working correctly

as above - curious about the error

[vDeepak]

What i believe one RCA for this issue was cabling fault. Even though when we check our network and cabling it all seems fine and Network is running fine.

For me this alarm was generated because of some fluctuation at Cables, when i get a deeper look over vmkernel logs then i found the same.

What i suggest to you is that just try to make deeper look over our vmkernal logs when the alarm was generated..i hope surely you will find some vmnic flaps etc..which was no longer exists and we can easily acknowledge the alarm.

Thanks

[BradL]

I will have a look at the vmkernel logs

Am I to understand that if at some point an error - a flap - occurs, the alarm will be displayed - but if the condition is resolved as you suggest I will discover in the logs - the alarm will not be cleared - it is intentionally left for me to clear.

[vDeepak]

For me, Yes this was the case.

Thanks

Deepak

[ericdaly]

Having the same issue approx once a month at a customer site. Seems to happen during high disk IO times (iscsi san) while vRanger is running. Interested to know your guys configs. I do know that the issue always occurs on the same nic (VMNIC3) of a random host each time.

HP BL460C G6 ESXi 4 U1 (Build 219382)

NC326m Dual Port PCIe

Embedded HP GBe2c L2/L3 Switch

Nothing in my ESXi logs anout vmnic3 disconnecting?

[Phokay]

I had the same issue once at a customer site (with iscsi san). It turned out that I missed configuring RSTP/Portfast on the switch and one of the switches blocks the traffic to the looped NICs. The switches were not stacked but connected with ISLs and Etherchannel was not configured on the switches.

Take a look at the the switches which are connect to ESX's NIC team.

Good luck.