Enable Management Network: Error

[RolandCH]

Hi  On one of our ESXi 4.1.0 hosts, it's not possible to enable the Management Network via the console. Of course it's strange that it's disabled at all. If I try to enable the Management Network again, it's show up with the message "Operation on the management network interface failed",  Even the Management Network is disabled (no connection is possible via vSphere Client or from the vCenter), it's still possible to make a ssh connection to that host or to ping that host via the Management Network Interface IP Address.  What can I do to restart the Management Network? To restart the host is not an option.  Thanks in advance,  Roland

[vMario156]

Allready tried to restart the management agents via the dcui?

Regards,

Mario

[RolandCH]

You meen with the command /sbin/services.sh? Yes I did, like just written in the other answer.
Otherwise I do not understand what you mean with dcui.

[Sreejesh_D]

since ssh is possible try restarting all services using the command /sbin/services.sh.

it will not affect the running VMs, its for restarting the services like hostd, vpxa, sfcb etc.

[RolandCH]

I tried this already two or three times. The command was working without any problem as far as I saw, but the problem didn't changed.

[nielse]

Did you check the logs for errors?

Also can u perfectly ping the management IP?

[RolandCH]

For me it's a chicken and egg question. I thought the following log error is caused because the management network is not up and running - but maybe this is the reason why it's not up and running. If I try to restart the management agents direct on the console of the host (DCUI), the following log (/var/log/vmware/hostd.log) is created (I deleted all lines with info and verbose, there are hundreds of such lines).

Section for VMware ESX, pid=9018516, version=4.1.0, build=build-260247, option=Release
...
[2012-03-29 11:54:47.986 FFFB8E90 error 'FSVolumeProvider'] FSVolumeVmkImplProvider::Start() called
...
[2012-03-29 11:54:53.894 FFFB8E90 warning 'NetworkProvider'] Got an invalid uplink [vmnic5_NAS02].
...
[2012-03-29 11:54:54.033 FFFB8E90 warning 'NetworkProvider'] Got an invalid uplink [vmnic6_NAS03].
...
[2012-03-29 11:54:57.856 71529B90 warning 'NetworkProvider'] Got an invalid uplink [vmnic5_NAS02].
...
[2012-03-29 11:54:57.996 71529B90 warning 'NetworkProvider'] Got an invalid uplink [vmnic6_NAS03].
...
[2012-03-29 11:54:58.085 FFFB8E90 panic 'App'] error: no element found
[2012-03-29 11:54:58.085 FFFB8E90 panic 'App'] backtrace:

[00] rip 1904f8b3
[01] rip 18ee1cae
[02] rip 18e81682
[03] rip 18ec3308
[04] rip 18ec0a65
[05] rip 18e7113b
[06] rip 03017996
[07] rip 03016b55
[08] rip 18e9c091
[09] rip 18ea3a89
[10] rip 18e9a643
[11] rip 03430c02
[12] rip 03426af3
[13] rip 03435445
[14] rip 1a472f0c
[15] rip 02942491

I don't worry about the lines "Got an invalid uplink". This is caused because of a removed NIC many weeks ago. The management network is on vmnic2.

Any ideas?

[Sreejesh_D]

hostd backtracing (crashing).

remove the entries for vmnic5_NAS02 and vmnic6_NAS03 in /etc/vmware/esx.conf and restart management agent. take the backup of esx.conf before doing it.

[RolandCH]

I removed the entries for vmnic5_NAS02 and vmnic6_NAS03 in /etc/vmware/esx.conf.

Then I restarted management agent direct on the host console several times and also via the ssh connection I used "./sbin/services.sh restart".

The Management Network is still disabled and if I try to enable it still shows up with "Enable Management Network: Error" and "Operation on the management network interface failed."

I compared the esx.conf with an esx.conf file from another host, and it looks fine.

The /var/log/vmware/hostd.log is now always empty and that worry's me.

[RolandCH]

Does anyone have any idea how to solve this problem?

Thank you for your input.

[RolandCH]

The warnings about vmnic5 and vmnic6 were not the problem at all. Even the error line "error 'FSVolumeProvider'] FSVolumeVmkImplProvider::Start() called" doesn't seem to be a real problem. I have this line on two hosts running with ESXi 4.1.0 260247.

I figured out with the command vdf, that the MAINSYS was 100% full - it was not possible to figure this out with the command df.

I deleted a overflowed logfile and then there was plenty of space. After that and following http://kb.vmware.com/kb/1018212 everything is now running again.

[rajeshkongu]

Try to restart the management network in dcui

[RolandCH]

I can perfectly ping the management network IP from my desktop. The time is between 0ms and 76ms and the the average is 4ms (tested with 100 pings by 100 answers).

I restarted the management network few times direct on the console of the host (dcui) and also with the command /sbin/services.sh via the ssh connection, nothing helped so far.

I checked the logs and could not find an idea, what's the problem causes.