Host Message Infrastructure Event Messages for NSX

[elihuj]

Recently I noticed several "Critical" messages in the Events manager for NSX. Here is the event:

Severity: Critical

Event Source: Host messaging infrastructure

Code: 391002

Event Message: Messaging infrastructure down on host.

Module: Messaging infrastructure.

Object Name: host-16

Now here's the thing.. there is no host-16 within my vCenter. There was a host-16, which was removed a few months ago. I was only able to find it after checking an older RVTools report. What can I do to remove prevent this from continuing to log?

[RaymundoEC]

‌hi there,

please check this link

http://www.douglassmith.me/2011/01/manually-removing-a-host-from-inventory/

hope this helps

[elihuj]

Hey Ray, thank you for the reply. I actually started looking around in the Postgres database yesterday with no luck. When I check the vpx_entity table and look for my host by name, it is not there. Similarly, if I view vpx_host_node table I do not see any hosts with an ID of 16. I know NSX leverages the vCenter database to store its information. Where else could this be coming from?

[RaymundoEC]

hey Elihuj,

Assuming you have VCSA the instructions I posted are for sql DB for vCenter DB, in the case of postgres database for VCSA (not NSX Postgres DB) it uses same schema standard tables like vpx_entity, vpx_host, son on so forth, than sql DB, there are a couple of unsuported ways to query this DB you may check this one:

VCSA (vCenter Server Appliance) 5.1 VCDB & SSODB Password | virtuallyGhetto

check there for the orphaned host.

hope this helps.

regards.

[RaymundoEC]

Just forgot to tell you to check MOB for the host ID in vCenter Server:

http://vCenter-IP-Address/mob

where vCenter-IP is the ip address of your vCenter Server

please refer to this document on page 471

http://pubs.vmware.com/NSX-62/topic/com.vmware.ICbase/PDF/nsx_62_api.pdf

regards

[elihuj]

Thank you again for the reply Ray. I checked out all of the vpx_host* tables that I could find, but still unfortunately there is no reference to host-16! I also checked out the mob browser as you suggested. I can view all of the hosts in my clusters.. but of course, still no host-16. I'm thinking about opening a case with VMware support. Strange stuff.

[RaymundoEC]

I always recommend to open a case but is good to know about a little piece of NSX, please keep me posted on how they solved it.

regards.

[elihuj]

Hey Ray, I just wanted to update you on the issue. I opened a ticket with VMware support, and right away he thought it was due to a stale host object in the database due to a host being removed from vCenter prior to uninstalling NSX. We checked out the vCenter database, as well as the MOB browser and confirmed that the host did not exist. Also, interesting to note that NSX Manager also has its own database separate from vCenter. You have to drop into root mode on the manager via the CLI with a special password that the support engineer had. He was fairly sure that there we would find the object here.. but we didn't.

After reviewing the log files, it turns out that his original hypothesis was true in that the errors were due to a host being removed prior to uninstalling NSX. He said that this is a bug in 6.2.0, which has been addressed in 6.2.1. The release notes shows:

NSX logging heartbeat failures for hosts that are not part of the NSX installation
When an NSX-prepared host is directly removed from the vCenter inventory (without first unpreparing it in NSX), NSX receives an unexpected 'Host Connected' DCN which causes partial removal of messaging infrastructure components from the host. As a result, the messaging link between NSX and the host may remain active when it should have been removed, and NSX may raise false 'Alert' SystemEvents for the host. This has been fixed in NSX 6.2.1.

He said the errors are purely cosmetic, so no need to immediately upgrade. I like things to be nice and neat though, so I'll be looking into getting the upgrade completed soon.

[RaymundoEC]

Thanks for the info and the update!!!

regards.