Upgrade pre-check Health check fails to retrieve data about service on vSphere ESX Agent Manager

[MCU_Techsupport]

When I run a pre-check on a host before upgrading the ESXI version, I receive the following error:
"Health check fails to retrieve data about service 'swcluster' on 'vSphere ESX Agent Manager'. Verify that the service 'swcluster' is running and try again."


In the eam.log I get the following various erros:
-Full stack trace: com.vmware.eam.EamRemoteSystemException: Unexpected error communicating with the vCenter server.
-Internal server error during dispatch
com.vmware.vim.binding.eam.fault.EamServiceNotInitialized: EAM is still loading from database. Please try again later.
-An unexpected error in the changes polling loop
com.vmware.eam.EamRemoteSystemException: Unexpected error communicating with the vCenter server.
-Caused by: com.vmware.vim.binding.vim.fault.NotAuthenticated: The session is not authenticated.



I have tried the following:
-Reconfigure the hosts for HA
-Restart vCenter appliance
-Restart service called "VMware ESX Agent Manager" on vCenter server

I find nothing remotely close to this issue when I Google and Bing for solutions.

Edit: Found this log entry in the vpxd.log:
2022-12-06T03:34:41.365Z info vpxd[06306] [Originator@6876 sub=Default opID=3150bf0] [VpxLRO] -- ERROR lro-36104 -- SessionManager -- vim.SessionManager.loginExtensionByCertificate: vim.fault.InvalidLogin:
--> Result:
--> (vim.fault.InvalidLogin) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = ""
--> }
--> Args:
-->
--> Arg extensionKey:
--> "com.vmware.vim.eam"
--> Arg locale:
-->

[MCU_Techsupport]

Never mind, I had to re-install from scratch. VMware support was of no help. That had no clue where to go.

[TechnoNotice666]

Hi, 

What did you reinstall?  

We have the same issue and have reinstalled the ESXi host.  

[MCU_Techsupport]

I reinstalled vcenter, something with the installation must have been corrupt, something with the files. I tried deleting the cluster and re-adding it to the old vcenter, but it gave the same error.

One caveat with the reinstall, I did not set an image as a baseline this time, I just used the default baselines. I doubt that was the issue, I would assume it still needs to contact the health agent no matter what baseline or policy it is checking against.

[TechnoNotice666]

Hi, 

VMWare, after analysing the logs, found a slightly different message which is caused by a mismatch in the certificates.  

2022-12-13T23:39:23.865Z | ERROR | vim-monitor | VcListener.java | 124 | An unexpected error in the changes polling loop
com.vmware.eam.EamRemoteSystemException: Unexpected error communicating with the vCenter server.
	at
..
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_345]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_345]
Caused by: com.vmware.vim.binding.vim.fault.NotAuthenticated: The session is not authenticated.

https://kb.vmware.com/s/article/2112577?lang=en_US

Running the commands to update the eam extension solved the Heathcheck failed error. 

[flobaumann]

 

Hi,

thanks for the solution. We were about to reinstall the vCenter, then I found your reply and it worked well for us!

Merry Christmas 

[Chris asdf]

This KB article solved the problem for us after a recent issue with enhanced linked mode. Thanks!

[washed_boogie]

good looks on that solution, def helped avoid a vcenter reinstall!

[Seirui]

This worked for me as well. I was getting this error while trying to remediate a server. Last week my STS certificate expired on me and I had to get support to help me regenerate it. It looks like this broke during the process as well, and this fixed it. Thanks!

[Nick_Fletcher]

This worked for me too. I have just updated all the certificates on the appliance, I guess one of them did not work as expected. All fixed now though. Thanks

[Najeeb Hajji]

Hi   Did you just follow the KB steps??

the KB only update EAM extension certificate.

can you please put some details or steps.

Thanks

[Nick_Fletcher]

More or less, yes.

I am using VCSA 8 so I used the steps in the second half of the document.

I did not run  shell.set --enabled true.

Everything else was run exactly as described in the document.

In step 5 make sure you add the correct FQDN of your vCenter in place of <PNID/FQDN of vCenter Server>. (For example vcenter.domain.com)

Good luck

[James Dorton]

Can confirm that this KB also resolved my issues.

[Najeeb Hajji]

Reinstall is not an option, there are other components depending on current vCenter configuration: VMWare Horizon and vSphere Replication.

 

is there any Other solution ???

[flobaumann]

Look at the solution from TechnoNotice666. This worked well for us.

[Najeeb Hajji]

Already did it, it doesn't work