vCenter 7.0.0c "vAPI Endpoint" service warning

[Maurizio71]

After upgrade vCenter from version 7.0.0b to version 7.0.0c I got this warning on the service monitor:

Failed to connect to 7ae7cbaa-9164-47f7-a9f8-1d1da7e2746b\com.vmware.vcenter.nsxd.vapi vAPI provider.

Current vApi Endpoint health status is created between 2020-08-24T16:21:29UTC and 2020-08-24T16:21:30UTC.

I've these lines in the vapi/endpoint/endpoint.log:

2020-08-24T15:33:00.213Z | WARN  | state-manager1            | ApiInterfacesFactory           | Retrieving interfaces for service 7ae7cbaa-9164-47f7-a9f8-1d1da7e2746b\com.vmware.vcenter.nsxd.vapi has failed.

2020-08-24T15:33:00.214Z | WARN  | state-manager1            | ApiInterfacesFactory           | The cache does not contain entry for service 7ae7cbaa-9164-47f7-a9f8-1d1da7e2746b\com.vmware.vcenter.nsxd.vapi. The interfaces will be skipped.

2020-08-24T15:33:00.214Z | INFO  | state-manager1            | HealthStatusCollectorImpl      | HEALTH YELLOW Failed to connect to 7ae7cbaa-9164-47f7-a9f8-1d1da7e2746b\com.vmware.vcenter.nsxd.vapi vAPI provider.

Thanks

Maurizio

[mattgyver]

Did restarting the vAPI Endpoint Service help it at all?

[Maurizio71]

The service restarts correctly, but after some minutes the warning appears again.

[mattgyver]

Both my vSphere 7 clusters are on 7.0.0.c and both have this issue. I'm running the appliance-based vCenter 7.0.0.c, not the windows version. None of the previous versions fixes are helping.

[Maurizio71]

I just re-installed from scratch, a new appliance-based VM with vCenter 7.0.0c (7.0.0.10600 build 16620007) and this issue is here again, so I think that it's a bug of this version.

[ThoDor]

Hello,

I got the same issue.

In my case I did a new installation of vCenter 7c (Build 16620013) not upgrade.

Tried the "fix" for 6.5 (VMware Knowledge Base ) didn't work....

2020-08-25T07:37:07.677+02:00 | WARN  | state-manager1            | ApiInterfacesFactory           | The cache does not contain entry for service de7852f0-2ce4-40e7-b058-475f15dabdfe\com.vmware.vcenter.nsxd.vapi. The interfaces will be skipped.

2020-08-25T07:37:07.677+02:00 | INFO  | state-manager1            | HealthStatusCollectorImpl      | HEALTH YELLOW Failed to connect to de7852f0-2ce4-40e7-b058-475f15dabdfe\com.vmware.vcenter.nsxd.vapi vAPI provider.

2020-08-25T07:37:08.139+02:00 | INFO  | state-manager1            | HealthConfigurationEventListener | HEALTH Failed to connect to de7852f0-2ce4-40e7-b058-475f15dabdfe\com.vmware.vcenter.nsxd.vapi vAPI provider.

2020-08-25T07:37:08.139+02:00 | INFO  | state-manager1            | HealthConfigurationEventListener | HEALTH Current vApi Endpoint health status is created between 2020-08-25T07:37:07CEST and 2020-08-25T07:37:08CEST.

[alokpaul]

I am also observing exactly the same issue after upgrading vCenter server to 7.0c.

[Maurizio71]

vCenter 7.0.0d is out, I'll try if the upgrade solves the problem:

VMware-VCSA-all-7.0.0-16749653.iso

[nachogonzalez]

Last year we faced the same issue on VCSA 6.5, the problem was we were unable to upgrade due to bureaucracy and we were getting tons of tickets a day (and unable to delete them, also due to bureaucracy)

I figured out that if i ran this api call agains the VCSA i could know if VAPI endpoint was working or not.

GET https://{server}/rest/appliance/health/applmgmt

If you get any response from the api (not error 404 or 503) the vapi endpoint was working.

Just a manual way to test if the service was working

[Maurizio71]

The VAPI endpoint is working, I got this response:

{"value":"green"}

[mattgyver]

Is that after you tried 7.0d? Or on 7.0c yet?

[Maurizio71]

It's on the 7.0c.

[nachogonzalez]

I'm pretty sure this might be a bug.
Do you have the ability to create a SR?

[Maurizio71]

I don't think I can create a SR.

[mattgyver]

I created an SR a little bit ago. If I hear back from VMWare Support, I'll say something here.

[nachogonzalez]

Thanks!

[ThoDor]

Well yesterday I tried to "Update" my VCSA 7.0.0c with 7.0.0d UI-Installer.

Got the Message, that the installed Version can't be updated to 7.0.0d.

Since I didn't wanted to reinstall completly I waited.

This morning I got the notification, that I can update the VCSA with build in Update function.

Because my environment isn't productiv I gave it a try.

It took pretty long downtime (50MBit/s WAN Link) isn't the best ...

Likewise 45 Minutes.

The Update was successfully:

Produkt:

VMware vCenter Server Appliance

Version:

7.0.0.10700

Build-Nummer

16749653

After the Update I still had the Error with vAPI Endpoint Service.

I used:

service-control --stop vmware-content-library;service-control --start vmware-content-library

And restarted vAPI-Endpoint too, after some minutes I got the same Error again.

I mean the patch notes told only to fix high CPU load, not our issue.

mattroslevich​ did you got an answer already?

[mariss35]

i didnt had this error with 7.0.0c , but it came up after upgrade to 7.0.0d on 2 separate deployments.

Failed to connect to xxxxxxxxxxxxxxxxxxxxxx\com.vmware.vcenter.nsxd.vapi vAPI provider.

Current vApi Endpoint health status is created between xxxxxxxxxUTC and xxxxxxxxxUTC.

p.s. it does actually looks like VMware Knowledge Base

[nachogonzalez]

scott28tt​ people here are suspecting about a bug in a feature.
Do you know whom can we reach in the forum?

[Scott Vessey]

While I am a moderator and an employee I am not a ”gateway” to either the support or engineering teams.

[mattgyver]

No, I didn't get a response yet. Though, it does look like what mariss35​ said in their post:

i didnt had this error with 7.0.0c , but it came up after upgrade to 7.0.0d on 2 separate deployments.

Failed to connect to xxxxxxxxxxxxxxxxxxxxxx\com.vmware.vcenter.nsxd.vapi vAPI provider.

Current vApi Endpoint health status is created between xxxxxxxxxUTC and xxxxxxxxxUTC.

p.s. it does actually looks like VMware Knowledge Base

Hopefully it is something ignorable, but one would think it'd be something they want to squash quickly...

Once I hear from VMware, I'll let you all know.

[mattgyver]

Got a response from VMware Support..

Hello Matthew,

Greetings!

I am Shweta Sharma and will be assisting you with this SR.

Yes, this is a known issue in vCenter 7.0.x versions.

Please check this KB article as it states the same-

https://kb.vmware.com/s/article/80409

This is into out notice and our engineering team is working upon it to resolve it in future releases.
Also, as of now, you can ignore this warning as it does not have any impact in the system.

There you have it - definitely a known issue that can be ignored for now. They're working on it. mariss35​ hit the nail on the head!

[Tyler Jurgens]

Has anyone tried vCenter 7.0 U1?  The KB linked looks accurate for the symptoms I am seeing and the logs I can correlate.

[mdiorio1]

VAPI endpoint is still erroring in the appliance manager in U1:  It's now "Healthy with warnings"

Failed to connect to ad332723-9a14-41b6-820c-9af189783ed5\com.vmware.vcenter.nsxd.vapi vAPI provider.

Configuration health status is created between 2020-11-27T15:33:06UTC and 2020-11-27T15:35:11UTC.

[GlebA]

I have faced the same issue soon after upgrade to 7.0U1c. It was fine on a previous version.

https://{vcenter}/vapiendpoint/health shows YELLOW status

https://{vcenter}/rest/appliance/health/applmgmt shows {"value":"green"}

Anyone managed to find a solution? Any information from vmWare?

[mdiorio1]

Gotta say - my vCenter completely blew up on me with no way to recover it.  The backups were corrupted too somehow, even though they showed as valid in the VSA.  So I rebuilt the whole thing from scratch using 7.0u1 ISO and everything is completely healthy this time.  (No fun rebuilding the distributed switches, and having to deal with the management network being on a vDS.  Had to do some manipulation, but it all came up fine.  I was surprised how easy it was bringing vSAN back up and health.

I have a feeling the vAPI issue was introduced in an early build and doesn't resolve itself after upgrades. 

[RichardVisage]

I have had nothing but one issue after another with my WVware farm ever since updating to 7.0. Updates or patches have not fixed this issue for me. It still persists.

 

Booooooo VMWare, booooooo!

[nrose]

Guess who the genius was that thought he was going to "casually" slam out a vCenter upgrade from 6.7 to 7.0 today and likely finish up with enough time to leave early for the holiday weekend...?

(vCenter 7.0 Update 1 and experiencing the same issue despite KB80409 citing it's resolved in 7.0U1.)

....and this is literally my (5th? 6th?) problem I've had to troubleshoot today from this upgrade. I've still gotta figure out why after every reboot I have to manually start vmware-vpxd as it doesn't start on it's own. Ya know, NOTHING IMPORTANT. Ugh