SOLVED: Could not connect to one or more vCenter Server systems:

[bleuze]

I am running the vsphere 6.5 vcenter appliance. It is the only vcenter server we have. I can connect to it through the webclient, I can log in as [e-mail]. But it does not show me any inventory (it just shows a spinning wheel).

There is an error message:

Could not connect to one or more vCenter Server systems:

https://[e-mail]:443/sdk

This vcenter appliance used to work. But it has had this problem now for a few months. If need be I can just delete the appliance and re-install. There is no special configuration in there - only connection to 2 hosts. I can still use web client on each host to manage each host one at a time, I just can't manage through the vcenter appliance. I really only use the appliance to apply updates to the hosts.

Message was edited by: Bill Leuze (removed false hyperlinks)

[diegodco31]

Hi

Check it link:

VMware Knowledge Base

[bleuze]

I already looked at that KB. I tried solution for vsphere 6.x >> For Appliance.

I assume when it says "Connect to Platform Services Controller using SSH" it means connect to the vcenter appliance as that is the only vcenter controller of any sort we have in our infrastructure. But the only username\password I have that I can use to connect to it is [e-mail] and that logs me in as user "nobody". at the very fist step in the solition it says I have no permissions. Is there a default root password for this appliance that I can use for root access?

[bleuze]

I just found KB How to reset the lost or forgotten root password in vCenter Server Appliance 6.5 (2147144)

so resetting password now... then I'll try the solution KB again

[diegodco31]

During the installation the root password was requested.

[diegodco31]

Let us know if you fix your problem with root password.

[bleuze]

Root password reset successful.

As for the solution in that KB, when I run the command to output the "list of the services registered within the Platform Services Controller". I do not get duplicates like shown in the example. So I don't think I have a duplicate service endpint to unregister. Probably I need a different solution that this KB unless you see somethi9ng here. Following is the output of the "list of the services registered within the Platform Services Controller":

        Name: AboutInfo.vpx.name
        Description: AboutInfo.vpx.name
        Service Product: com.vmware.cis
        Service Type: vcenterserver
        Service ID: 4e1085a8-2bdf-45b8-9d36-75ca6d624205
        Site ID: ccv-vcenter-srv
        Node ID: 0f80f893-3f4f-4002-a9b5-36de3bb9d9da
        Owner ID: [e-mail]
        Version: 6.5
        Endpoints:
                Type: com.vmware.cis.data.provider
                Protocol: vmomi
                URL: https://ccv-vCenter-srv.colonial.local:443/sdk
                SSL trust: MIID2TCCAsGgAwIBAgIJANy7Zpl+SRBfMA0GCSqGSIb3DQEBCwUAMIGlMQswCQYDVQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZFgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJzAlBgNVBAoMHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDEbMBkGA1UECwwSVk13YXJlIEVuZ2luZWVyaW5nMB4XDTE4MDIyMzAwMDE0MVoXDTI4MDIxODAwMDE0MVowNjEnMCUGA1UEAwweY2N2LXZDZW50ZXItc3J2LmNvbG9uaWFsLmxvY2FsMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANAg0+lrBBWzzvAFIob9CP0jYysX3nP09DPbrMDu5+Gz615ZlJgELebqGRup4D2U8C8ub6pmISEjhxKsBt4x0+NM/qXOXgtuLAtGpzU62CRkXp8jOPCcVUuUL0SvSKbI9lmZQZ0g3jACQudQQm4/DHKhytS+S1FG0sjBl9ui3ICydfEjMPk0OVVJmvNNwZe0ceR8iTUIwBh6BPW+8XqPuJDN1Ur8AgdaK3pzrfzaRkLmYpWQlv5LvoiJdrO7Xj96V/O2X/A7mroxi2xaaNzOv2xj3OObNUdqeu2djSp3Hpi45V20P4fUHKLJ+bxmHgAhxyGEHO1jM9D3IKtYGTIkFqMCAwEAAaN6MHgwCwYDVR0PBAQDAgOoMCkGA1UdEQQiMCCCHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDAdBgNVHQ4EFgQUPbbqIUez7oCCCLYCFF7A0/MY9zkwHwYDVR0jBBgwFoAUNi+Dh0DifI0UAuEK/sf385t/AacwDQYJKoZIhvcNAQELBQADggEBAAxQTE9wzwdmhVwM6DLrAem/D0xubApm8hYq4d3vMw0gmqRsJd3LEyhx87Wjwvof5+RKFipsjI5o0IKm0IkfeJQcPS0sRf5Il2Sd5w3YAR//MUSRnnOJ6CEECAYOcjLG06oMFYxmqurygqSBKeF8ooNB5pt+8fSeLLXw0+CEZKpcQvt4olWmacaxkxe5UpLgAJu4SFacWF9p9bPDQ+xEOqVylv9/SJ8zCd7eDekRkcFBnFUp2mh+i3DPB/YrGtQ9yP6feIWhWALCaEIOOZz8t37lQxQPwoyDiEBPU06/xgoCwxGusuOtshmAdXier4WxMjyLkEVMA2DEaKMaQ/RYAlA=
                Endpoint Attributes:
                        cis.common.ep.localurl: http://localhost:8085/sdk
                Type: com.vmware.cdc.provider
                Protocol: vmomi
                URL: https://ccv-vCenter-srv.colonial.local:443/sdk
                SSL trust: MIID2TCCAsGgAwIBAgIJANy7Zpl+SRBfMA0GCSqGSIb3DQEBCwUAMIGlMQswCQYDVQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZFgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJzAlBgNVBAoMHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDEbMBkGA1UECwwSVk13YXJlIEVuZ2luZWVyaW5nMB4XDTE4MDIyMzAwMDE0MVoXDTI4MDIxODAwMDE0MVowNjEnMCUGA1UEAwweY2N2LXZDZW50ZXItc3J2LmNvbG9uaWFsLmxvY2FsMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANAg0+lrBBWzzvAFIob9CP0jYysX3nP09DPbrMDu5+Gz615ZlJgELebqGRup4D2U8C8ub6pmISEjhxKsBt4x0+NM/qXOXgtuLAtGpzU62CRkXp8jOPCcVUuUL0SvSKbI9lmZQZ0g3jACQudQQm4/DHKhytS+S1FG0sjBl9ui3ICydfEjMPk0OVVJmvNNwZe0ceR8iTUIwBh6BPW+8XqPuJDN1Ur8AgdaK3pzrfzaRkLmYpWQlv5LvoiJdrO7Xj96V/O2X/A7mroxi2xaaNzOv2xj3OObNUdqeu2djSp3Hpi45V20P4fUHKLJ+bxmHgAhxyGEHO1jM9D3IKtYGTIkFqMCAwEAAaN6MHgwCwYDVR0PBAQDAgOoMCkGA1UdEQQiMCCCHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDAdBgNVHQ4EFgQUPbbqIUez7oCCCLYCFF7A0/MY9zkwHwYDVR0jBBgwFoAUNi+Dh0DifI0UAuEK/sf385t/AacwDQYJKoZIhvcNAQELBQADggEBAAxQTE9wzwdmhVwM6DLrAem/D0xubApm8hYq4d3vMw0gmqRsJd3LEyhx87Wjwvof5+RKFipsjI5o0IKm0IkfeJQcPS0sRf5Il2Sd5w3YAR//MUSRnnOJ6CEECAYOcjLG06oMFYxmqurygqSBKeF8ooNB5pt+8fSeLLXw0+CEZKpcQvt4olWmacaxkxe5UpLgAJu4SFacWF9p9bPDQ+xEOqVylv9/SJ8zCd7eDekRkcFBnFUp2mh+i3DPB/YrGtQ9yP6feIWhWALCaEIOOZz8t37lQxQPwoyDiEBPU06/xgoCwxGusuOtshmAdXier4WxMjyLkEVMA2DEaKMaQ/RYAlA=
                Endpoint Attributes:
                        cis.common.ep.localurl: http://localhost:8085/sdk
                Type: com.vmware.vim
                Protocol: vmomi
                URL: https://ccv-vCenter-srv.colonial.local:443/sdk
                SSL trust: MIID2TCCAsGgAwIBAgIJANy7Zpl+SRBfMA0GCSqGSIb3DQEBCwUAMIGlMQswCQYDVQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZFgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJzAlBgNVBAoMHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDEbMBkGA1UECwwSVk13YXJlIEVuZ2luZWVyaW5nMB4XDTE4MDIyMzAwMDE0MVoXDTI4MDIxODAwMDE0MVowNjEnMCUGA1UEAwweY2N2LXZDZW50ZXItc3J2LmNvbG9uaWFsLmxvY2FsMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANAg0+lrBBWzzvAFIob9CP0jYysX3nP09DPbrMDu5+Gz615ZlJgELebqGRup4D2U8C8ub6pmISEjhxKsBt4x0+NM/qXOXgtuLAtGpzU62CRkXp8jOPCcVUuUL0SvSKbI9lmZQZ0g3jACQudQQm4/DHKhytS+S1FG0sjBl9ui3ICydfEjMPk0OVVJmvNNwZe0ceR8iTUIwBh6BPW+8XqPuJDN1Ur8AgdaK3pzrfzaRkLmYpWQlv5LvoiJdrO7Xj96V/O2X/A7mroxi2xaaNzOv2xj3OObNUdqeu2djSp3Hpi45V20P4fUHKLJ+bxmHgAhxyGEHO1jM9D3IKtYGTIkFqMCAwEAAaN6MHgwCwYDVR0PBAQDAgOoMCkGA1UdEQQiMCCCHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDAdBgNVHQ4EFgQUPbbqIUez7oCCCLYCFF7A0/MY9zkwHwYDVR0jBBgwFoAUNi+Dh0DifI0UAuEK/sf385t/AacwDQYJKoZIhvcNAQELBQADggEBAAxQTE9wzwdmhVwM6DLrAem/D0xubApm8hYq4d3vMw0gmqRsJd3LEyhx87Wjwvof5+RKFipsjI5o0IKm0IkfeJQcPS0sRf5Il2Sd5w3YAR//MUSRnnOJ6CEECAYOcjLG06oMFYxmqurygqSBKeF8ooNB5pt+8fSeLLXw0+CEZKpcQvt4olWmacaxkxe5UpLgAJu4SFacWF9p9bPDQ+xEOqVylv9/SJ8zCd7eDekRkcFBnFUp2mh+i3DPB/YrGtQ9yP6feIWhWALCaEIOOZz8t37lQxQPwoyDiEBPU06/xgoCwxGusuOtshmAdXier4WxMjyLkEVMA2DEaKMaQ/RYAlA=
                Endpoint Attributes:
                        cis.common.ep.localurl: http://localhost:8085/sdk
                Type: com.vmware.cis.common.resourcebundle
                Protocol: http
                URL: https://ccv-vCenter-srv.colonial.local:443/catalog/catalog.zip
                SSL trust: MIID2TCCAsGgAwIBAgIJANy7Zpl+SRBfMA0GCSqGSIb3DQEBCwUAMIGlMQswCQYDVQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZFgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJzAlBgNVBAoMHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDEbMBkGA1UECwwSVk13YXJlIEVuZ2luZWVyaW5nMB4XDTE4MDIyMzAwMDE0MVoXDTI4MDIxODAwMDE0MVowNjEnMCUGA1UEAwweY2N2LXZDZW50ZXItc3J2LmNvbG9uaWFsLmxvY2FsMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANAg0+lrBBWzzvAFIob9CP0jYysX3nP09DPbrMDu5+Gz615ZlJgELebqGRup4D2U8C8ub6pmISEjhxKsBt4x0+NM/qXOXgtuLAtGpzU62CRkXp8jOPCcVUuUL0SvSKbI9lmZQZ0g3jACQudQQm4/DHKhytS+S1FG0sjBl9ui3ICydfEjMPk0OVVJmvNNwZe0ceR8iTUIwBh6BPW+8XqPuJDN1Ur8AgdaK3pzrfzaRkLmYpWQlv5LvoiJdrO7Xj96V/O2X/A7mroxi2xaaNzOv2xj3OObNUdqeu2djSp3Hpi45V20P4fUHKLJ+bxmHgAhxyGEHO1jM9D3IKtYGTIkFqMCAwEAAaN6MHgwCwYDVR0PBAQDAgOoMCkGA1UdEQQiMCCCHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDAdBgNVHQ4EFgQUPbbqIUez7oCCCLYCFF7A0/MY9zkwHwYDVR0jBBgwFoAUNi+Dh0DifI0UAuEK/sf385t/AacwDQYJKoZIhvcNAQELBQADggEBAAxQTE9wzwdmhVwM6DLrAem/D0xubApm8hYq4d3vMw0gmqRsJd3LEyhx87Wjwvof5+RKFipsjI5o0IKm0IkfeJQcPS0sRf5Il2Sd5w3YAR//MUSRnnOJ6CEECAYOcjLG06oMFYxmqurygqSBKeF8ooNB5pt+8fSeLLXw0+CEZKpcQvt4olWmacaxkxe5UpLgAJu4SFacWF9p9bPDQ+xEOqVylv9/SJ8zCd7eDekRkcFBnFUp2mh+i3DPB/YrGtQ9yP6feIWhWALCaEIOOZz8t37lQxQPwoyDiEBPU06/xgoCwxGusuOtshmAdXier4WxMjyLkEVMA2DEaKMaQ/RYAlA=
                Endpoint Attributes:
                        com.vmware.cis.common.resourcebundle.basename: cis.vc.action:cis.vc.alarm:cis.vc.auth:cis.vc.cluster:cis.vc.default:cis.vc.enum:cis.vc.evc:cis.vc.event:cis.vc.eventaux:cis.vc.fault:cis.vc.gos:cis.vc.host:cis.vc.locmsg:cis.vc.option:cis.vc.perf:cis.vc.question:cis.vc.stask:cis.vc.task:cis.vc.vm:cis.vc.profile:cis.vc.hostdiag.enum:cis.vc.hostdiag.eventaux:cis.vc.hostdiag.event:cis.vc.hostdiag.locmsg:cis.vc.VirtualCenter.enum:cis.vc.VirtualCenter.eventaux:cis.vc.VirtualCenter.event:cis.vc.VirtualCenter.extension:cis.vc.VirtualCenter.locmsg
                Type: com.vmware.vim.extension
                Protocol: vmomi
                URL: https://ccv-vCenter-srv.colonial.local:443/sdkTunnel
                SSL trust: MIID2TCCAsGgAwIBAgIJANy7Zpl+SRBfMA0GCSqGSIb3DQEBCwUAMIGlMQswCQYDVQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZFgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExJzAlBgNVBAoMHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDEbMBkGA1UECwwSVk13YXJlIEVuZ2luZWVyaW5nMB4XDTE4MDIyMzAwMDE0MVoXDTI4MDIxODAwMDE0MVowNjEnMCUGA1UEAwweY2N2LXZDZW50ZXItc3J2LmNvbG9uaWFsLmxvY2FsMQswCQYDVQQGEwJVUzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANAg0+lrBBWzzvAFIob9CP0jYysX3nP09DPbrMDu5+Gz615ZlJgELebqGRup4D2U8C8ub6pmISEjhxKsBt4x0+NM/qXOXgtuLAtGpzU62CRkXp8jOPCcVUuUL0SvSKbI9lmZQZ0g3jACQudQQm4/DHKhytS+S1FG0sjBl9ui3ICydfEjMPk0OVVJmvNNwZe0ceR8iTUIwBh6BPW+8XqPuJDN1Ur8AgdaK3pzrfzaRkLmYpWQlv5LvoiJdrO7Xj96V/O2X/A7mroxi2xaaNzOv2xj3OObNUdqeu2djSp3Hpi45V20P4fUHKLJ+bxmHgAhxyGEHO1jM9D3IKtYGTIkFqMCAwEAAaN6MHgwCwYDVR0PBAQDAgOoMCkGA1UdEQQiMCCCHmNjdi12Q2VudGVyLXNydi5jb2xvbmlhbC5sb2NhbDAdBgNVHQ4EFgQUPbbqIUez7oCCCLYCFF7A0/MY9zkwHwYDVR0jBBgwFoAUNi+Dh0DifI0UAuEK/sf385t/AacwDQYJKoZIhvcNAQELBQADggEBAAxQTE9wzwdmhVwM6DLrAem/D0xubApm8hYq4d3vMw0gmqRsJd3LEyhx87Wjwvof5+RKFipsjI5o0IKm0IkfeJQcPS0sRf5Il2Sd5w3YAR//MUSRnnOJ6CEECAYOcjLG06oMFYxmqurygqSBKeF8ooNB5pt+8fSeLLXw0+CEZKpcQvt4olWmacaxkxe5UpLgAJu4SFacWF9p9bPDQ+xEOqVylv9/SJ8zCd7eDekRkcFBnFUp2mh+i3DPB/YrGtQ9yP6feIWhWALCaEIOOZz8t37lQxQPwoyDiEBPU06/xgoCwxGusuOtshmAdXier4WxMjyLkEVMA2DEaKMaQ/RYAlA=
                Endpoint Attributes:
                        cis.common.ep.localurl: http://localhost:8085/sdkTunnel
        Attributes:
                com.vmware.cis.cm.GroupInternalId: com.vmware.vim.vcenter
                com.vmware.vim.vcenter.instanceName: ccv-vCenter-srv.colonial.local
                com.vmware.cis.cm.ControlScript: service-control-default-vmon
                com.vmware.cis.cm.HostId: a065d362-06f6-4fb6-9504-cab10e2d5a90

[diegodco31]

What is the build of vcsa 6.5?

[Raj1988]

That looks good. Only one registration .

Match instance.cfg from /etc/vmware-vpx/ with Service ID: 4e1085a8-2bdf-45b8-9d36-75ca6d624205 . This should be same.

Also run service-control --status --all and check if vmware-vpxd si up and running . This error can also come if vpxd service is down. If vpxd is dowsn ; please share vpxd.log .

[bleuze]

Diego:

the vcsa build is 6.5.0.5100

Ajay:

root@ccv-vCenter-srv [ /etc/vmware-vpx ]# cat instance.cfg

applicationDN=dc\=virtualcenter,dc\=vmware,dc\=int

instanceUuid=4e1085a8-2bdf-45b8-9d36-75ca6d624205

ldapPort=389

ldapInstanceName=VMwareVCMSDS

ldapStoragePath=/etc/vmware-vpx/

looks same as Service ID: 4e1085a8-2bdf-45b8-9d36-75ca6d624205 in the previously dumped psc_services.txt

root@ccv-vCenter-srv [ /etc/vmware-vpx ]# service-control --status --all

Running:

applmgmt lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-psc-client vmware-rhttpproxy vmware-sca vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmware-vpxd-svcs vmware-vsm vsphere-client vsphere-ui

Stopped:

vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmware-sps vmware-updatemgr vmware-vcha vmware-vpxd vmware-vsan-health

Hmm, vmware-vpxd is stopped. How do I start it?

[bleuze]

Hmm, I started VMware-vpxd but it still shows as stopped:

root@ccv-vCenter-srv [ /etc/rc.d ]# init.d/vmware-vpxd

root@ccv-vCenter-srv [ /etc/rc.d ]# Usage: vmware-vpxd {start|stop|status|restart|clean-embedded-db}

init.d/vmware-vpxd start

<30>Jul  9 22:29:53 vmware-vpxd: VC SSL Certificate does not exist, it will be generated by vpxd

Executing pre-startup scripts...

Traceback (most recent call last):

  File "/usr/lib/vmware-vpx/py/vccfg.py", line 493, in <module>

    sys.exit(VcvaConfigCmd().run())

  File "/usr/lib/vmware-vpx/py/vccfg.py", line 455, in run

    opMethod()

  File "/usr/lib/vmware-vpx/py/vccfg.py", line 396, in readCmd

    result = self.readValues(args)

  File "/usr/lib/vmware-vpx/py/vccfg.py", line 156, in readValues

    cfgViewsList = self.getKeyCfgViewList(configKey, readOnly=True)

  File "/usr/lib/vmware-vpx/py/vccfg.py", line 87, in getKeyCfgViewList

    self.cfgFileStores[filename] = cfgViewClass.createFileStore(filename)

  File "/usr/lib/vmware-vpx/py/shellvarcfg.py", line 53, in createFileStore

    return PlainTextFile(filename)

  File "/usr/lib/vmware-vpx/py/shellvarcfg.py", line 21, in __init__

    raise CfgException("Cannot open file: " + self.filename)

cfgexception.CfgException: Cannot open file: /usr/lib/vmware-invsvc/lib/server/config/dataservice.properties

<30>Jul  9 22:29:53 vmware-vpxd: Starting vpxd by administrative request.

success

<30>Jul  9 22:29:54 vmware-vpxd: Waiting for vpxd to start listening for requests on 8089

Waiting for vpxd to initialize: .success

<30>Jul  9 22:30:05 vmware-vpxd: vpxd has initialized.

root@ccv-vCenter-srv [ /etc/rc.d ]# service-control --status --all

Running:

applmgmt lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-psc-client vmware-rhttpproxy vmware-sca vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmware-vpxd-svcs vmware-vsm vsphere-client vsphere-ui

Stopped:

vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmware-sps vmware-updatemgr vmware-vcha vmware-vpxd vmware-vsan-health

[diegodco31]

Please, share the file /var/log/vmware/vpx/vpxd.log.

[SUMoses]

Just restart all the servers in the following sequence ,  1,  passive 2, witness 3,vcsa ,     then see that all your services are started , then log in again ,

[bleuze]

sorry about the big paste above. lets see if I can do a file attach...

Yes, log attached. So the above long pasted message is now deleted

[diegodco31]

Check the following:

VCSA 6.5 Web GUI Error

[Raj1988]

bleuz​

vpxd.log

2018-07-09T22:30:25.448Z error vpxd[7F4FACAC3700] [Originator@6876 sub=Default opID=HostSync-host-28-37dc5435] [VdbStatement] SQLError was thrown: "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";

--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"

2018-07-09T22:30:25.461Z error vpxd[7F4FACAC3700] [Originator@6876 sub=vpxCommon opID=HostSync-host-28-37dc5435] [Vpxd_HandleVmRootError] Received unrecoverable VmRootError. Generating minidump ...

2018-07-09T22:30:25.461Z error vpxd[7F4FACAC3700] [Originator@6876 sub=Default opID=HostSync-host-28-37dc5435] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device";

--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_VM_VIRTUAL_DEVICE (ID, DEVICE_KEY, CONN_AL_GU_CONTROL_FLAG, CONN_CONNECTED_FLAG, CONN_START_CONNECTED_FLAG, CONN_STATUS, DEVICE_INFO_SUMMARY, DEVICE_INFO_LABEL, DEVICE_CONTROLLER_KEY, DEVICE_UNIT_NUMBER, DEVICE_TYPE, DEVICE_TEXT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"

2018-07-09T22:30:25.466Z panic vpxd[7F4FACAC3700] [Originator@6876 sub=Default opID=HostSync-host-28-37dc5435]

-->

--> Panic: Unrecoverable VmRootError. Panic!

Thats the crash .

You need to see the postgres.log during the same time frame for the exact duplicate key ...

Example:-

postgresql.log

UTC 583a9c81.5814 127750 VCDB vc ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device"
UTC 583a9c81.5814 127750 VCDB vc DETAIL: Key (id, device_key)=(361, 3002) already exists.

Take a snapshot . Connect to VCSA DB ..

/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

Delete the duplicate ID using the below command. Replace with the ID values you get from postgres.log

DELETE FROM vpx_vm_virtual_device where id='xxx' and device_key='xxxx';

[bleuze]

Thanks Ajay

I will look into this tomorrow. I am heading out the door now

[diegodco31]

Hi

Let us know if you fix your problem.

Remember to take the snapshot vCenter.

Step to step:

Could not connect to one or more vCenter Server systems | DesertPenguin.org | Andrew J. Brehm

[bleuze]

SOLVED:

OK, In postgresql-09.log I found:

2018-07-09 22:30:25.447 UTC 5b43e1fa.9ad9 1964284 VCDB vc ERROR:  duplicate key value violates unique constraint "pk_vpx_vm_virtual_device"

2018-07-09 22:30:25.447 UTC 5b43e1fa.9ad9 1964284 VCDB vc DETAIL:  Key (id, device_key)=(37, 3002) already exists.

Actually I found 6 more instances of this message all with the same id, device key pair. But this is the one at the exact same timestamp as the Ajay identified in the vpxd.log

So, I made a snapshot, then:

root@ccv-vCenter-srv [ /var/log/vmware/vpostgres ]# /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres
psql.bin (9.4.9 (VMware Postgres 9.4.9.1-4536683 release))
Type "help" for help.

VCDB=# DELETE FROM vpx_vm_virtual_device where id='37' and device_key='3002';
DELETE 1
VCDB=# \q

problem still exists, so, rebooting the vcenter appliance...

et viola, that fixed it. All inventory now displays in the webgui. Thanks so much Ajay!!!