vCenter Problem - vpxd wont start

[nidla5]

Hi,

we have following VCSA environment:

Type: vCenter Server with an embedded Platform Services Controller

Product: VMware vCenter Server Appliance

Version: 6.5.0.14000

recentlly our Postgresql DB grows to 90+% and I have looked how to shrink the database and found following KB: VMware Knowledge Base 

I have stoped vpxd, run the script and after that vpxd wont start anymore...

vCenter appliance managemnt shows no error, and I am able to login to vCenter but Inventory is empty and in Flash client I have this error:

Could not connect to one or more vCenter Server systems:

https://fqdn:443/sdk

here are also errors from vpxd.log file:

/var/log/vmware/vpxd/vpxd.log | grep -i error

2018-07-11T07:54:42.716+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoOptionMgr] Unable to read from '/etc/motd':N7Vmacore23FileIONotFoundExceptionE(FileIO error: Could not find file  : /etc/motd)

2018-07-11T07:54:42.743+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoOptionMgr] [OptionMgr] Ignoring unknown entry from DB: alarms.upgraded

2018-07-11T07:54:42.744+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoOptionMgr] [OptionMgr] Ignoring unknown entry from DB: VirtualCenter.LDAPAdminPrincipal

2018-07-11T07:54:42.744+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoOptionMgr] [OptionMgr] Ignoring unknown entry from DB: VirtualCenter.VimWebServicesUrl

2018-07-11T07:54:42.744+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoOptionMgr] [OptionMgr] Skipping bad entry config.vpxd.enableDebugBrowse from DB. Resetting to default.Exception: vmodl.fault.InvalidArgument

2018-07-11T07:54:56.973+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-46] (moId:host-46) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.974+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-7501] (moId:host-7501) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.977+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-13729] (moId:host-13729) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.979+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-982] (moId:host-982) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.981+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-13800] (moId:host-13800) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.983+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-6810] (moId:host-6810) nic:NetworkInfo.vnic[1].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.986+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-17706] (moId:host-17706) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.992+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-21058] (moId:host-21058) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:56.996+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.119] conflict between host [host-10] (moId:host-10) nic:NetworkInfo.vnic[2].ipAddress and entity [host-11] (moId:host-11)

2018-07-11T07:54:56.997+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-845] (moId:host-845) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:57.003+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-13820] (moId:host-13820) nic:NetworkInfo.vnic[1].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:57.005+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-17509] (moId:host-17509) nic:NetworkInfo.vnic[1].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:54:57.009+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=InvtId] [VpxdInvtId::CheckIp] detected IP [IMM_Network0:169.254.95.120] conflict between host [host-981] (moId:host-981) nic:NetworkInfo.vnic[2].ipAddress and entity [host-10749] (moId:host-10749)

2018-07-11T07:55:02.614+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=profileUtil] [DeserializeFromFile] reading failed: FileIO error: Could not find file  : /var/lib/vmware/hpMetadataCache.xml

2018-07-11T07:55:05.477+02:00 error vpxd[7FA97E3C7700] [Originator@6876 sub=vmomi.soapStub[5]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T07:55:05.665+02:00 error vpxd[7FA97F66C700] [Originator@6876 sub=vmomi.soapStub[7]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT 4;

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 291 ms

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is 58P01:0:7:ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory;

--> Error while executing the query

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] Connection diagnostic data from driver is 58P01:0:110:ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] Bind parameters:

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] value = 76357270

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] [1]datatype: 1, size: 4, arraySize: 0

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] value = 76357091

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] [2]datatype: 1, size: 4, arraySize: 0

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] value = 76357274

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] [3]datatype: 1, size: 4, arraySize: 0

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] value = 102133024

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from 58P01:7

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from 58P01:110

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T07:55:06.172+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoEvent] [EventManagerMo] Failed to load events: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T07:55:06.173+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=ConfigIssue] [VpxdManagedEntity::Load] Processing of persisted config issues failed: vim.fault.DatabaseError

2018-07-11T07:55:06.173+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Main] Init failed. SystemError: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)

2018-07-11T07:55:06.174+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] Failed to intialize VMware VirtualCenter. Shutting down

any help is welcome :smileyhappy:

[diegodco31]

vpxd.log

2018-07-11T07:55:06.113+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T07:55:06.172+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=MoEvent] [EventManagerMo] Failed to load events: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12321449": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T07:55:06.173+02:00 error vpxd[7FA9930A5800] [Originator@6876 sub=ConfigIssue] [VpxdManagedEntity::Load] Processing of persisted config issues failed: vim.fault.DatabaseError

Thats the crash .

You need to see the postgres.log during the same time frame for the exact duplicate key.

Example:-

postgresql.log

UTC 583a9c81.5814 127750 VCDB vc ERROR: duplicate key value violates unique constraint "pk_vpx_vm_virtual_device"

UTC 583a9c81.5814 127750 VCDB vc DETAIL: Key (id, device_key)=(361, 3002) already exists.

Take a snapshot . Connect to VCSA DB ..

/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

Delete the duplicate ID using the below command. Replace with the ID values you get from postgres.log

DELETE FROM vpx_vm_virtual_device where id='xxx' and device_key='xxxx';

Check this link:

Could not connect to one or more vCenter Server systems | DesertPenguin.org | Andrew J. Brehm

[nidla5]

Hi Diego,

thank you for your prompt reply.

i don't have any duplicate key in postgresql.log or in vpxd.log

in postgresql.log I have only a lot of these "could not open file.." errors

2018-07-10 07:54:57.514 UTC 5b435020.40de 0   LOG:  Updating instance status...

2018-07-10 07:54:57.515 UTC 5b435020.40de 0   LOG:  Memory check: flag = rss, mem_used = 24036 kB, mem_avail = 2066432 kB

2018-07-10 07:54:57.515 UTC 5b435020.40de 0   LOG:  Writing instance status...

2018-07-10 07:54:57.515 UTC 5b435020.40de 0   LOG:  Wrote instance status successfully.

2018-07-10 07:54:57.515 UTC 5b435020.40de 0   LOG:  Updated instance status successfully.

2018-07-10 07:54:59.316 UTC 5b446649.8e4 273609119   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171780": No such file or directory

2018-07-10 07:54:59.316 UTC 5b446649.8e4 273609119   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_arg_50"

2018-07-10 07:55:07.391 UTC 5b446658.931 273609120   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171759": No such file or directory

2018-07-10 07:55:07.391 UTC 5b446658.931 273609120   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_50"

2018-07-10 07:55:10.449 UTC 5b446658.931 273609121   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15228308": No such file or directory

2018-07-10 07:55:10.449 UTC 5b446658.931 273609121   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_51"

2018-07-10 07:55:14.305 UTC 5b446658.931 273609122   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171780": No such file or directory

2018-07-10 07:55:14.305 UTC 5b446658.931 273609122   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_arg_50"

2018-07-10 07:55:22.470 UTC 5b446667.97e 273609123   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171759": No such file or directory

2018-07-10 07:55:22.470 UTC 5b446667.97e 273609123   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_50"

2018-07-10 07:55:25.418 UTC 5b446667.97e 273609124   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15228308": No such file or directory

2018-07-10 07:55:25.418 UTC 5b446667.97e 273609124   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_51"

2018-07-10 07:55:29.241 UTC 5b446667.97e 273609125   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171780": No such file or directory

2018-07-10 07:55:29.241 UTC 5b446667.97e 273609125   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_arg_50"

2018-07-10 07:55:37.371 UTC 5b446676.9c5 273609126   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171759": No such file or directory

2018-07-10 07:55:37.371 UTC 5b446676.9c5 273609126   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_50"

2018-07-10 07:55:40.325 UTC 5b446676.9c5 273609127   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15228308": No such file or directory

2018-07-10 07:55:40.325 UTC 5b446676.9c5 273609127   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_51"

2018-07-10 07:55:44.130 UTC 5b446676.9c5 273609128   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171780": No such file or directory

2018-07-10 07:55:44.130 UTC 5b446676.9c5 273609128   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_arg_50"

2018-07-10 07:55:52.464 UTC 5b446685.a4b 273609129   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15171759": No such file or directory

2018-07-10 07:55:52.464 UTC 5b446685.a4b 273609129   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_50"

2018-07-10 07:55:55.385 UTC 5b446685.a4b 273609130   ERROR:  could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/15228308": No such file or directory

2018-07-10 07:55:55.385 UTC 5b446685.a4b 273609130   CONTEXT:  automatic analyze of table "VCDB.vc.vpx_event_51"

[diegodco31]

See this link:

VMware Knowledge Base

Remember to take snapshot vCenter.

[nidla5]

i have recreated files in /storage/seat/vpostgres/eventtblsp/PG_9.4_201409291/16385

postgresql seams to be fine now

2018-07-11 14:15:34.229 UTC 5b460f25.a48 0   LOG:  Memory check: flag = rss, mem_used = 14336 kB, mem_avail = 2066432 kB

2018-07-11 14:15:34.229 UTC 5b460f25.a48 0   LOG:  Writing instance status...

2018-07-11 14:15:34.229 UTC 5b460f25.a48 0   LOG:  Wrote instance status successfully.

2018-07-11 14:15:34.229 UTC 5b460f25.a48 0   LOG:  Updated instance status successfully.

but vmxd still could not be started:

2018-07-11T16:08:59.755+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=Locale] LoadResources path (/etc/vmware-vpx/extensions/com.vmware.vsan.health/locale/) locale (fr) removeable (true) resMap (00007fd65c025820) extKeys (00007fd65c06ec80)

2018-07-11T16:09:00.095+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=VpxProfiler] Init [AlarmMoManager::Init()] took 1560 ms

2018-07-11T16:09:00.170+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=vpxUtil] [VpxVmomi] Found previous domain socket /var/run/vmware/vpxd-webserver-pipe. Removing...

2018-07-11T16:09:00.170+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for nonChunkingAgents

2018-07-11T16:09:00.170+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for agentsNeedingContentLength

2018-07-11T16:09:00.170+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=HttpSvc.HTTPService] Max buffered response size is 104857600 bytes

2018-07-11T16:09:00.269+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.stats.report

2018-07-11T16:09:00.270+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.vcha

2018-07-11T16:09:00.311+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension hostdiag

2018-07-11T16:09:00.775+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoDiagnosticMgr] [DiagnosticManagerMo] Running support script located at /bin/vc-support.sh

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.persist

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.level

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2018-07-11T16:09:00.996+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.duration

2018-07-11T16:09:01.113+02:00 error vpxd[7FD66E4AF700] [Originator@6876 sub=vmomi.soapStub[5]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T16:09:01.113+02:00 warning vpxd[7FD66E4AF700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-07-11T16:09:01.163+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=licenseServiceDiscovery] Discovered license service endpoint URL: https://VMM.scidom.de:443/ls/sdk

2018-07-11T16:09:01.163+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=licenseServiceProxy] Using ls URL: https://VMM.scidom.de:443/ls/sdk

2018-07-11T16:09:01.263+02:00 error vpxd[7FD66D897700] [Originator@6876 sub=vmomi.soapStub[7]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T16:09:01.264+02:00 warning vpxd[7FD66D897700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-07-11T16:09:01.308+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=componentManagerUtil] Discovered VC service folder ID: d5b33cf1-3fde-11e3-9ca0-000c299d97e3

2018-07-11T16:09:01.328+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=Default] Creating SSL Contexts

2018-07-11T16:09:01.328+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Command invoked: /sbin/pidof

2018-07-11T16:09:01.381+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoReverseProxy] Finished executing command: /sbin/pidof

2018-07-11T16:09:01.381+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Pid of reverse proxy is: 2284

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT 4;

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 265 ms

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is XX002:0:7:ERROR: index "pk_vpx_event_4" contains unexpected zero page at block 0;

--> Error while executing the query

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] Connection diagnostic data from driver is XX002:0:110:ERROR: index "pk_vpx_event_4" contains unexpected zero page at block 0

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] Bind parameters:

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] value = 76357270

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] [1]datatype: 1, size: 4, arraySize: 0

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] value = 76357091

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] [2]datatype: 1, size: 4, arraySize: 0

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] value = 76357274

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] [3]datatype: 1, size: 4, arraySize: 0

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] value = 102133024

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from XX002:7

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from XX002:110

2018-07-11T16:09:01.674+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (XX002) - ERROR: index "pk_vpx_event_4" contains unexpected zero page at block 0;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T16:09:01.784+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=MoEvent] [EventManagerMo] Failed to load events: "ODBC error: (XX002) - ERROR: index "pk_vpx_event_4" contains unexpected zero page at block 0;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T16:09:01.784+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=ConfigIssue] [VpxdManagedEntity::Load] Processing of persisted config issues failed: vim.fault.DatabaseError

2018-07-11T16:09:01.784+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Main] Init failed. SystemError: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)

--> [context]zKq7AVECAAAAAIStcgAKdnB4ZAAAeF4rbGlidm1hY29yZS5zbwAAEBcbAMppGAEO1FR2cHhkAAEj1lQBsllVAb37VAFqn1MC4AUCbGliYy5zby42AAF1l1M=[/context]

2018-07-11T16:09:01.785+02:00 warning vpxd[7FD67D7EA800] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 25838 ms

2018-07-11T16:09:01.785+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=Default] Failed to intialize VMware VirtualCenter. Shutting down

2018-07-11T16:09:01.785+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=SupportMgr] Wrote uptime information

2018-07-11T16:11:01.429+02:00 info vpxd[7FD67D7EA800] [Originator@6876 sub=Default] Forcing shutdown of VMware VirtualCenter now

[diegodco31]

Also run service-control --status --all and check if vmware-vpxd si up and running.

Run this command to start a service:

service-control --start vmware-vpxd

or boot vcenter.

[diegodco31]

Let us know if you fix your problem

[msripada]

This is a problem with the database and events table. Probably you can truncate the events table.

2018-07-11T16:09:01.784+02:00 error vpxd[7FD67D7EA800] [Originator@6876 sub=MoEvent] [EventManagerMo] Failed to load events: "ODBC error: (XX002) - ERROR: index "pk_vpx_event_4" contains unexpected zero page at block 0;

Truncate vpx_event_4 cascade;

Ensure to take snapshot on the vcenter and backups before performing the same

Thanks,

MS

[nidla5]

VCDB=# truncate pk_vpx_event_4 cascade;

ERROR:  "pk_vpx_event_4" is an index

VCDB=# truncate vpx_event_4 cascade;

TRUNCATE TABLE

VCDB=#

services restarted but the vpxd is still down

[msripada]

can you share the latest log after restart?

[nidla5]

2018-07-11T17:46:53.907+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=VpxProfiler] Init [AlarmMoManager::Init()] took 586 ms

2018-07-11T17:46:53.907+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=vpxUtil] [VpxVmomi] Found previous domain socket /var/run/vmware/vpxd-webserver-pipe. Removing...

2018-07-11T17:46:53.907+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for nonChunkingAgents

2018-07-11T17:46:53.907+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=HttpSvc.HTTPService] Using default for agentsNeedingContentLength

2018-07-11T17:46:53.907+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=HttpSvc.HTTPService] Max buffered response size is 104857600 bytes

2018-07-11T17:46:53.916+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.stats.report

2018-07-11T17:46:53.916+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension com.vmware.vim.vcha

2018-07-11T17:46:53.927+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=MoExtensionMgr] [ExtensionManagerMo] Public key file  or VECS infois not found or is not valid for extension hostdiag

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoDiagnosticMgr] [DiagnosticManagerMo] Running support script located at /bin/vc-support.sh

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.persist, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.persist

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.level, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.level

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key vpxd.usageStats.duration, pre commit

2018-07-11T17:46:53.968+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoOptionMgr] No change to vpxd.usageStats.duration

2018-07-11T17:46:54.162+02:00 error vpxd[7F0C3EEF6700] [Originator@6876 sub=vmomi.soapStub[5]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T17:46:54.162+02:00 warning vpxd[7F0C3EEF6700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-07-11T17:46:54.189+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=licenseServiceDiscovery] Discovered license service endpoint URL: https://VMM.scidom.de:443/ls/sdk

2018-07-11T17:46:54.189+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=licenseServiceProxy] Using ls URL: https://VMM.scidom.de:443/ls/sdk

2018-07-11T17:46:54.378+02:00 error vpxd[7F0C3C421700] [Originator@6876 sub=vmomi.soapStub[7]] initial service state request failed, disabling pings. error=HTTP Status:500 'Internal Server Error'

2018-07-11T17:46:54.378+02:00 warning vpxd[7F0C3C421700] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-07-11T17:46:54.426+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=componentManagerUtil] Discovered VC service folder ID: d5b33cf1-3fde-11e3-9ca0-000c299d97e3

2018-07-11T17:46:54.427+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] Creating SSL Contexts

2018-07-11T17:46:54.428+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Command invoked: /sbin/pidof

2018-07-11T17:46:54.481+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoReverseProxy] Finished executing command: /sbin/pidof

2018-07-11T17:46:54.481+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=MoReverseProxy] [VpxdReverseProxy] Pid of reverse proxy is: 17177

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT 4;

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 53 ms

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is 58P01:0:7:ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12380355": No such file or directory;

--> Error while executing the query

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] Connection diagnostic data from driver is 58P01:0:110:ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12380355": No such file or directory

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] Bind parameters:

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] [0]datatype: 1, size: 4, arraySize: 0

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] value = 76357270

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] [1]datatype: 1, size: 4, arraySize: 0

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] value = 76357091

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] [2]datatype: 1, size: 4, arraySize: 0

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] value = 76357274

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] [3]datatype: 1, size: 4, arraySize: 0

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] value = 102133024

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from 58P01:7

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from 58P01:110

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12380355": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T17:46:54.535+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=MoEvent] [EventManagerMo] Failed to load events: "ODBC error: (58P01) - ERROR: could not open file "pg_tblspc/16398/PG_9.4_201409291/16385/12380355": No such file or directory;

--> Error while executing the query" is returned when executing SQL statement "SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME,           USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME,           COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME,          DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME,           NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME,           STORAGEPOD_ID, STORAGEPOD_NAME, CHANGE_TAG_ID           FROM VPXV_EVENT_ALL            WHERE (EVENT_ID IN (?,?,?,?)) ORDER BY EVENT_ID ASC           LIMIT "

2018-07-11T17:46:54.536+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=ConfigIssue] [VpxdManagedEntity::Load] Processing of persisted config issues failed: vim.fault.DatabaseError

2018-07-11T17:46:54.536+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Main] Init failed. SystemError: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)

--> [context]zKq7AVECAAAAAIStcgAKdnB4ZAAAeF4rbGlidm1hY29yZS5zbwAAEBcbAMppGAEO1FR2cHhkAAEj1lQBsllVAb37VAFqn1MC4AUCbGliYy5zby42AAF1l1M=[/context]

2018-07-11T17:46:54.536+02:00 warning vpxd[7F0C4DBA4800] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 6374 ms

2018-07-11T17:46:54.536+02:00 error vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] Failed to intialize VMware VirtualCenter. Shutting down

2018-07-11T17:46:54.536+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=SupportMgr] Wrote uptime information

2018-07-11T17:48:54.537+02:00 info vpxd[7F0C4DBA4800] [Originator@6876 sub=Default] Forcing shutdown of VMware VirtualCenter now

[nidla5]

does it make any sense to recreate these files that are missing?

I have created some of them but constantly coming the new one.

[SupreetK]

Since these are binaries specific to the environment, it does not help recreating them. Redeploying the vCenter is THE option I think.

Cheers,

Supreet

[diegodco31]

I think redeploying the vCenter is the options more fast.

Sorry.

[msripada]

No that does not make sense unless we know what that file is.. It is a database table file.. It would be tough anyways.

What is the environment size?Do you have dvswitch?

[msripada]

Hello

I have last option for you to try. I am hoping it would work.

Truncate vpx_event_1 cascade;

Truncate vpx_event_2 cascade;

Truncate vpx_event_4 cascade;

Truncate vpx_event_3 cascade;

Go till Truncate vpx_event_99 cascade;

Note:  This would truncate all events and tasks

Thanks,

MS

[nidla5]

Hello and thank you for your help. after the truncate vCenter is again up & running :smileyhappy:

thanks to other guys also who gave very valuable tips.

[Weizhao1]

I have the same problem, vcenter cannot work .please how to fix it

[virtual-power]

This is a very old thread. Appreciate starting a new one and updating the vpxd log

[SupreetK]

Since pg_tblspc are binaries specific to the environment, it is not feasible to recreate them or copy them from another working environment. With every error that we fix, it will report a new error. I would recommend redeploying the vCenter. However, give the below a try before deciding on redeploying -

1) Take a snapshot or backup the DB.

2) Run the command - <truncate table pk_vpx_event_4 cascade;> on the DB

3) Restart the services.