Hello, guys!
VCSA 6.7 is showing "Certificate Status" alert, we found expired certificate in STORE MACHINE_SSL_CERT and updated it, creating new CSR, getting valid certificate and installing it to VCSA.
Now all certificates, except __MACHINE_CSR show expiration date next year and later.
__MACHINE_CSR seems to contain private key, according to my searches and expires right after creation:
Alias : __MACHINE_CSR
Entry type : Private Key
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
ff:98:45:69:35:0d:60:87
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=CA, DC=vsphere, DC=local, C=US, ST=California, O=xxxxxxxx103.xxx.xxxxx.xx.xx, OU=VMware Engineering
Validity
Not Before: Mar 24 16:16:35 2021 GMT
Not After : Mar 24 16:16:41 2021 GMT
But we still get "Certificate Status" alert, when I reset to green and restart server it comes up with same alert.
I searched for errors in vpxd.log and found this:
2021-03-26T19:04:50.842Z info vpxd[04690] [Originator@6876 sub=HostGateway] stsUrlFromConfig: https://xxxxxxxx103.xxx.xxxxx.xx.xx/sts/STSService/vsphere.local ssoAdminUrlFromConfig: https://xxxxxxxx103.xxx.xxxxx.xx.xx/sso-adminserver/sdk/vsphere.local
2021-03-26T19:04:50.870Z info vpxd[04690] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2021-03-26T19:04:50.891Z info vpxd[04690] [Originator@6876 sub=vpxCrypt] Failed to read X509 cert; err: 151441516
2021-03-26T19:04:50.891Z info vpxd[04690] [Originator@6876 sub=HostGateway] stsUrlFromLs: https://xxxxxxxx103.xxx.xxxxx.xx.xx/sts/STSService/vsphere.local ssoAdminUrlFromLs: https://xxxxxxxx103.xxx.xxxxx.xx.xx/sso-adminserver/sdk/vsphere.local
2021-03-26T19:04:50.892Z info vpxd[04690] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] Try to connect to SSO VMOMI endpoint
2021-03-26T19:04:50.928Z info vpxd[04690] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] Retrieved trusted STS certificate: CN=ssoserverSign, TP = 57:13:3D:B6:49:B1:C5:BE:C8:60:8A:58:4A:5E:D5:3F:CA:7E:24:C5
2021-03-26T19:04:50.962Z warning vpxd[04963] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f2a18001a60, h:26, <TCP '127.0.0.1 : 56062'>, <TCP '127.0.0.1 : 18090'>>, e: 111(Connection refused)
2021-03-26T19:04:50.965Z error vpxd[04690] [Originator@6876 sub=HostGateway] [CisConnection]: ComponentManager->LoginByToken failed: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:50.965Z warning vpxd[04690] [Originator@6876 sub=HostGateway] State(ST_CM_LOGIN) failed with: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:50.992Z warning vpxd[04970] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f29f4001a60, h:26, <TCP '127.0.0.1 : 56064'>, <TCP '127.0.0.1 : 18090'>>, e: 111(Connection refused)
2021-03-26T19:04:50.992Z error vpxd[04690] [Originator@6876 sub=HostGateway] [CisConnection]: ComponentManager->LoginByToken failed: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:50.992Z warning vpxd[04690] [Originator@6876 sub=HostGateway] State(ST_CM_LOGIN) failed with: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:51.015Z warning vpxd[04976] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f29e0001a60, h:26, <TCP '127.0.0.1 : 56066'>, <TCP '127.0.0.1 : 18090'>>, e: 111(Connection refused)
2021-03-26T19:04:51.015Z error vpxd[04690] [Originator@6876 sub=HostGateway] [CisConnection]: ComponentManager->LoginByToken failed: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:51.015Z warning vpxd[04690] [Originator@6876 sub=HostGateway] State(ST_CM_LOGIN) failed with: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:51.040Z warning vpxd[04984] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f29bc001a60, h:26, <TCP '127.0.0.1 : 56068'>, <TCP '127.0.0.1 : 18090'>>, e: 111(Connection refused)
2021-03-26T19:04:51.041Z error vpxd[04690] [Originator@6876 sub=HostGateway] [CisConnection]: ComponentManager->LoginByToken failed: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:51.041Z warning vpxd[04690] [Originator@6876 sub=HostGateway] State(ST_CM_LOGIN) failed with: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections.
2021-03-26T19:04:51.042Z warning vpxd[04690] [Originator@6876 sub=HostGateway] ComponentManager service is not available! Will attempt a lazy init of CmClient on first use!
Found couple of KBs related to "Failed to read X509 cert; err: 151441516", but still not able to find the cause of alert.
Please suggest!
Thank you.
Here is certificate expiration statuses:
for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done
STORE MACHINE_SSL_CERT
Alias : __MACHINE_CERT
Not After : Apr 18 17:01:19 2022 GMT
Alias : __MACHINE_CSR
Not After : Mar 24 16:16:41 2021 GMT
STORE TRUSTED_ROOTS
Alias : 17f5a32c553de219ab8df24c1e98a729ea86d4d8
Not After : Feb 14 16:59:02 2031 GMT
Alias : d0da552c55b6a3145e25bf824bd7b2fe2ed18221
Not After : Feb 14 15:55:31 2031 GMT
Alias : 182528a844fe0d4f478292e27cf9d21bde8cad6d
Not After : Oct 7 16:31:50 2026 GMT
Alias : 9b4f823bafd8c088a7b97a4f171e4858a612f46c
Not After : Sep 27 18:25:08 2036 GMT
STORE TRUSTED_ROOT_CRLS
Alias : a7261037faf320bf3b8757a618ed288c6c7e7597
Alias : c4758510accecc5201f1f82b4279c2f37f0f3583
STORE machine
Alias : machine
Not After : Feb 19 16:50:12 2023 GMT
STORE vsphere-webclient
Alias : vsphere-webclient
Not After : Feb 19 16:50:13 2023 GMT
STORE vpxd
Alias : vpxd
Not After : Feb 19 16:50:13 2023 GMT
STORE vpxd-extension
Alias : vpxd-extension
Not After : Feb 19 16:50:14 2023 GMT
STORE APPLMGMT_PASSWORD
Alias : location_password_default
STORE data-encipherment
Alias : data-encipherment
Not After : Feb 19 16:51:47 2023 GMT
STORE SMS
Alias : sms_self_signed
Not After : Feb 19 17:03:02 2031 GMT
STORE BACKUP_STORE
Alias : bkp___MACHINE_CERT
Not After : Mar 19 17:48:18 2023 GMT
Alias : bkp_machine
Not After : Feb 19 16:50:12 2023 GMT
Alias : bkp_vsphere-webclient
Not After : Feb 19 16:50:13 2023 GMT
Alias : bkp_vpxd
Not After : Feb 19 16:50:13 2023 GMT
Alias : bkp_vpxd-extension
Not After : Feb 19 16:50:14 2023 GMT