vCenter Alert: Certificate will expire soon

[Sungpill Han]

I have below alert today. What certificate is it talking about? SSL to connect to vCenter administration URL from any PC?

And how to fix this?

Thank you in advance.

[daphnissov]

It's probably the machine certificate for vCenter itself. Have you checked it?

Also, as a point of getting better help, there is absolutely zero reason to redact portions of a private IP address in your screenshots. You just make it more difficult for people to help you.

[Sungpill Han]

The IP address is the vCenter itself.

How do I check machine certificate and what's the consequence ignoring it?

[daphnissov]

Check the certificate in your browser when you access vCenter server. The consequences of letting it expire are that very likely vCenter will be broken, either immediately or when it reboots.

[Sungpill Han]

Itis' vcenter 6.5 and has same version of 3 ESXi.

I'm searching for a guide to resolve the issue. Do you have any link to the solution?

What I found are multiple articles, not sure exactly what this situation fits to them.

[daphnissov]

Do you have any link to the solution?

The solution is to first identify the certificate which is expiring as I mentioned in my last post. Once you know that, you can proceed to the next step: replacing the certificate.

[Sungpill Han]

Now I"m up to this, shows multiple cetificates expiring on 7/11.

How to update these certificates? there a bunch expiring including machine, vpxd and is that root cert 'C=US, CN=10.225.2.150'?

Does anyone have any guide or instruction link? We are in the middle of support renewal and can't get support from vmware and have things to do except this, time is too short.

[daphnissov]

We are in the middle of support renewal and can't get support from vmware and have things to do except this, time is too short.

Well, it looks like you better make time, because otherwise your vCenter is likely not going to work past Saturday.

Easiest thing for you to probably do is run /usr/lib/vmware-vmca/bin/certificate-manager and choose option #4 to generate a new VMCA root certificate and replace all certificates. Regardless of which you perform snapshot your vCenter first.

[Sungpill Han]

daphnissov,

I have experience renewing AD CA root, but this is first time doing for vCenter.

So, the path is;

1. generate new VMCA root cert

2. generate all other certs even including certs for ESXi host which is still valid until 2023?

   2-b. And if I have to generate new certs for ESXi, how do I distribute thme?

Do you have any good article from VMware KB or online on how to generate and replace certs in vCenter for this situation? And also things to read about VMCA and how it works with ESXi in cert chain?

[Sungpill Han]

OK,

Now I renewed cetificates through PSC.

However, there are two things not resolved.

1. When I access vCenter (https://10.225.2.150), the certificate is still the old one expiring on 7/12 2AM on my chrome browser.

2. This VMware article suggests to stop all services, then restart the 3 below services. Doesn't vCenter have more services? Why just those 3

vCenter Server Appliance (article: Replace Certificates with New VMCA-Signed Certificates from the vSphere Client )

[daphnissov]

Your screenshot shows you replaced the machine TLS certificate as well, so it should be presenting it. You may need to restart your vCenter and check in a private browsing interface that the new cert is getting presented.

[Sungpill Han]

I finished renewing 5 certs,

1 machine cert and 4 solution certs. However, there's 1 cert that doesn't seem not renewed, 'CN=ssoserverSign'. Others have similar names and new ones were created, but not this one. What is this cert for?

I renewed certs using PSC web interface.

1 machine cert and 4 solution certs

[Sungpill Han]

Hello,

I finanlly renewed and generated certs for vCenter.

But when I log into vCenter, the cluster and 3 hosts show red exclmation icon. I tried to renew or refresh certficate on host from vCenter 'right-click on a host> Certificate> Renew Cetificate', but nothing happens. How can I bring back the hosts and HA to vCenter?

[Sungpill Han]

I successfully tackled the certificate issues, spent 8 hours, hope anyone having the same cert renewal issue may find help from my notes.

=========================================================================

* VMware certificates in vCenter, PSC, VMCA and certificates

- Notes

- PSC (Platform Service Controller): controlls  idendity service (LDAP integration, SSO) and certficates(VMCSA)

- VMCA (vmware certificate authority) is a part of PSC controlling certificates used between vCenter and ESXi(Machine Certifictes), service to service (Solution User Certificates). The root certificate is self-signed by VMCA.

- VECS: repository for SSL certs and private keys.

- vCenter Applicance Manager web

https://vcenter:5480

- vCenter PSC web (VMCA)

https://vcenter/psc

- certificates renewed, replaced using PSC web interface or /usr/lib/vmware-vmca/bin/certificate-manager

- How to use vSphere Certificate Manager to Replace SSL Certificates (2097936) https://kb.vmware.com/s/article/2097936

- Cert Types

- STS cert (SSO. This is not managed by PSC GUI or certificate-manager.

- 1 CA root (VMCA self-signed)

- 1 Machine cert

- 4 Solution User certs

- Check current certificates in PSC

- STS cert(SSO): You need to use scripts to check(checksts.py) and renew(fixsts.sh) https://kb.vmware.com/s/article/79248

- https://vcenter/psc ([e-mail] / ...)

  - Certificates> Certificate Management

- 1 _MACHINE_CERT: proxy cert. all the endpoints communicate through this trusted ssl cert.

- 4 Solution user Certs:

- vpxd: cert for vCenter

- vpxd-extention: used by Auto Deploy, Inevntory Service--

- vsphere-webclient: used for vSphere Web Client

- machine: logging-service, compoment manager, license server.

- 1 Trusted Root Cert: VMCA self-signed root cert

- Renews certificates

- How to regenerate vSphere 6.x certificates using self-signed VMCA (2112283) https://kb.vmware.com/s/article/2112283

- Before renewing certificates, Make sure STS certificate(SSO token singing cert) is not expired. Please renew.

- Run checksts.py to see if STS certs are not expired. Checking Expiration of STS Certificate on vCenter Server (79248) https://kb.vmware.com/s/article/79248 or you can check from vCenter>Administration> Single Sign-On> Configuration> Certificate> STS Signing.

- Run fixsts.sh STS certificates. "Signing certificate is not valid" - Regenerating and replacing expired STS certificate using PowerShell script on vCenter Server 6.5/6.7 installed on Windows (79263) https://kb.vmware.com/s/article/79263

- Renew CA root(if needed), 1 Machine cert and 4 Solution User certs

- GUI method

- go to PSC web interface: https://vcenter/psc ([e-mail] / ...)

- menu: Certificates> Certificate Management

- _MACHINE_CERT: click renew

- Solution user Certs: click renew all

- Trusted Root Cert: We keep root cert signed by VMCA self-signed.

- Command method

- https://kb.vmware.com/s/article/2112283

- /usr/lib/vmware-vmca/bin/certificate-manager

- Option 3(machine cert) then Option 6(4 solution user certs)

-or Option 4. Regenerate a new VMCA Root Certificate and replace all certificates

-  Restart vCenter applicance instead or do;

Command> shell

service-control --stop --all

service-control --start vmafdd

service-control --start vmdird

service-control --start vmcad

service-control --start --all (this step is missing from the above linked article) 

- Stopping, Starting or Restarting VMware vCenter Server Appliance 6.x & above services (2109887)https://kb.vmware.com/s/article/2109887

- Disconnect and Reconnect hosts to vCenter

- When new certs are generated in vCenter, the ESXi hosts will have issues in communication to vCenter and show red exclamation mark on hosts.  You need to re-joint host to vCenter, then push certs to hosts.

- RC on a host> Connection> Disconnect

- RC on a host> Connection> Connect (This will fail then bring up Add Host wizard instead. Follow it to add the host to vCenter)

- RC on a host> Certificate> Renew Certificate (This will push vCenter machine cert to hosts)

- Other commands

/usr/lib/vmware-vmafd/bin/vecs-cli store list (List cert store list)

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text |less (show a trusted root cert detail)

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text |less (show a machine cert cert detail)

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store machine --text |less (show a solution user cert detail)

cat /var/log/vmware/vmcad/certificate-manager.log (certificate-manager command logs)