vcenter 5.5 ssl certificate expired

[Ben Tang]

Hello,everyone

My vsphere vcenter version is 5.5, I got an error "failed to connect to vmware lookup service https://xxxx:7444/lookupservice/sdk-ssl certificate verification failed" when I intended to log in to the vsphere web client and got an other error "wrong user or password" when logged in to the vsphere client. So then I searched the Internet, I found that those errors were caused by SSL certificate expired for vcenter. I verified this after I checked the relevant certificates in my vcenter for suse linux machine. I found the certificates were expired in Feb,2025. And then I got a tutorial about fixing this problem in the url "https://paulgrevink.wordpress.com/?s=Implementing+CA+signed+SSL+certificates+with+vSphere+5.x+". I followed it step by step. But when I proceeded to Part 3,  I got "VC_CFG_RESULT=650" error(it must be code 0 if the command succeeded) when I used the "/usr/sbin/vpxd_servicecfg certificate change" command. I wanted to find out what this meant. But I got nothing when I searched the broadcom website. The website no longer provided any service for vcenter 5.5. And the KB article about 5.5 were not found in it.

I've been trying to search out how to solve this, but still haven't. So anyone can help??

[Ben Tang]

Anyone could help?

[Ben Tang]

Could anyone else help?

[Andrea Consalvi]

Hi Ben,

vCenter 5.5 has been officially out of support since 2020, so it's expected that tools and documentation related to cert-manager and SSL certificates are no longer available through current Broadcom/VMware channels.

The errors you received:
"failed to connect to vmware lookup service" and
"sdk-ssl certificate verification failed"
are unfortunately a classic sign that the vCenter SSL certificates have expired.

The command vpxd_servicecfg certificate change often no longer works once the certificates have already expired, and the error VC_CFG_RESULT=650 is just a generic confirmation that the procedure failed. If you want to attempt a recovery, here's a method (not guaranteed, but it has worked in some cases):

Rename the expired certificates:

mv /etc/vmware-vpx/ssl/rui.crt /etc/vmware-vpx/ssl/rui.crt.bak  
mv /etc/vmware-vpx/ssl/rui.key /etc/vmware-vpx/ssl/rui.key.bak

Generate a new self-signed certificate:

openssl req -newkey rsa:2048 -nodes -keyout rui.key -x509 -days 730 -out rui.crt 

You'll be asked for some parameters. Fill them out, making sure the CN matches the FQDN of the vCenter (e.g., vcenter.lab.local).

Copy the new files to the vCenter SSL directory:

cp rui.crt /etc/vmware-vpx/ssl/  
cp rui.key /etc/vmware-vpx/ssl/

Restart the vCenter services:

service vmware-vpxd restart 

After this attempt, try accessing the vSphere Web Client or connecting via browser again. If it works, you might get a warning about an untrusted certificate, but the service should be operational again.

[Ben Tang]

Hi Andrea,

Thanks for your help.

I try the method you provide. But unfortunately, it don't take effect. After I copy the rui.crt and rui.key, I try to restart the vmare-vpxd service, but I get the error "Getting SSL certificates for https://FQDN:7444/lookupservice/sdk     com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified    Return code is: SslHandshakeFailed", and show "Waiting for vpxd to initialize: .failed". So when I try to login in to the web client, it shows the error "can not connect to the vCenter Single Sign-On Server https://FQDN:7444/sts/STSService/vsphere/local", the error I also got before. 

What should I do next? I had also tried to modify the system datetime before, but it didn't take effect.

[Marco_Luvisi]

Hi Ben,

try use this guide in my case have solved more and more time:
https://blogs.vmware.com/professional-services/2023/02/how-to-renew-an-expired-vmware-vcenter-service-appliance-certificate.html
vCenter 5.5 is not supported, but you can try.

[Ben Tang]

Hi, Marco

Appreciate your help.

I visit the url you provide. But the methods it provides I'd ever tried and is not suitable for the version 5.5. Like the .py and .sh scripts it mentions, couldn't take any effect.

[Marco_Luvisi]

Sorry,

do you have any backup of VCSA ?

[Ben Tang]

I have two snapshots for the VCSA.

[Marco_Luvisi]

I think you have tried to restore this snapshot without solve the issue right ?

[Ben Tang]

I tried them all. But no effects.

[Marco_Luvisi]

I'm sorry :-(

[Ben Tang]

Never mind. Also thanks for your help!! ^--^

[Andrea Consalvi]

Hi Ben,

The error you're seeing now about the certificate chain not being verified is expected when using a generic self-signed certificate that doesn't include all the required fields. In vCenter 5.5, internal components like LookupService or SSO require not only a valid certificate, but one that includes the correct Subject Alternative Names (SANs), otherwise the handshake fails.

I suggest regenerating the certificate using OpenSSL, but this time making sure to add the SAN extension with the FQDN, IP address, and optionally localhost. This has solved the same issue in other vSphere 5.5 environments where the vpxd service would start but LookupService would still reject the cert.

example:

openssl req -new -x509 -nodes -days 730 \
 -out rui.crt -keyout rui.key \
 -subj "/CN=your-vcenter-fqdn" \
 -extensions v3_req \
 -config <( cat /etc/ssl/openssl.cnf \
   <(printf "\n[v3_req]\nsubjectAltName=DNS:your-vcenter-fqdn,IP:your.vcenter.ip,DNS:localhost") )

Make sure to replace the FQDN and IP with your actual vCenter hostname and IP address. Once the files are generated, copy them back to /etc/vmware-vpx/ssl/, then restart the vpxd service:

service vmware-vpxd restart 

This usually allows internal services to trust the certificate again and resolve the handshake issue.

[Ben Tang]

I made a config file for cert like the following before:

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req
input_password = testpassword
output_password = testpassword

[ v3_req ]
basicConstraints = CA:false
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS:vcva55, IP: 10.0.0.10, IP:ServerIPv6Address, DNS: vcva55.vmware.com

[ req_distinguished_name ]
countryName = US
stateOrProvinceName = NY
localityName = New York
0.organizationName = VMware
organizationalUnitName = vCenterApplianceUniqueServer
commonName = vcva55.vmware.com

and the SANs information I got from the old rui.crt in the location "/etc/vmware-vpx/ssl/" with the openssl command. So far, I still use this template to generate the various certificates with the openssl command.

The procedures I follow are from the url "https://community.broadcom.com/vmware-cloud-foundation/discussion/vcenter-server-550-windows-server-certificate-expired". But it can also do nothing.

[Andrea Consalvi]

Hi Ben,
in vCenter 5.5, even with a correctly generated certificate, components like SSO and LookupService often reject it if the SAN section is missing or incomplete. This is usually the cause behind the "certificate chain not verified" and "SSLHandshakeFailed" messages you're seeing.

Your current .cnf includes the SAN, but depending on how it's parsed and passed by openssl, some fields might be skipped or ignored. That's why I suggested injecting the SAN block dynamically via cat and printf, which helps force OpenSSL to merge it correctly during the request generation - especially in older vCenter setups.

If you're still seeing the error even with this config, I'd retry the certificate creation using the inline openssl command I posted earlier, making sure the SAN includes:

• The vCenter FQDN

• The vCenter IP address

• Optionally: localhost

Once regenerated, overwrite rui.crt and rui.key under /etc/vmware-vpx/ssl/, then restart the service:

service vmware-vpxd restart 

If after that you're still stuck at the handshake step, we can try reviewing the STS certificate and truststore chain next.

[Ben Tang]

Hi Andrea,

I use your command to get the rui.key and rui.crt, but it still doesn't work.

[Andrea Consalvi]

Hi Ben, I knew it was going to be tough with 5.5. The error you're getting (CertificateValidationException: Server certificate chain not verified) suggests that the certificate was generated, but it's still missing one of the Subject Alternative Names (SANs) expected by vCenter Lookup Service.

Here are a few things to check that are essential:

1. The CN must exactly match the vCenter's FQDN
   → e.g. /CN=vcenter.lab.local

2. The SAN must include at least:

   • DNS:<FQDN> (same as the CN)

   • IP:<vCenter IP>

   • Optionally: DNS:localhost

   If you skip any of those, Lookup Service will fail.

3. Verify there are no hidden characters in the generated cert
   Sometimes, copying commands from the browser can introduce invisible characters or wrong quotes (like smart quotes "" instead of ""). Use a raw text editor on Linux (like nano or vi).

Try this updated OpenSSL command (replace FQDN and IP accordingly):

openssl req -new -x509 -nodes -days 730 \
 -out rui.crt -keyout rui.key \
 -subj "/CN=vcenter.lab.local" \
 -extensions v3_req \
 -config <( cat /etc/ssl/openssl.cnf \
 ; echo -e "\n[v3_req]\nsubjectAltName=DNS:vcenter.lab.local,DNS:localhost,IP:10.0.0.10" )

Then copy the certs:

cp rui.crt /etc/vmware-vpx/ssl/
cp rui.key /etc/vmware-vpx/ssl/

And restart the service:

service vmware-vpxd restart 

Let me know how it goes if it still fails, we'll look at the SAN content in the certificate directly using:

openssl x509 -in rui.crt -noout -text | grep -A1 "Subject Alternative Name"

This will confirm exactly what SANs were included.

[Ben Tang]

I use your updated command, the result and the cert content about SANs are following. And the error still occurs.  I think it's a big trouble. LOL!!!

Thanks again for your help!!!

[Andrea Consalvi]

Hi Ben,

I'm sorry to hear the issue is still persisting even after including the SANs.

At this point, the error "Server certificate chain not verified" and "SslHandshakeFailed" indicates that the problem is not with the SANs anymore, but with the trust chain of the certificate itself.

When using vCenter 5.5, the certificate must either:

• Be self-signed and correctly located in the vCenter paths,
  OR
• Be signed by a trusted Certificate Authority (CA) that is present in the Java truststore used by vCenter services.

Here's what you can do next to verify:

1. Double-check that your certificate is self-signed using:

openssl x509 -in rui.crt -noout -issuer -subject

1. If issuer and subject are identical, it's self-signed.
2. If it's self-signed, make sure:
   
• It's 2048-bit RSA.
• It includes valid SANs (which you've already done).
• It's copied correctly to /etc/vmware-vpx/ssl/.
3. If it's not self-signed, you must import the issuing CA into the Java truststore used by vCenter.
   Unfortunately, in vCenter 5.5 this process was fragile and not always reliable sometimes a full reinstallation with new certs was the only way out.

[Ben Tang]

I check the cert rui.crt I generated and the native cert rui.crt.bak.

The rui.crt's issuer and subject are identical. But the rui.crt.bak's are not.

The rui.crt.bak's issuer is "C=US, ST=California, L=Palo Alto, O=VMware Inc., CN=FQDN CA beffae4f/emailAddress=[e-mail]". And the subject is "C=US, ST=California, L=Palo Alto, O=VMware Inc., OU=VMware vCenter Server Certificate/emailAddress=[e-mail], CN=FQDN/unstructuredName=..."

And I try to use the sha256 and 2048 key bits arguments(Because the rui.crt.bak used these arguments) with your command to generate my cert. But it also doesn't work.

[Monkee C]

My guess at this is that your client and server may not have a cipher suite in common. vCenter 5.5 is very old and most, if not all, of the cipher suites from that time have been deprecated.

You can test by running this command on both ends and looking for a common suite in both lists:

openssl ciphers -s

[Andrea Consalvi]

Thanks for jumping in, Monkee! In this case, though, the error is being triggered internally by the LookupService when loading the local certificate. It fails with:

CertificateValidationException: Server certificate chain not verified  
Return code is: SslHandshakeFailed

So it's not an external client handshake issue the certificate is being rejected within the vCenter stack itself, even though it's self-signed, 2048-bit, and includes proper SANs.
We suspect it's related to vCenter 5.5 validating against an internal Java truststore or some hardcoded CA constraints, which can't be easily bypassed anymore unless the original VMware CA is preserved or reimported.
Still, your point is absolutely valid in general cipher mismatch is often the silent killer in legacy systems.

[Andrea Consalvi]

Hi Ben,

Thanks a lot for checking all that. From what you've confirmed the certificate is self-signed, has the correct SANs, is 2048-bit, and uses SHA256 everything looks structurally correct.

However, since the error still persists and you're getting:

CertificateValidationException: Server certificate chain not verified  
Return code is: SslHandshakeFailed

We're hitting one of the biggest limitations of vCenter 5.5 even though you place a valid certificate in /etc/vmware-vpx/ssl/, some internal services like LookupService and SSO may still perform validation against a Java truststore or internal keystore, and reject any cert that's not explicitly trusted there.

This was a well-known issue back in the day, and unfortunately, there wasn't a fully documented or reliable way to update all trust anchors in vCenter 5.5 once the SSL chain broke.

At this point, there are two real paths forward:

1. Restore a snapshot or backup from before the cert expired - if one is available.
2. Deploy a fresh vCenter 5.5 instance, rejoin it to the environment (if applicable), and manually reconfigure.

I know that's not ideal, but if the Java truststore rejected the cert once, it likely will continue to, unless you can somehow regenerate the original cert with the exact same VMware CA structure which isn't officially supported.

[Ben Tang]

It's very thankful for you.

I ever tried to modify the system datetime of the vcenter server to the value between the start time and the end time of the rui.crt, but after that I restarted the service vpxd, it also failed.

And if I want to reinstall the vcenter, I must find out the product's serial numbers. But I ain't sure the customer still has them.

Now, I can only operate the VMs by the Esxi client. So it's a great trouble with me.

[Ben Tang]

Hi Andrea,

I checked the vpxd.log when I restarted the service vpxd and then it showed the SslHandshakeFailed error. I found the following errors:

Do I need to do something about the postgre database after I update the cert?