UIM v20.3.3 - importing CA certificate

[NARONGKIJ TEJASAKULSIN]

Hi All,

I'm trying to enable https and we have our own CA, but when I tried to follow the instructions ( https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/unified-infrastructure-management/20-3/installing/optional-post-installation-tasks/Configure-HTTPS-in-Admin-Console-or-OC-(Authority-Signed-SSL-Certificate).html#concept.dita_cb13e2ee-2988-4c0f-9401-1f561f2acf68_ModifywasptoUseHTTPS2  ), it doesn't seem to work.

The above link has the "Generate a Public and Private Key Pair" section, which I did, but after importing my .cer file at the "Import the Certificates", the wasp starts but the webpage will not load.

It works with self-signed certificate.

When I do the "keytool -list -keystore wasp.keystore...", it will show the imported certificates, but only the self-signed will show up in the web browser.

Further more, after doing the "keytool -list -keystore wasp.keystore..." command, I'm getting the following error message:
The JKS keystore uses a proprietary format. it is recommended to migrate to pkcs12 which is an industry standard format using "keytool -importkeystore -srckeystore...".


Help greatly appreciated.

[David Michel]

perhaps:
Failed to load response data when loading CABI site after installing signed cert
Article Id: 186517
https://knowledge.broadcom.com/external/article?articleId=186517

------------------------------
Support Engineer
Broadcom
NY
------------------------------

[NARONGKIJ TEJASAKULSIN]

I'm also confused with the "high level" instructions from the documentation vs this Article ID: 16748 
https://knowledge.broadcom.com/external/article/16748/how-to-configure-umpwasp-to-use-ssl-cert.html

The high level doc does not indicate to include hostname and IP address when using the keytool command.

[Andrew Cooper]

We also have our own CA and we use the standard pkcs12 format.

First the wasp is a standard tomcat portal server so it supports the standard pkcs12 format keystore automatically.

So after first resetting the wasp keystore so you know the location and password you can then build the new keystore in the desired format.
As we have a private pki and the server is "known"​ our security service provides 3 files
1. a server cert file,
2. a file containing the intermediate and root certificates and
3. a key file
first we add/append the server cert to the chain file (server cert + chain certs)
then we delete the "old" keystore
now we create a new keystore (in pkcs12 format) and load the full cert chain and the key file with the name/alias wasp using:
  openssl pkcs12 -export -in <full_chain_file> -inkey <key_file> -name wasp -out <keystore> -passout pass:<key_password>
finally restart the wasp

when the the cert is about to expire we get sent a new set of files.  The update is scripted and security sends a person who enters the password so we never know the password for the key.

This works very well on all the wasps (AC, OC and CABI) and we do the process every 6 months or so, downtime is the time to restart the wasp to force a re-read of the update keystore. Another advantage of doing it this way is we have no CSR requirement, and security manages the password not ops (security cannot access the key file and ops does not know the password to use it).

Regards, Andrew

------------------------------
Knows a little about UIM/DXim, AE, Automic
------------------------------

[NARONGKIJ TEJASAKULSIN]

I'm trying to import the cer file from our own CA to UIM using keystore, but I'm getting an error message: Failed to establish chain from reply

[NARONGKIJ TEJASAKULSIN]

Hi Andrew,
Can you give me a step by step on how to do this on the UIM wasp.  I've been trying to figure out how to do this and it's just not working for me.  I'm on UIM 20.3.3 with wasp v20.33,  btw, the step by step documentations was what I tried to follow and it doesn't work... leading me to article after article.

Thanks.

[NARONGKIJ TEJASAKULSIN]

I'm able to import the root and intermediate certs.  It's rather confusing at first, as this is the first time I'm doing it, as well as the file format being different.  Problem now, when I try to access the https page I get the browser error message: This site can't be reached

[Andrew Cooper]

As wasp is a wrapper around tomcat, and it is tomcat that is managing the web interface, you need to ensure that everything is correct for tomcat.

• https port is defined in wasp config file (if it is not then https is not enabled on the wasp)
• the certs are in base64 format (ie. .pem files)
• all the certs are in a single file (ie the chain from the server cert to your private root is complete.  we have 4 certs in the chain for example) that is then loaded into the keystore, it seems that the wasp/tomcat does not support having separate entries in the keystore for the server cert and the CA certs even though this is supported by the keystore format.
• that the certs and key file are registered in the keystore with alias/name of "wasp" as this is what the wasp uses to find the keys it is to use in the keystore.

Finally check with netstat that something is listening on the port you are using in the browser. Does http work ? or is it both protocols that fail?​

------------------------------
Knows a little about UIM/DXim, AE, Automic
------------------------------

[NARONGKIJ TEJASAKULSIN]

• https port is defined in wasp config file (if it is not then https is not enabled on the wasp)
  • Yes, enabled by following the documentation by adding the "https_port" key.
• the certs are in base64 format (ie. .pem files)
  • Yes, received the files in .p7b files and converted to .cer files.
• all the certs are in a single file (ie the chain from the server cert to your private root is complete.  we have 4 certs in the chain for example) that is then loaded into the keystore, it seems that the wasp/tomcat does not support having separate entries in the keystore for the server cert and the CA certs even though this is supported by the keystore format.
  • I was given 3 files; Root, Issuing/Intermediate, and a file containing both the Root and Issuing/Intermediate in .cer format.
• that the certs and key file are registered in the keystore with alias/name of "wasp" as this is what the wasp uses to find the keys it is to use in the keystore.
  • Yes, imported the cert containing the root and Issuing, using command:  keytool -import -trustcacerts -alias wasp -file <PATH TO CERT>file.cer -keystore <PATH>/Nimsoft/probes/service/wasp/conf/wasp.keystore

Http works, https doesn't work.  But I when I "played" around using a self-signed certificate, https works (I just get that annoying security warning in the browser).

When I requested the CSR, I used the command:
keytool -certreq -alias wasp -validity 730 -keystore <PATH>/Nimsoft/probes/service/wasp/wasp.keystore -ext SAN=dns:<FQDN> -file <FQDN>.csr

[Andrew Cooper]

You are not doing item 3 correctly.   all the certs need to be in a single file (by appending them all together) so the server certificate + the intermediate certificate + the root certificate .
the trustcecerts and cacerts options for the keystore do not work with the wasp.

I don't use keytool but rather openssl and you see the command above to create the keystore  and load with the correct information above, I dont know the equivalent keytool command.  As you are generating a CSR the private key is already in your keystore so you only need to load the certificates (but you must load all the certificates in one go) we never got the system to work if we load the intermediate and root certificates separately from the server cert.

Regards, Andrew

------------------------------
Knows a little about UIM/DXim, AE, Automic
------------------------------

[NARONGKIJ TEJASAKULSIN]

The certificate I imported contained the Root and Issuing.  (Below).  You're saying this isn't correct?  And that I should import the Root and Issuing by appending them all together with what ever command from keytool, correct?

[Andrew Cooper]

The file you import needs to contain the hostname certificate as well as the issuing and root certificates.  This is not a keytool function but a text file append.
we use the OS copy command to achieve the concatenation of all the certificate files into one file.  It is this file that we use openssl to load into the keystore under the wasp alias.

------------------------------
Knows a little about UIM/DXim, AE, Automic
------------------------------

[Andrew Cooper]

Looking back a few years I had a UMP running on RHEL in this situation we finally worked out the following

1. manually stop wasp
2. on the server run script which did:
   1. delete existing keystore
   2. create new keystore and generate CSR
   3. CSR passed to Redhat Satellite which return a new machine certificate (in base64 format)
   4. append the returned machine certificate to a local copy of the intermediate and root certificate file
   5. import the certificate chain above into the keystore with alias wasp
   6. update the wasp config with the new keystore password
3. manually start the wasp

This more accurately follows the steps you need to do.  It took us a long time to figure out that the whole chain needed to be in a single chain file of certificates and that the cacerts entries are ignore by the wasp

Regards, Andrew

------------------------------
Knows a little about UIM/DXim, AE, Automic
------------------------------

[NARONGKIJ TEJASAKULSIN]

Thanks a bunch with the steps.  I'm still having problems though.

The problem after importing the certs and when I try to access the https, I get a "ERR_CONNECTION_TIMED_OUT" message on the browser.  It's like the browser can't connect via https, but http works.

If I remove the certs from the keystore, I am able to access the https fine.

I'm stumped.

[Marius Nitu]

Hello,

I have faced this problem a few times. It is an interesting take, Andrew, to convert the keystore to PKCS12. I have always tried to import the PKCS12 file to the JKS keystore.

I am not an expert on this, I will just share what worked for me.

1. If I can, I generate a CSR and follow the instructions in the documentation
2. If I receive the certificates in Base64 (ASCII) format I first convert them to PKCS12 and then import them
   • create a backup of wasp.keystore
   • keytool -delete -alias wasp -keystore wasp.keystore
   • (This will concat all the files to one file:)
   • cat server.crt intermediate.crt root.crt > server-cert.txt
   • (now use openssl to change the format:)
   • openssl pkcs12 -export -inkey server-key.pem -in server-cert.txt -out server-cert.p12
   • (I used changeit for password, but you can use another:)
   • keytool -importkeystore -deststorepass changeit -destkeypass changeit -destkeystore wasp.keystore -srckeystore server-cert.p12 -srcstoretype PKCS12 -srcstorepass changeit -alias wasp
3. If I receive the certificate chain in PKCS12 format, i follow these steps:
   • create a backup of wasp keystore
   • keytool -delete -alias wasp -keystore wasp.keystore
   • keytool -v -importkeystore -srckeystore server-cert.p12 -srcstoretype PKCS12 -destkeystore wasp.keystore -deststoretype JKS
   • keytool -list -keystore wasp.keystore
   • (The certificate was imported with another name so it must be renamed to wasp:)
   • keytool -changealias -alias imported_alias -destalias wasp -keystore wasp.keystore
4. Sometimes in case of 2 or 3, I also had to import manually the intermediate and root certificates. I'm not sure why. If I received the certificates in PKCS12 format, I just import them into the Windows keystore and export them from there with the MMC snap-in
   • keytool -import -trustcacerts -alias root -file root.cer -keystore wasp.keystore
   • keytool -import -trustcacerts -alias intermediate -file intermediate.cer -keystore wasp.keystore

Regards,
Marius

[NARONGKIJ TEJASAKULSIN]

Hi Andrew, and all,

I've got a test environment and I have no issues importing the certs.. got it all working correctly with no errors.. I'm now thinking that my WASP probe on the UIM hub is corrupted.  For some odd reason, it works with self-signed certs, but fails with CA certs.

I would like to bounce this idea to you guys:
I'm thinking of reinstalling the WASP on my UIM but I'm afraid of loosing data and I also don't want to issue out another CSR for the new cert.
As for the CSR, I think I can copy the wasp.keystore file to a safe location before removing the WASP probe.
Then when I reinstall the WASP, I just copy and paste the wasp.keystore file back to the original location.

Your thoughts on the above?

[Kiran Kumar Peramareddy]

Hi Narongkij,

Can you try the below knowledge-based article?
https://knowledge.broadcom.com/external/article/160027/error-keytool-error-javalangexception-fa.html

Regards
Kiran Peramareddy

[Kiran Kumar Peramareddy]

Can you try this KB article?

https://knowledge.broadcom.com/external/article/160027/error-keytool-error-javalangexception-fa.html

[NARONGKIJ TEJASAKULSIN]

Yup, tried it.  no go.

[Kiran Kumar Peramareddy]

Hi Narongkij,
The error message "Failed to establish chain from reply" will appear whenever we try to add a CA-signed certificate to Keystore and at the same time the Root certificate of the same CA is not available in the Keystore.
First, get the root certificate of the CA and try adding it using the following command and then add the signed certificate. The following commands may help you.

-- Import Root certificate:
keytool -import -file <CA root certificate name> -alias rootCA -keystore wasp.keystore -trustcacerts

-- Import Signed certificate:
keytool -import -alias wasp -file <CA signed certificate> -keystore wasp.keystore

Hope this helps..

Regards
Kiran

[NARONGKIJ TEJASAKULSIN]

I underlined the text in Red, how do I this?  There aren't any steps and commands, furthermore what would be the password?

[Marius Nitu]

Hello,

Password is "changeit".
The command would be something like this:

keytool -importcert -keystore cacerts -alias <alias> -storepass changeit -file <file> -trustcacerts

Marius

[NARONGKIJ TEJASAKULSIN]

This is what I did, would it be ok?

<path>\Nimsoft\jre\jre8u282b08\bin\keytool -importkeystore -srckeystore <path>\Nimsoft\jre\jre8u282b08\lib\security\cacerts -destkeystore <path>\Nimsoft\jre\jre8u282b08\lib\security\new\cacerts -deststorepass changeit

[Marius Nitu]

Sorry, I read the docs now.

So there is the java keystore cacerts that contains root certificates of many certificate authorities. You can upload your root certificate in there and it will work, but probably the documentation meant you should upload the root certificate into the wasp keystore instead.

The command I gave you uploaded the certificate into cacerts.
Your command I think will create another keystore that will contain the certificates in the cacerts keystore, but java will not know of your new keystore and you don't have your root certificate imported.

You should just follow the instructions in the documentation on how to import the root certificate in the wasp keystore.

If your certificate authority provided a root certificate, import the root certificate: <OC or UIM server_installation>/jre/<jre_version>/bin/keytool -import -trustcacerts -alias <root_certificate> -file <root_certificate>.cer -keystore <OC or UIM server_installation>Nimsoft/probes/service/wasp/conf/wasp.keystore


If you are not using certificate requests then the  instructions from Andrew that he already gave should be enough.
I also gave some instructions above:

• Sometimes in case of 2 or 3, I also had to import manually the intermediate and root certificates. I'm not sure why. If I received the certificates in PKCS12 format, I just import them into the Windows keystore and export them from there with the MMC snap-in
  • keytool -import -trustcacerts -alias root -file root.cer -keystore wasp.keystore
  • keytool -import -trustcacerts -alias intermediate -file intermediate.cer -keystore wasp.keystore

[NARONGKIJ TEJASAKULSIN]

Thanks.
I followed the instructions and imported the root, intermediate, etc..   but it seems that I have a bigger issue than just a simple cert import.

In my test environment, I have absolutely no problems doing either the self-cert or CA certs into UIM.
In my production environment, the moment I go past the step step 3 in (Generate a Public and Private Key Pair), it seems that WASP https just doesn't work.  WASP just doesn't open the port as if it's not running.  But the regular port 80, non ssl, works.  The weird thing is that up until step 3, https does work in wasp, showing in the browser with a warning as a "self-signed certificate", at the specified https port.  The moment I go beyond that step and import my CA certs, wasp https stops.

In any case, support is remoting in to investigate.  Hopefully I'll be able to share with everyone the solution.

[NARONGKIJ TEJASAKULSIN]

Finally got it to work!
Getting the certificate is dependent on the organizations requirements and my organization was a little particular with the requirements.
The online document does provide a generic method of creating an Authority Signed certificate, but my organization required the SAN entry in the certificate.