Secure Proxy Server - Failed to load Java keystore (SSL)

[Legacy User]

I am trying to import a signed certificate so that SPS can communicate with https on the backend server and I am receiving a "Failed to load keystore" error in the server.log. (see attachment)

 

I am assuming that one error is preventing me from using https. 

 

Any ideas on what could be wrong or step that I may have missed to complete the SSL process for the keystore to be loaded properly?

 

I have performed the following steps...

 

1. Create the Cert Request

cd <install-dir>\SSL

..\bin\openssl req -out client2-CSR.csr -new -newkey rsa:2048 -nodes -keyout client2-privateKey.key -config ..\bin\openssl.cnf

 

2. Sign the Request

CSR was signed by CA

Convert cert to DER encoding

..\bin\openssl x509 –in client2-Cert_x509.pem -out client2-Cert_x509.cer -outform der

 

3. Convert private key to encrypted pkcs#8 DER encoding

..\bin\openssl.exe pkcs8 -in client2-privateKey.key -topk8 -v2 des3 -out client2-privateKey-DER.key -outform DER

 

4. Put files in right location:

     Place DER encoded client cert in :  <install-dir>\SSL\clientcert\certs\

            client-Cert_x509.cer

 

     Place encrypted DER encoded private key in :  <install-dir>\SSL\clientcert\key\

            client2-privateKey-DER.key

5. Generate Encrypted Password for server.conf file:

cd <install-dir>\SSL\bin
EncryptUtil.sh password

…

Encrypted string: U2FsdGVkX18VcMWDmBEJG7CL2edypl03V6Ig1F3gON4=

   

6. Modify the server.conf file :

             

              ClientKeyFile="client2-privateKey-DER.key"

              ClientPassPhrase=U2FsdGVkX1+wxoEp8DCUZ6/pcaHpitr6v88GproScgQ=

 

7. Restart SPS and check the server.log file: 

  

[Alfred Macedo]

The steps you listed appear to be correct. If we increase the logging we should be able to see the exact reason it has failed. 

 

Http Client/Java SSL Logging can be enabled by making this change:

1) Java has facility to log network SSL Connections 

   "-Djavax.net.debug=all" that should be applied to the appropriate file depending on the OS.

   a) Windows - C:\Program Files (x86)\CA\secure-proxy\proxy-engine\conf\SmSpsProxyEngine.properties

   b) Unix - proxy-engine/proxyserver.sh

 

If everything works out, then you should an additional DEBUG message.

For example, below indicates a private key error:

[ERROR] - RSASSLConfig.java : Failed to load keystore 
[DEBUG] - RSASSLConfig.java : com.rsa.ssl.SSLException: com.rsa.ssl.SSLException: Could not read private key. com.rsa.jsafe.JSAFE_UnimplementedException: Could not decode the data.(Not a valid RSA private SSLC key, missing header) 
at com.rsa.jsafe.JSAFE_SymmetricCipher.a(Unknown Source) 
at com.rsa.jsafe.JSAFE_SymmetricCipher.getInstance(Unknown Source) 

 

Hope this helps!

[Legacy User]

Thank you David. I enabled the DEBUG level and it appears to be a lot of java errors that complain about an unknown source. (See screenshot)

 

Any ideas on why it is complaining?

 

[Legacy User]

Jawaan wasja02

 

 

If the requirement is to enable for communicating successfully with a backend server running on https; then importing the Root CA (of the Certificate on the backend server) on CA AG is sufficient. This is similar to one way SSL (from browser to server - in our case CA AG is the Client and backend is the server).

 

The Steps being followed is pertaining to enabling TWO WAY SSL. Is that needed to beginwith ? If Yes, then we can debug ahead.

Enable Client Auth (2 Way SSL) From CA Access Gate - CA Knowledge 

[Legacy User]

Thanks Hubert. 

 

Yes. Customer would like to see an https frontend URL, so therefore two-way SSL would be necessary.

[Legacy User]

Jawaan wasja02

 

Not necessary. Here is my understanding (happy to be corrected).

 

Browser -->  CA AG https : Is handled by enabling SSL on Apache layer.  This is One Way SSL.

 

CA AG --> Backend https : Is handled by importing Root CA Certificate of backend server into CA AG as a Trusted Root CA. This is One Way SSL.

 

So Two Way SSL is not mandatory. It seems like we are over-killing by enabling Two Way SSL. Unless there is a reason like CLIENT AUTHENTICATION that is needed by backend, then we need Two Way SSL enabled.

[Mark ODonohue]

Hi wasja02 

 

It may be best to open a support case - it used to be a bit tricky and you had to get the steps exactly right - I remember trying to debug it to figure out the exact steps - but those look to be the ones you have.   

 

But also recently (12.8) the SPS changed to use bouncy castle crypto provider, not RSA cryptoj and there have been some spots in the code where the RSA classes were explicitly  used - that looks like it could be one of them - since it is  called RSASSLConfig.java

 

Cheers - Mark

Snr Principle Support Engineer 

[Legacy User]

Thanks Mark. This is actually 12.7 of Access Gateway. So that is why we are seeing the RSASSLConfig cryto.