Not able to login and navigate IDM via https after upgrading Identity Suite Virtual Appliance from 14.1 to 14.3

[tjmuchenje]

Hi All, 

I have upgraded the Identity suite from 14.1 to 14.3 CP 02 (Virtual Appliance)  but i have noticed that access Identity Manager via https://<server>/iam/im/identityEnv/? kicks the out after authentication or when navigating but http://<server>:8080 /iam/im/identityEnv/? works just fine.

N.B Identity portal and Identity Governance all work fine. Just Identity Manager not working as expected.

Any pointers? 

Kind Regards
Tav

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,
When you access https://<server>/iam/im/identityEnv/? you hit the httpd, i.e. the Internal Proxy WebServer/Load Balancer. The httpd should route the https requests to the backend wildfly-idm (it could be to wildfly-idm on the other vApp node). What do you see in /etc/httpd/logs/ssl_error_log file?


------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Hi Widjaja,

Thank you for you response. Please see the output of my ssl_error_log. 

[Tue Sep 29 14:08:50 2020] [debug] ssl_engine_kernel.c(1908): OpenSSL: Exit: failed in error [Tue Sep 29 14:08:50 2020] [info] [client 10.10.247.53] SSL library error 1 in handshake (server CA_IMAG_VAPP:443) [Tue Sep 29 14:08:50 2020] [info] SSL Library Error: 336151574 error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown [Tue Sep 29 14:08:50 2020] [info] [client 10.10.247.53] Connection closed to child 4 with abortive shutdown (server CA_IMAG_VAPP:443) [Tue Sep 29 14:08:51 2020] [info] [client 10.10.247.53] Connection to child 3 established (server CA_IMAG_VAPP:443) [Tue Sep 29 14:08:51 2020] [info] Seeding PRNG with 288 bytes of entropy [Tue Sep 29 14:08:51 2020] [debug] ssl_engine_kernel.c(1876): OpenSSL: Handshake: start [Tue Sep 29 14:08:51 2020] [debug] ssl_engine_kernel.c(1884): OpenSSL: Loop: before/accept initialization [Tue Sep 29 14:08:51 2020] [debug] mod_proxy_http.c(2018): proxy: HTTP: serving URL http://caim-srv-02:8080/iam/im/identityEnv/ui/images/favicon.ico [Tue Sep 29 14:08:51 2020] [debug] proxy_util.c(2102): proxy: HTTP: has acquired connection for (caim-srv-02) [Tue Sep 29 14:08:51 2020] [debug] proxy_util.c(2158): proxy: connecting http://caim-srv-02:8080/iam/im/identityEnv/ui/images/favicon.ico to caim-srv-02:8080 [Tue Sep 29 14:08:51 2020] [debug] proxy_util.c(2289): proxy: connected /iam/im/identityEnv/ui/images/favicon.ico to caim-srv-02:8080 [Tue Sep 29 14:08:51 2020] [debug] mod_proxy_http.c(1775): proxy: start body send [Tue Sep 29 14:08:51 2020] [debug] mod_headers.c(743): headers: ap_headers_output_filter() [Tue Sep 29 14:08:51 2020] [debug] mod_deflate.c(687): [client 10.10.247.53] Zlib: Compressed 3952 to 1525 : URL /iam/im/identityEnv/ui/images/favicon.ico, referer: https://<<server>>/iam/im/identityEnv/index.jsp [Tue Sep 29 14:08:51 2020] [debug] mod_proxy_http.c(1885): proxy: end body send [Tue Sep 29 14:08:51 2020] [debug] proxy_util.c(2120): proxy: HTTP: has released connection for (caim-srv-02) [Tue Sep 29 14:08:51 2020] [debug] mod_proxy_balancer.c(633): proxy_balancer_post_request for (balancer://caim) [Tue Sep 29 14:08:51 2020] [debug] ssl_engine_kernel.c(1894): OpenSSL: Write: SSL negotiation finished successfully [Tue Sep 29 14:08:51 2020] [info] [client 10.10.247.53] Connection closed to child 7 with standard shutdown (server CA_IMAG_VAPP:443)​
What do you think?


------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,
It seems a Certificate issue?

[info] SSL Library Error: 336151574 error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown [Tue Sep 29 14:08:50 2020]

Have you done any Certificate configurations for vApp Web UI (httpd)? Can you verify if the Certificate is valid?
Please refer
https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/identity-suite/14-3/virtual-appliance/administering-virtual-appliance.html#concept.dita_484b93c7f06198e8b27adcc2537229358eb17777_ReplacingVirtualApplianceWebUISSLCertificate

------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Hi Widjaja,

I can confirm i have a valid cert and key under

/opt/CA/VirtualAppliance/custom/apache-ssl-certificates 

• localhost.crt (public key)
• localhost.key (private key)

Kind Reagrds
Tavernt

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,

If you access wildfly-idm https port, i.e. tcp/8443 directly (by-passing httpd), does it work?

   https://<server>:8443/iam/im/identityEnv/?

Regards,
Widjaja.

------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Hi Widjaja, 

Yes, bypassing httpd (  https://<server>:8443/iam/im/identityEnv/?) works fine.

Regards
Tavernt

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,

I think we narrow down that the problem is with httpd. Please go to /opt/CA/VirtualAppliance/custom/apache-ssl-certificates directory and run the following 2 commands to compare the modulus of the cert and the key

openssl x509 -noout -modulus -in localhost.crt | openssl md5
openssl rsa -noout -modulus -in localhost.key | openssl md5

The results have to be the same. If no then something wrong with the cert or key.

------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Hi Widjaja,

Yes - the results are exactly the same 

• openssl x509 -noout -modulus -in localhost.crt | openssl md5
  (stdin)= 688982825eba3d9992fc72eb44c74b8c
• openssl rsa -noout -modulus -in localhost.key | openssl md5
  (stdin)= 688982825eba3d9992fc72eb44c74b8c

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[tjmuchenje]

Error in /etc/httpd/logs/ssl_error_log

[debug] ssl_engine_kernel.c(1889): OpenSSL: Read: SSLv3 read client certificate A

[Fri Oct 02 10:32:32 2020] [debug] ssl_engine_kernel.c(1908): OpenSSL: Exit: failed in error

[Fri Oct 02 10:32:32 2020] [info] [client 10.10.247.48] SSL library error 1 in handshake (server CA_IMAG_VAPP:443)

[Fri Oct 02 10:32:32 2020] [info] SSL Library Error: 336151574 error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown

[Fri Oct 02 10:32:32 2020] [info] [client 10.10.247.48] Connection closed to child 4 with abortive shutdown (server CA_IMAG_VAPP:443)

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[tjmuchenje]

Hi Widjaja,
 Could this be the reason for my problem?

openssl s_client -connect test.local:443

CONNECTED(00000003)

depth=0 C = ZA, ST = KwaZulu Natal, L = Durban, O = ACME, CN = test.local, emailAddress = [e-mail]

verify error:num=20:unable to get local issuer certificate

What do you think?



------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,

That error merely says that openssl doesn't know the root certificate who issues the certificate you are trying to see using openssl. It is not necessary relevant to the problem you are having.
If you restore the default vApp's localhost.crt and localhost.key files, do you have the problem? If not then definitely something with your certificate/key, can you test that to further narrow down the problem?
If it is something with the certificate then you may want to compare the cert with default vApp's cert.

 

------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

I hear you .

The weird thing here is Identity Portal and Identity Sigma work find with the same certificate.
It's only Identity Manager that logging the user out after authenticating.

Regards
Tav

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[tjmuchenje]

Hi Widjaja,

The default vApp's localhost.crt and localhost.key files expired two years ago. I have put in the cert that was working before the upgrade and we still seeing the problem. 

Regards
Tavernt

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

[Widjaja Sangtoki]

Hi Tav,

I have SSHed to my vApp in the lab and run the following command

   openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt

I have keyed in the prompts appropriately and the key and cert files, i.e. localhost.key and localhost.crt files were created. I have backed up the original files under /opt/CA/VirtualAppliance/custom/apache-ssl-certificates and replace them with the ones I newly created. I have restarted httpd

    service httpd stop
    service httpd start

And I can access IM normally via httpd. Can you try and verify?

------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Hi Widjaja, 

Thank you very much for all your help. Unfortunately still not working.

I have attached a shot clip to show you the behavior of the system.  

https:<servername>/iam/im/identityEnv/?

1. I login with the correct password. Its refreshes the log in page and clears the fields
2. I enter the wrong password and it does show "Use not authenticated". So authentication works
3. I enter the right password and it takes me in but when i click on any menu item. I get logged out immediately and sent back to the login page.


If I use https:<servername>:8443/iam/im/identityEnv/? or http:<servername>:8080/iam/im/identityEnv/? all works fine...

Kind Regards
Tav
  



------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------

 Video

• edd3af3d-fec1-4caf-8bd4-25b376d04d17_file

[Widjaja Sangtoki]

Hi Tav,

I would suggest you can raise a Support call ticket to investigate this issue further.


------------------------------
Regards,
Widjaja
====================
Perhaps there are others in the communities who have experience in doing this and we invite them to comment here also.

Another option may be to reach out to our partner HCL Technologies to see in what way they can assist further. The Enterprise Studio team of HCL can be reached at [e-mail]. https://www.hcltech.com/enterprise-studio
------------------------------

[tjmuchenje]

Yes, we have a ticket open but no solution yet. 
Thank you very much Widjaja

------------------------------
Snr IAM Architect
I'CURITY SOLUTIONS
------------------------------