For a https connection to an external service we have the intermediate certificate (which signed the server certificate of the external service) in the api gateway trust store. This works fine.
Unfortunately these intermediate certificates changes a lot, which causes backend routing errors.
That's why I tried to import only the root ca (which signed the intermediate certificates) for the https connection. But It is not able to validate the server certificate with the Root ca even though the backend server provides the whole chain until the root ca.
Is this even possible or do I always have to add intermediate CAs?
PS: I already tried out every option (Import as trust anchor, import certification chain) but it didn't help...
I'm curious, do you have an expired intermediate cert or existing intermediate cert that matches the cert in question?
Currently, we had exactly the case, that an existing intermediate cert got changed even though it wasn't expired. And that's why we had to trust the new one in order to be able to send requests again.
For the Server cert to be implicitly trusted, the Intermediate cert that signed the server cert has to exist in the store.
Based on the use case, since the intermediate changes, the chain of implicit trust is now broken and the server cert is essentially a self signed cert. For the chain of trust to be re-established, you will have to generate a new CSR from the server cert and then have the new intermediate cert sign the CSR, then replace the signed cert in Gateway's Manage Certificates.
For your use case when the intermediate changes, do you get a new server cert where the new intermediate signed the server cert? Would you be able to elaborate on the steps that occur when new intermediates are generated?
Yes, that was exactly what I did.
That's why I thought of a solution for avoiding this problem e.g. by trusting only the root certificate. But it seems this isn't possible in the API Gateway and I have to maintain it manually :/
I thought it was possible to do this by adding only the Root certificate and marking it as Trust Anchor. If done, gateway trusts all the certificates generated using the same Root CA.
I tried that, but when I test it, it always comes up with this error:
Problem routing to <url>. Error msg: Unable to obtain HTTP response from <url>: Certificate not verified. Caused by: Certificate [<cert name>] path validation and/or revocation checking failed
And the <cert name> is the intermediate certificate, which it cannot find in the trust store. So just trusting the root certificate does not seem to be enough.
When recieving a client certifciate to be used for mutaul authenticaiton you must have the issue of that client in your truststore (of the server) otherwise TLS protocol specification dictates that the cleint shall not send their certificate.
On the other side, when connecitng to a server, the client must have the root certificate of the server's certifciate chain in their trust store AND be able to construct the full trust chain (otherwise how do they know the root and server certifciate are related). There are two ways to get there, one is for the client to have the intermedate certifiate(s) in their truststore, the other is for the server to provide the trustchain allong with thier server certifciate during the handshake. On the gateway you can see this by checking your private keys, if the chain is in there then the gateway will send it to clients, if you only see the private key/certificate and not the chain then the chain is not being provided to clients when they connect.
I know that is not exactly what you asked, but if you understand my explanation then you will see what can and cannot be done.
Thanks for your explanation, I think I understood it.
But there is something which is confusing. I created this thread because of an issue with a specific external server. Now after your explanations I looked it up in the private keys and I can see the whole chain from the server cert to the root ca. But this root ca is not the same like the one I have to trust for opening a connection. So we have the following minimally required certs for the mentioned external server:
Private Keys: client key (with cert chain to QuoVadis Global SSL --> QuoVadis Root CA)
Certificates: Intermediate cert (Symantec Class 3 Secure Server CA - G4)
So against your statement, these two chains are not the same (is this a special case?).
I also tested the chain by executing a openssl s_client -connect <ip:port> -showcerts and I can see the whole chain in this query (server cert, intermediate cert Symantec, Root cert Symantec) so the external server should send the symantec root ca to me and as far as I understand, it should be enough to only trust the root ca in this case. Is this correct?
Even i have the same thought. Add the root certificate presented by the target server you are routing request to and check option Signing Certificates for Outbound SSL Connections.
There are some useful text on the add certificate wizard. This is the method that we use to connect to all our servers internally which uses the same Root certificate and it works for us.
Signing Certificates for Outbound SSL Connections: Using HTTPS or LDAPS, the Gateway can connect to protected Web services or LDAPS directories hosted on SSL servers whose certificates were signed by this certificate authority. If several SSL server certificates are signed using an in-house certificate authority, then the Gateway can connect to all of them if this option is used.
I think the disconnect between what you are expiriencing and my explanation is that your gatway is acting as the client and the trust chain that you can see on your private key is what the gateway delivers to it's clients when they connect. Your backend server should do the same (deliver it's server certificate chain) to the gateway when the gateway connects to that backend, if it did then you would not have to import the new intermediate; the gateway would only need to have the root in its truststore. So the automatic acceptance of intermediates of servers to which you are routing is dependant on settings of the backend server itself, not something you can control from the gateway.
I see now where the root of the problem is: When executing a openssl s_client -connect <ip:port> -showcerts, I was always thinking that the whole chain until the root cert is provided but unfortunately they just send the server and the intermediate cert in the format:
The root cert is only mentioned as issuer ca of the intermediate cert but not delivered in the format above. So in fact I really only get the server and intermediate cert back and I see that there is no other way than trusting the intermediate cert in this case.
But now I know that if the backend server would send the whole chain correctly, I could only trust the root ca and it would work
Thank you all for your explanations!