Tech Note: email blocked by API Portal 4.2 internal use of subnets 10.0.0.0 and 10.0.1.0

[Mark ODonohue]

Issue:

We have configured the Our Test Portal server to use our SMTP host as per the documentation.

The Portal server IP has been added to the SMTP relay allow list. However we are unable to send any emails.

 

it turns out the IP address of the SMTP server was 10.0.1.X  and this is also an IP subnet that is used by the api portal docker images. 

The solution here was to add another SMTP server with a different IP address,  it may also be possible to reassign the subnets being used by the docker images, but we didn't test that option. 

Environment:

API Portal 4.2

Cause:

There was a conflict in subnet specification.  

In this case the email server was in the range 10.0.1.X we found that the docker container could resolve the mail server name to the correct IP address, but that because docker was using the subnet 10.0.1.X it was not possible to route to the external 10.0.1.X mail server machine. 

In setup of docker it is using internally three created networks -when we connect to the container trying to connect to the SMTP server  : 
 

docker exec -it 143701b52e37 sh

Then run the route command : 

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.18.0.1      0.0.0.0         UG    0      0        0 eth1
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth2
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth1

 

Unfortunately one of them is the 10.0.1.X network, which blocks access to the external subnet 10.0.1.X, and hence blocks access to the mail server.   In the container, the DNS resolution works and we get the right IP for the mail server,
 

To print out details of all the networks used in docker, the following command can be run from the portal machine : 

for i in `docker network ls | awk '{ print $1 }' `; do echo $i; (docker network inspect $i ); done
 

Resolution:

In this case, we were able to setup SMTP server in another subnet.

Alternatively it may be possible to change the subnet used for the API Portal docker  as per : 

https://docops.ca.com/ca-api-developer-portal-enhanced-experience/4-2/en/install-configure-and-upgrade/troubleshoot-the-installation

This documents the setup of the 172.18.0.0 network, but may also be applicable to the 10.0.0.0 and 10.0.1.0 networks as well - in our case we did not need to go that far as we had an alternative.

 

Additional Information:

# To find which image did the SMTP message : we used : 

While
    journalctl -f 
will give trace of all the logs (assuming /etc/docker/daemon.json is setup correctly) - but it will not identify which container the logs come from. 

 

Here we needed to find which container was making the SMTP call - so we did this : 

 for i in `docker ps | awk '{ print $1 }' `; do echo $i; (docker logs $i 2>&1 | grep SMTP); done

 

This gave us the container :  143701b52e37 as the container trying to connect to the mail server.

143701b52e37

Jul 19 05:34:28 islandportal.example.com dockerd[27108]: WARNING: 4: Unable to send email: Unknown SMTP host: mail.example.com.  Exception caught!

 

Then we could connect to that container :

docker exec -it 143701b52e37 sh

to print out the network and test resolving the SMTP host name:

 

 

 

Link to the knowledge doc: 

https://comm.support.ca.com/kb/unable-to-send-emails-2/kb000107567