NSX-V redundancy in route-based IPSec VPN

[MihailsA]

I've been playing with IPSEC VPN connections between on-prem NSX-V and AWS VPC. As AWS Site-to-Site VPNs are route-based VPS, I've set up NSX part of the VPN tunnel as a route-based VPN. Everything works fine as far as I can see. Now, as a last step I'm looking to making this connection redundant. AWS side has two tunnels available, with two separate endpoints and two separate VTIs. That is the exact architecture as we see in NSX documentation about NSX route-based VPNs:

Route-Based IPSec VPN

The Site A looks exactly like Amazon, and Site B as our NSX part. Unfortunately, there is no procedure describing configuring this tunnel redundancy on NSX Edge. REST API guide is not clear on how to make this thing work. I tried creating this as two sites (because you configure external endpoints at site level within the XML) - didn't work. There is an option to configure two <tunnelInterface> instances, but these only appear to have just the VTI part that is unique, and they share the same remote endpoint - not really what we need.

Has anybody ever managed to deploy this configuration?

We can't, unfortunately, achieve it by using two separate Edge routers, because AWS VPN has one-to-one relationship between it's VPCs-VPG-Customer Gateway, so one AWS internal network - one remote endpoint (like exactly as it is in the picture).

[MihailsA]

Ok, I think I got it right finally. I had to create two separate Sites, one for each AWS VPN tunnel:

On NSX Edge the VPN configuration looks like this:

Both tunnels are up and running:

It's worth noting that AWS considers Tunnel 1 to be the primary connection to use and Tunnel 2 is utilized only when Tunnel 1 is down. It achieves this by not advertising BGP routes to VPC's IP CIDR blocks via VTI of Tunnel 2, so our case the route to VPC's is only available via 169.254.192.145 (AWS side VTI's IP address). As soon as the IPSEC via Tunnel 1 is down, it start advertising the route via 169.254.249.253.

In case anybody need help building route-based VPN from NSX to AWS, there is a good blog that covers this topic: https://vmguru.com/2016/11/connecting-a-vpn-between-aws-and-vmware-nsx/

https://vmguru.com/2016/11/connecting-a-vpn-between-aws-and-vmware-nsx/

However, to make use of redundancy of AWS VPN's tunnels, you'd have to add an additional <site></site> section in your REST API body that would set up the second connection.

Mine looked like this:

<?xml version="1.0" encoding="UTF-8"?>
<ipsec>
<version>38</version>
<enabled>true</enabled>
<disableEvent>false</disableEvent>
    <logging>
        <enable>true</enable>
        <logLevel>warning</logLevel>
    </logging>
<sites>
<site>
    <enabled>true</enabled>
    <name>Amazon_VPC_1</name>
    <description>Route-based VPN to Amazon</description>
    <localId>217.28.xxx.xxx</localId>
    <localIp>217.28.xxx.xxx</localIp>
    <peerId>13.48.xxx.xxx</peerId>
    <peerIp>13.48.xxx.xxx</peerIp>
    <ipsecSessionType>routebasedsession</ipsecSessionType>
    <tunnelInterface>
        <ipAddress>169.254.192.146/30</ipAddress>
        <mtu>1500</mtu>
    </tunnelInterface>
    <encryptionAlgorithm>aes256</encryptionAlgorithm>
    <enablePfs>true</enablePfs>
    <dhGroup>dh2</dhGroup>
    <localSubnets>
        <subnet>0.0.0.0/0</subnet>
    </localSubnets>
    <peerSubnets>
        <subnet>0.0.0.0/0</subnet>
    </peerSubnets>
    <psk>YoUr.FiRsT.kEy.HeRe</psk>
    <authenticationMode>psk</authenticationMode>
    <ikeOption>ikev1</ikeOption>
    <digestAlgorithm>sha1</digestAlgorithm>
    <responderOnly>false</responderOnly>
</site>
<site>
    <enabled>true</enabled>
    <name>Amazon_VPC_2</name>
    <description>Route-based VPN to Amazon</description>
    <localId>217.28.xxx.xxx</localId>
    <localIp>217.28.xxx.xxx</localIp>
    <peerId>13.48.xxx.xxx</peerId>
    <peerIp>13.48.xxx.xxx</peerIp>
    <ipsecSessionType>routebasedsession</ipsecSessionType>
    <tunnelInterface>
        <ipAddress>169.254.249.254/30</ipAddress>
        <mtu>1500</mtu>
    </tunnelInterface>
    <encryptionAlgorithm>aes256</encryptionAlgorithm>
    <enablePfs>true</enablePfs>
    <dhGroup>dh2</dhGroup>
    <localSubnets>
        <subnet>0.0.0.0/0</subnet>
    </localSubnets>
    <peerSubnets>
        <subnet>0.0.0.0/0</subnet>
    </peerSubnets>
    <psk>YoUr.SeCoNd.KeY.hErE</psk>
    <authenticationMode>psk</authenticationMode>
    <ikeOption>ikev1</ikeOption>
    <digestAlgorithm>sha1</digestAlgorithm>
    <responderOnly>false</responderOnly>
</site>
</sites>
<global><psk></psk><caCertificates/><crlCertificates/></global>
</ipsec>