DLR BGP Advertise /32 addresses

[aqualityplacem]

I am not sure if this is a unique situation, but we are migrating a number of virtual machines to a vCloud Director environment using NSX. Network isolation works slightly differently on the current platform, but all VM's have an external IP addresses on a /24 network. These are currently advertised from some third party routers to our core routers as /32's.

We have set up a DLR with an uplink which we are using for BGP to our core networks. We then have a single LIF which uses our internet facing /24 prefix, we have used .254 on this interface (not currently in use anywhere on the network). Each customer will have their own edge gateway connected to the DLR which we are treating as an external network. The Edge Gateways will use the DLR as the default gateway.

As we move a customer machine we will be configuring their edge gateway with their external IP address. As soon as we power down the VM on the old platform the third party routers stop advertising the address, so no problems there.

The problem is I don't want NSX to advertise the entire /24 and mess up customers we haven't moved yet.

I have added the /32 to the "IP Prefixes" list on the DLR, but nothing seems to get advertised to our core routers. The only way I can get the /32 network to be advertised is to add the /32 as a static route using the same /32 address as the next hop. On the "route distribution table" I can create an entry which uses the IP Prefix and also tick the static route tick box. Without that selected I cant get the DLR to advertise the route of the edge gatewat, regardless of whats configured in the prefix list or distribution table.

I have done the same to advertise the IP we are using on the LIF of the DLR and selected "Connected" on the route distribution table. This seems to work without any problems

The edge gateway address is correctly advertised to the core routers, but we don't get any external connectivity. I suspect the static route on the DLR isn't helping us, or maybe its just something odd with the Edge Gateway not responding

Has anyone else done anything similar or have any advise on advertising /32's

[cnrz]

Addition of IP Prefixes to the Prefix List or Redistribution table creates a Filter that selects which entries of the Routing Table is redistributed to dynamic routing Protocol (BGP is configured).  If the routing table is empty, then even there are entries they don't match the routing table, so are not redistributed. In order to populate the routing table, statci or dynamic routing can be used.  If the DLR and the Customer ESGs don't have BGP configured between them (and this may not be possible if the DLR LIFs connected to ESGs are of type Internal), the only way to redistribute is to enter /32 static routes manually pointing to the next-hop of ESG External IP address for this specific Customer.  (or dynamically as a similar scenario for understanding the VM has been migrated to new ESG Side and enter /32 static route automatically https://networkinferno.net/ingress-optimisation-with-nsx-for-vsphere )

Why is the next-hop of the /32 static entry points to itself and not  the IP address of the Edge Uplink Interface?

If there is NAT configured on the EDGEs converting the VM IP to the External Network this may be helpful: For example if the Edge Uplink IP address is 192.168.10.1/24, and VM IP address is 172.16.10.11/24, it may be NATed to 192.168.10.11. The static route may be as 192.168.10.11/32 pointing to 192.168.10.1.

[aqualityplacem]

hank you for the reply. As long as us adding the /32 static routes is supported we dont have any problems. Trying to avoid ospf/bgp between the customer edges and our DLR.

When I said its self I mean the static route for the edge gateway would be 192.168.0.1/32 next hop 192.168.0.1  (DLR IP is 192.168.254)

We have a ticket open with VMware, we only just upgraded to NSX 6.2.4 I am wondering if we have stumbled across some crazy edge/drl problem

We can now ping the DLR externally but we cant ping 8.8.8.8 from it. If we ping the edge from an external machine we can even see the traffic reach it using show network flow but the edge/dlr is just not managing to get back to us

DLR seems to have the default-orginate routes advertised to it all OK as well.

Ill update if we manage to fix, but if you have any other ideas please let me know

[aqualityplacem]

we have since put a windows machine behind the DLR (same network as the external interface of the edge gateway) and routing to this works without any issue.

The edge gateway was set with the same gateway as our test windows VM, so looks like we have something odd, or some issue with the firewall on the edge

[cnrz]

Next-hop should be the IP Address of the Edge Uplink Interface. IIt could be valid if  NATed IP is the same as Edge Uplink Ip, but if I understood correctly each VM has a different external IP address so this is not the case.

192.168.0.1/32 next hop 192.168.0.1  (DLR IP is 192.168.254) -->  External IP of the VM (the NATed IP) should be different thatn next hop IP, like 192.168.0.2. So 192.168.0.2/32 next hop 192.168.0.1 could be bette syntax. For some physical routers they may even reply with Proxy arp for NATed IP, but not sure for Edge if it behaves the same. Also for NAT to function Firewall on the Edge should be enabled, and rule should exist allowing source 192.168.0.2 icmp.

This article may be helpful for configuration of the Edge Firewall with NAT.  (Example gives physical IP address but for current use it may be 192.168.0.2 external IP). Both SNAT and DNAT is configured simultaneously, so if the current problem is icmp then it may be different if the ICMP is from inside-->external or external--> inside

http://www.routereflector.com/2015/01/configuring-nat-and-firewall-on-a-nsx-edge-router/

[aqualityplacem]

thank you for the clarification on the required routes to allow the advertisements. This is how we have it setup and can confirm it works.

Problem with us being unable to ping wasn't related to the advertisements unfortunately. I was a bit inpatient, as the system wsa not in production I removed the whole config, transport zones the lot - rebuilt it and it started to work.... :smileyhappy: hopefully just one of those things