VMware NSX

Expand all | Collapse all

Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

Mauricio Queiroz Santos Amorim

Mauricio Queiroz Santos AmorimJul 14, 2020 10:49 AM

  • 1.  Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 10, 2020 05:27 PM

    Even after enabling BGP on UI on Tire0 Gateway, still on the CLI, it shows as BGP as "Administratively shut down".

    I am able to ping neighbor address from nsxt-edge1(tier0_sr). Only BGP shows as "Administratively shut down"

    Can anyone pls let me know, how to fix this?

    Tire-1 gateway is working fine.

    nsxt-edge01(tier0_sr)> get bgp neighbor

    BGP neighbor is 192.168.100.1, remote AS 65100, local AS 65000, external link

    Administratively shut down

      BGP version 4, remote router ID 0.0.0.0, local router ID 192.168.100.2

      BGP state = Idle

      Last read 19:53:42, Last write never

      Hold time is 180, keepalive interval is 60 seconds

      Configured hold time is 180, keepalive interval is 60 seconds

      Graceful restart information:

        Local GR Mode  : Helper*

        Remote GR Mode : NotApplicable

        R bit          : False

        Timers :

         Configured Restart Time(sec)  : 180

         Received Restart Time(sec)    : 0

      Message statistics:

        Inq depth is 0

        Outq depth is 0

                             Sent       Rcvd

        Opens:                  0          0

        Notifications:          0          0

        Updates:                0          0

        Keepalives:             0          0

        Route Refresh:          0          0

        Capability:             0          0

        Total:                  0          0

      Minimum time between advertisement runs is 0 seconds

      Update source is 192.168.100.2

    For address family: IPv4 Unicast

      Not part of any update group

      Community attribute sent to this neighbor(all)

      0 accepted prefixes

      Connections established 0; dropped 0

      Last reset never

    BGP Connect Retry Timer in Seconds: 10

    Read thread: off  Write thread: off

    nsxt-edge01(tier0_sr)> ping 192.168.100.1 source 192.168.100.2 repeat 6

    PING 192.168.100.1 (192.168.100.1) from 192.168.100.2: 56 data bytes

    64 bytes from 192.168.100.1: icmp_seq=0 ttl=64 time=2.020 ms

    64 bytes from 192.168.100.1: icmp_seq=1 ttl=64 time=2.382 ms

    64 bytes from 192.168.100.1: icmp_seq=2 ttl=64 time=2.396 ms

    64 bytes from 192.168.100.1: icmp_seq=3 ttl=64 time=1.961 ms

    64 bytes from 192.168.100.1: icmp_seq=4 ttl=64 time=1.687 ms

    64 bytes from 192.168.100.1: icmp_seq=5 ttl=64 time=1.891 ms

    --- 192.168.100.1 ping statistics ---

    6 packets transmitted, 6 packets received, 0.0% packet loss

    round-trip min/avg/max/stddev = 1.687/2.056/2.396/0.257 ms



  • 2.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 10, 2020 06:27 PM

    Can you perform a BGP debug on both the routers ?

    Also please provide get interfaces output from tier-0



  • 3.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 10, 2020 07:46 PM

    Thanks for the reply. Pls find the details of get interfaces....

    On the other end (router), i did the debug, actually, it is establishing TCP connection and the other end, sends the BGP-OPEN message, then NSX sends TCP reset message...

    Other end (router) is trying to establishing the TCP and sending BGP OPEN message, but NSX-T send TCP reset message, and not sending any BGP OPEN message...

    its is like there is not BGP configured on NSX-T tire-0 gateway... On tire-0 it shows as "Administratively shutdown" , even with BGP enabled on UI...

    On NSX-T, Snippet of the uplink interface, where BGP connection trying to establish:

    =================================================================

        Interface     : 5093c7c7-b3ad-400e-9782-583a4047eae2

        Ifuid         : 288

        Name          : uplink01

        Fwd-mode      : IPV4_ONLY

        Internal name : uplink-288

        Mode          : lif

        Port-type     : uplink

        IP/Mask       : 192.168.100.2/24

        MAC           : 00:50:56:b6:c2:38

        VLAN          : None

        Access-VLAN   : None

        LS port       : de6659c2-b0af-4862-ba71-d5e1d02af763

        Urpf-mode     : STRICT_MODE

        DAD-mode      : LOOSE

        RA-mode       : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)

        Admin         : up

        Op_state      : up

        MTU           : 9000

    Complete Output on vrf:

    -----------------------------

    nsxt-edge01(tier0_sr)> get interfaces

    Logical Router

    UUID                                   VRF    LR-ID  Name                              Type                      

    f2dbfc36-b3df-4b9f-8c50-95107e4fea02   5      2052   DR-Tire0-GW                       DISTRIBUTED_ROUTER_TIER0  

    Interfaces (IPv6 DAD Status A-Assigned, D-Duplicate, T-Tentative)

        Interface     : 1eb95b92-3457-442a-861d-1919ba8b8d77

        Ifuid         : 301

        Name          : Tire0-GW-Tire1-GW-t0_lrp

        Fwd-mode      : IPV4_ONLY

        Internal name : downlink-301

        Mode          : lif

        Port-type     : downlink

        IP/Mask       : 100.64.112.0/31;fc7c:29ed:e1ac:d000::1/64(NA);fe80::50:56ff:fe56:4452/64(NA)

        MAC           : 02:50:56:56:44:52

        VNI           : 71689

        Access-VLAN   : None

        LS port       : d895832e-c514-48bf-820b-d3bb875c1cc6

        Urpf-mode     : PORT_CHECK

        DAD-mode      : LOOSE

        RA-mode       : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)

        Admin         : up

        Op_state      : up

        MTU           : 1500

        Interface     : 41a82333-39b2-45e1-96aa-9c749805ca88

        Ifuid         : 296

        Name          : bp-dr-port

        Fwd-mode      : IPV4_ONLY

        Mode          : lif

        Port-type     : backplane

        IP/Mask       : 169.254.0.1/25;fe80::50:56ff:fe56:4452/64(NA)

        MAC           : 02:50:56:56:44:52

        VNI           : 71690

        Access-VLAN   : None

        LS port       : 8c064dea-e15b-43a5-b789-bd437eb0c7b9

        Urpf-mode     : PORT_CHECK

        DAD-mode      : LOOSE

        RA-mode       : RA_INVALID

        Admin         : up

        Op_state      : up

        MTU           : 1500

        Interface     : 1c535e28-1211-5c61-af7c-7f01c834d3d4

        Ifuid         : 293

        Mode          : cpu

        Port-type     : cpu

        Interface     : 57306016-9058-5171-82fc-0559d6e6e108

        Ifuid         : 294

        Mode          : blackhole

        Port-type     : blackhole

    Logical Router

    UUID                                   VRF    LR-ID  Name                              Type                      

    e8aae4b9-d9bf-4966-9721-4ba970bdc53f   3      2053   SR-Tire0-GW                       SERVICE_ROUTER_TIER0      

    Interfaces (IPv6 DAD Status A-Assigned, D-Duplicate, T-Tentative)

        Interface     : 56d780ea-31b0-501a-9030-310b4b5fd645

        Ifuid         : 281

        Mode          : cpu

        Port-type     : cpu

        Interface     : 610111d2-abb6-57c1-a8ef-31418ac71cc4

        Ifuid         : 282

        Mode          : blackhole

        Port-type     : blackhole

        Interface     : 24420827-5a60-424d-ae7b-4240ad08849f

        Ifuid         : 286

        Name          : sr0-internal-routing-port

        Fwd-mode      : IPV4_ONLY

        Internal name : inter-sr-286

        Mode          : lif

        Port-type     : internal-routing

        IP/Mask       : 169.254.0.130/25;fe80::50:56ff:fe56:5201/64(NA)

        MAC           : 02:50:56:56:52:00

        VNI           : 71687

        Access-VLAN   : None

        LS port       : 42fd04d2-90a3-4194-a595-88f0432db89a

        Urpf-mode     : PORT_CHECK

        DAD-mode      : LOOSE

        RA-mode       : RA_INVALID

        Admin         : up

        Op_state      : up

        MTU           : 1500

        Interface     : a3f81c59-d705-4524-b0a6-78890102abfd

        Ifuid         : 290

        Name          : bp-sr0-port

        Fwd-mode      : IPV4_ONLY

        Internal name : downlink-290

        Mode          : lif

        Port-type     : backplane

        IP/Mask       :

        MAC           : 02:50:56:56:53:00

        VNI           : 71690

        Access-VLAN   : None

        LS port       : a73cbb1c-986d-4258-86b3-c04eff9afac0

        Urpf-mode     : NONE

        DAD-mode      : LOOSE

        RA-mode       : RA_INVALID

        Admin         : up

        Op_state      : down

        MTU           : 1500

        Interface     : 5093c7c7-b3ad-400e-9782-583a4047eae2

        Ifuid         : 288

        Name          : uplink01

        Fwd-mode      : IPV4_ONLY

        Internal name : uplink-288

        Mode          : lif

        Port-type     : uplink

        IP/Mask       : 192.168.100.2/24

        MAC           : 00:50:56:b6:c2:38

        VLAN          : None

        Access-VLAN   : None

        LS port       : de6659c2-b0af-4862-ba71-d5e1d02af763

        Urpf-mode     : STRICT_MODE

        DAD-mode      : LOOSE

        RA-mode       : SLAAC_DNS_TRHOUGH_RA(M=0, O=0)

        Admin         : up

        Op_state      : up

        MTU           : 9000

        Interface     : a812a72c-27a6-4152-9904-f90a0f5d2272

        Ifuid         : 291

        Mode          : loopback

        Port-type     : loopback

        IP/Mask       : 127.0.0.1/8;::1/128(NA)

    Output on tire-0 box: (removed eth0 IP details from the output, rest of them, are all correct)

    --------------------------------------------------------------------------------------------------------------

    nsxt-edge01> get interfaces

    Interface: bond0

      Address: unknown

      MAC address: be:7c:bb:9a:76:d4

      MTU: 1500

      Broadcast address: None

      KNI: False

      Bond mode: ROUND_ROBIN

      Bond slaves:

      Link status: down

      Admin status: down

      RX packets: 0

      RX bytes: 0

      RX errors: 0

      RX dropped: 0

      TX packets: 0

      TX bytes: 0

      TX errors: 0

      TX dropped: 0

      TX collisions: 0

    Interface: eth0

      Address: [REMOVED FROM OUTPUT]

      MAC address: 00:50:56:b6:0f:ca

      MTU: 1500

      Default gateway: [REMOVED FROM OUTPUT]

      Broadcast address: [REMOVED FROM OUTPUT]

      KNI: False

      Link status: up

      Admin status: up

      RX packets: 2498917

      RX bytes: 176892161

      RX errors: 0

      RX dropped: 0

      TX packets: 80014

      TX bytes: 18575173

      TX errors: 0

      TX dropped: 0

      TX collisions: 0

    Interface: fp-eth0

      ID: 0

      Link status: up

      MAC address: 00:50:56:b6:dc:b5

      MTU: 1600

      PCI: 0000:0b:00:00

      Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO RX_VLAN_STRIP RX_IPV4_CKSUM RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO

      Polling Status: active

      Driver: net_vmxnet3

      Rx queue: 2

      Tx queue: 2

      Socket: 0

      RX packets: 9555

      RX bytes: 859890

      RX errors: 0

      RX badcrc: unknown

      RX badlen: unknown

      RX misses: 0

      RX nombufs: 0

      RX pause xoff: unknown

      RX pause xon: unknown

      TX packets: 139993

      TX bytes: 5879706

      TX errors: 0

      TX pause xoff: unknown

      TX pause xon: unknown

    Interface: fp-eth1

      ID: 1

      Link status: up

      MAC address: 00:50:56:b6:c2:38

      MTU: 1600

      PCI: 0000:13:00:00

      Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO RX_VLAN_STRIP RX_IPV4_CKSUM RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO

      Polling Status: active

      Driver: net_vmxnet3

      Rx queue: 2

      Tx queue: 2

      Socket: 0

      RX packets: 2820

      RX bytes: 247591

      RX errors: 0

      RX badcrc: unknown

      RX badlen: unknown

      RX misses: 0

      RX nombufs: 0

      RX pause xoff: unknown

      RX pause xon: unknown

      TX packets: 8192

      TX bytes: 539708

      TX errors: 0

      TX pause xoff: unknown

      TX pause xon: unknown

    Interface: fp-eth2

      ID: 2

      Link status: up

      MAC address: 00:50:56:b6:2c:cc

      MTU: 1500

      PCI: 0000:1b:00:00

      Offload Capabilities: TX_VLAN_INSERT TX_UDP_CKSUM TX_TCP_CKSUM TX_TCP_TSO RX_VLAN_STRIP RX_IPV4_CKSUM RX_UDP_CKSUM RX_TCP_CKSUM RX_TCP_LRO

      Polling Status: active

      Driver: net_vmxnet3

      Rx queue: 2

      Tx queue: 2

      Socket: 0

      RX packets: 3059847

      RX bytes: 205469196

      RX errors: 0

      RX badcrc: unknown

      RX badlen: unknown

      RX misses: 0

      RX nombufs: 0

      RX pause xoff: unknown

      RX pause xon: unknown

      TX packets: 0

      TX bytes: 0

      TX errors: 0

      TX pause xoff: unknown

      TX pause xon: unknown



  • 4.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 10, 2020 09:07 PM

    Interface config looks fine. Well i believe its AS number issue . You should cross check if BPG peering config AS is correct on both the sides

    As per your config remote AS is 65100 and local AS is 65000



  • 5.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 10, 2020 09:30 PM

    I cross-checked, configs on router and its correct....

    Router side:

    ------------------

    local-as: 65100

    peer-as: 65000

    tier-0 (NSX-T):

    --------------------

    local-as: 65000

    peer-as: 65100

    I am puzzled, why on tier-0 it shows BGP peer as "Administratively shutdown", even with UI shows as BGP enabled...

    Even the ping between two peers are successful on both the side...

    Any idea, under what all the scenarios, on nsx-t (tire-0 GW), it shows BGP peer as "Administratively shutdown"?



  • 6.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 12, 2020 01:06 PM

    If possible please share the screenshots of Interface and BGP config from U.I as well.



  • 7.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 13, 2020 05:50 PM

    Thanks for looking into this issue... Please find the details below:

    Note: I tried with BFD enabled and disabled, in both case, BGP neighbor shows as "Administratively shut down"

    With different VIew:

    ------------------------------



  • 8.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 14, 2020 10:49 AM

    Do you have a single Edge node in the Edge cluster where this T0 is on?



  • 9.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 14, 2020 05:13 PM

    Yes, currently I have single Edge Node in the Edge cluster. Will having two edge-node in the same cluster will solve the BGP - "Administratively shut down" issue?

    Thanks.



  • 10.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 14, 2020 06:53 PM

    It should work even with a single edge node in the cluster.

    There is an option to administratively disable a BGP neighbor and it seems that it was switched on. This is the only way I found to reproduce what you have. This switch is only available in the Advanced Networking and Security (NSX 2.5 and below) or on Manager mode on NSX-T 3.0+.

    The steps below were tested on NSX-T 3.0:

    If you do not have the Policy/Manager button in the upper right corner go to System > User Interface Settings (Last option) and Toggle Visibility so that it appears.

    Then go to Networking (click and make sure the Manager option in the upper right corner is selected)

    Click on Tier-0 Logical Routers > Click on the T0

    Under Routing > BGP > select the neighbor in the lower part of the screen and click on EDIT

    There will be an option to change de Admin Status > Change to enable

    Hope this helps



  • 11.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 14, 2020 10:41 PM

    Thanks for looking into this issue... yeah, i have verified the same on UI and the BGP neighbor admin-status is "Enabled:... Still no luck... on the CLI it shows BGP "Administratively shutdown"...

    Please find the details below:

    Does this related to any physical NIC supportablility? I tried with both 10G and 1G physical NIC too...

    But, i am able to successfully ping between the neighbor router and Tire-0 interface.

    With different VIew:

    ------------------------------

     



  • 12.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 15, 2020 01:37 PM

    Does your edge node have any alarms?



  • 13.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 20, 2020 06:41 PM

    Sorry for the late reply. I am seeing the below behavior:

    Please let me know, if there is anything I am missing during my config? I am not sure, why my BGP is "Administratively shut down", as soon as I attach VLANs (either to Tire-0 or Tire-1 or VRF)

    Note: I have VMs attached to VLAN segments.

    Case-1) If I do not attach any VLAN segments to either Tire-0 or Tire-1 or VRF, then BGP session is up between Tire-0 and external-router.

    Case-2) If I attach VLAN segment to any node (either Tire-0 or Tire-1 or VRF), then the BGP on Tire-0 goes to "Administratively shut down"

    Note: I have NOT configured BFD...

    Reg Alarms, I see the below in the NSX-Manager:

    ==========================================

    1)

    Routing Routing Down nsxt-edge1

    Open

    Description

    All BGP/BFD sessions are down.

    Recommended Action

    Invoke the NSX CLI command `get logical-routers` to get the tier0 service router and switch to this vrf, then invoke the following NSX CLI commands. 1. `ping <BFD peer IP address>` to verifyconnectivity. 2. `get bfd-config` and `get bfd-sessions` to check if BFD is running well. 3. `get bgp neighbor summary` to check if BGP is running well. Also check /var/log/syslog to see if there are any errors related to BGP connectivity.

    2)

    Infrastructure Communication  Edge Tunnels Down  nsxt-edge1  nsxt-edge1

    Open

    Description

    The overall tunnel status of Edge node 01054703-43cc-4348-93b8-be2c9d38aded is down.

    Recommended Action

    Invoke the NSX CLI command `get tunnel-ports` to get all tunnel ports, then check each tunnel's stats by invoking NSX CLI command `get tunnel-port <UUID> stats` to check if there are any drops. Also check /var/log/syslog if there are tunnel related errors.

    3)

    Routing   BGP Down   nsxt-edge1

    Description

    In Router 66e152c1-606c-49e8-a89a-6c25e46fea9a, BGP neighbor ip_address:192.168.100.1 is down, reason: Network or config error.

    Recommended Action

    1. Invoke the NSX CLI command `get logical-routers`. 2. Switch to service-router ed9bf441-e57f-4ce7-a69a-26cec15fa5cf. 3. Invoke the NSX CLI command `get bgp neighbor summary` to check the BGP neighbor status. 4. Check /var/log/syslog to see if there are any errors related to BGP connectivity.



  • 14.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 20, 2020 07:52 PM

    Are you attaching VMs to the same VLAN as the uplinks of the Tier0? Please send us some screenshots of your Edge node config.



  • 15.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 20, 2020 08:58 PM

    When I create VM on vcenter, I use the segment (overlay-segment). Pls see the below screen-shot for "app90" name...

    As soon as I create VM with "app90" as network adaptor (and attach to VRF), then BGP goes to "Administratively shut down".

    I am attaching the VM to the segment (overlay-segment).

    vlan-segment is attached to the Tier-0 interface (and NOT creating any VM on this vlan-sgment). This is just for uplink (connection to router for BGP session)...

    Please find the config below and let me know, if you need more details:

    ==========================================================

    overlay-segment:

    ---------------------

    vlan-segment:

    ------------------

    Tire-1:

    ------------

    Tire-0:

    ------------

    VRF:

    -----------



  • 16.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 20, 2020 09:01 PM

    overlay-segment:

    ---------------------



  • 17.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jul 20, 2020 09:02 PM

    vlan-segment:

    ------------------



  • 18.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Broadcom Employee
    Posted Jul 21, 2020 10:53 AM

    There are some problems with the screenshots that are not showing.

    Please don't forget to send the screenshots of the Edge node configuration and status (System > Fabric > Nodes > Edge Transport Nodes).



  • 19.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Dec 18, 2020 05:29 PM

    What was the resolution to this problem? We have the same issue in one environment (of 5 built).



  • 20.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Jan 19, 2021 07:38 PM

    For us the issue got resolved by making sure all the Tunnels for Edges are in UP state in NSX-T Manager



  • 21.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Aug 22, 2021 05:03 PM

    I was facing the same issue and just resolved it after 2hrs of troubleshooting.

    First, I thought BGP peer was not responding that I built on VYOS in my lab. Did multiple resets/clear/reconfiguration, but nothing happened.

    Then, I saw removed the T1 and BGP peer was established back. So, thought some problem with T0 to T1 connectivity.

    Thereafter, when I joined T0 to T1, BGP remained established, but this time Segment was not connected to T1. But when I connected the Segment, BGP went down again. So, I thought may be some problem with VM joining that segment. So, I removed the VM from that Segment and BGP came up again.

    So, then I checked the EDGE monitoring and seen the tunnels are down, and interestingly I noticed tunnel was established to wrong TEP IP of Transport compute node. Probably, not getting the IP from the Pool that I created, so this time I statically created some 8 TEP IP.

    And, tunnel came up, so as BGP.........



  • 22.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Apr 15, 2022 11:07 AM

    Still no luck. As soon as I attaching a logical segment to a VM the BGP goes down with the error Administrative shutdown.. 

    Is it any solutions?

    Thanks 

    Giovanni



  • 23.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Nov 26, 2022 11:49 AM

    Hello everyone,

    I have the same problem. I'm working on NSX-T data center Version 3.1.3.1.0.18504668

    BGP session is UP between T0 (172.28.50.2) and upstream router (172.28.50.1). The segment is attach on the T1.

    Upstream router can see the network I announce, but when i start a VM in this segment, bgp session go DOWN... "Administratively shut Down" when i do a get bgp neighbor command.

    If i shutdown the VM, the bgp session go UP.

    In the logs, locate at /var/log/syslog/

    root@nsx-edge-t0:/var/log# tail -f syslog | grep bgp
    2022-11-26T11:43:43.703Z nsx-edge-t0 NSX 25105 - [nsx@6876 comp="nsx-edge" subcomp="cli" username="admin" level="INFO"] CMD: get bgp neighbor
    2022-11-26T11:44:25.395Z nsx-edge-t0 NSX 2551 - [nsx@6876 comp="nsx-edge" subcomp="agg-service" tid="2937" level="INFO"] ExecCmd would call: /usr/bin/timeout -k 10 10 /usr/bin/vtysh -c "show bgp vrf all neighbors json"
    2022-11-26T11:44:42.604583+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="rcpm-db-pb" level="INFO"] num_redist_rules: 1, num_bgp_peers: 1, num_static_routes: 1, num_vrf_configs: 0 asn_change 0, num_ospf_interfaces: 0
    2022-11-26T11:44:43.805017+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="rcpm-db-pb" level="INFO"] num_redist_rules: 1, num_bgp_peers: 1, num_static_routes: 1, num_vrf_configs: 0 asn_change 0, num_ospf_interfaces: 0
    2022-11-26T11:44:44.609Z nsx-edge-t0 NSX 3215 - [nsx@6876 comp="nsx-edge" s2comp="nsx-monitoring" entId="a19573b8-e74e-4beb-a615-2e5c02ae2a11" tid="3293" level="ERROR" eventState="On" eventFeatureName="routing" eventSev="error" eventType="bgp_down"] Context report: {"entity_id":"a19573b8-e74e-4beb-a615-2e5c02ae2a11","sr_id":"2c7af594-58dc-40b4-ab6b-041740a36439","lr_id":"ee0b1cf9-e320-4826-9f0b-b67da06af6f6","bgp_neighbor_ip":"172.28.50.1","failure_reason":"Network or config error"}
    2022-11-26T11:44:44.609899+00:00 nsx-edge-t0 bgpd 5958 - - %NOTIFICATION: sent to neighbor 172.28.50.1 6/2 (Cease/Administratively Shutdown) 0 bytes
    2022-11-26T11:44:44.609Z nsx-edge-t0 NSX 17 ROUTING [nsx@6876 comp="nsx-edge" subcomp="nsxa" s2comp="routing" level="INFO"] Received routing state indication 'Down', bg
    _ngh_count 1 bgp_up_count 0 for 2c7af594-58dc-40b4-ab6b-041740a36439
    2022-11-26T11:44:44.610835+00:00 nsx-edge-t0 bgpd 5958 - - %ADJCHANGE: neighbor 172.28.50.1(Unknown) in vrf default Down Admin. shutdown
    2022-11-26T11:44:44.611618+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="bgp-adapter" level="INFO"] BGP State Update - VRF:default SRC:0.0.0.0 DST:172.28.50.1 State:DOWN
    2022-11-26T11:44:44.611738+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="rcpm-db" level="INFO"] update_routing_parameters_bgp_up_count - Old:1 New:0
    2022-11-26T11:44:44.611799+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="routing-up" level="INFO"] Sent notification to NSXA, For TIER0, vrf_id=2 Routing state=DOWN bgp_config_count=1, bgp_up_count=0
    2022-11-26T11:44:45.016707+00:00 nsx-edge-t0 NSX 3215 ROUTING [nsx@6876 comp="nsx-edge" subcomp="rcpm" s2comp="rcpm-db-pb" level="INFO"] num_redist_rules: 1, num_bgp_peers: 1, num_static_routes: 1, num_vrf_configs: 0 asn_change 0, num_ospf_interfaces: 0

     

    How did you solve this problem ?

    thank you

    Alexandre



  • 24.  RE: Even after enabling BGP on UI on Tire0-GW, still on the CLI, it shows as BGP as "Administratively shut down".

    Posted Nov 30, 2022 12:51 PM

    Hello, it's me again

    My problem was located on the configuration of the edge node T0. I solve it by redeploying new edge gateway, with one edge node profile:

    - one nsxHostswitch with two transport zone (overlay and vlan) and uplink configure with a segment trunk. As described in the article below:

    NSX-T 3.1 Enhancement - Shared ESXi and Edge Transport VLAN with a Single Uplink | virten.net

    Also a KB : Article Detail (vmware.com) about overlay vlan, communication between edge node and host node.

    Have a good day

    Alexandre