Brocade Fibre Channel Networking Community

Expand all | Collapse all

isl port 0 is down

  • 1.  isl port 0 is down

    Posted 03-26-2019 06:19 AM

    Good day SAN experts!

     

    May i seek for your kind advice on current status of our DCX 8510 ISL switches

     

    On switch 1, port 0 on blade 1 which is ISL port is on G- Port status(before it is on E-port) as shown on switchshow output below

     

    DCX8510_SW1:TIM> switchshow | more
    switchName: DCX8510_SW1
    switchType: 120.0
    switchState: Online
    switchMode: Native
    switchRole: Principal
    switchDomain: 11
    switchId: fffc0b
    switchWwn: 10:00:50:eb:1a:72:4e:00
    zoning: ON (DCX8510_SW1_SW3_CFG)
    switchBeacon: OFF
    FC Router: OFF
    FC Router BB Fabric ID: 128
    Address Mode: 0
    HIF Mode: ON

    Index Slot Port Address Media Speed State Proto
    ============================================================
    0 1 0 0b0000 id 2G Online FC LS G-Port

     

    On switch 3, port 0 on blade 1 which is ISL port is on E- Port status but on In_Sync state unknown wwn as shown on switchshow output below

     

    switchshow :
    switchName:DCX8510_SW3
    switchType:121.3
    switchState:Online
    switchMode:Native
    switchRole:Subordinate
    switchDomain:13
    switchId:fffc0d
    switchWwn:10:00:50:eb:1a:72:35:00
    zoning:ON (DCX8510_SW1_SW3_CFG)
    switchBeacon:OFF
    FC Router:OFF
    FC Router BB Fabric ID:128
    Address Mode:0
    HIF Mode:ON

    Index Slot Port Address Media Speed State Proto
    ============================================================
    0 1 0 0d0000 id 2G In_Sync FC LS E-Port (unknown)

     

     

    i have done portstatsclear on both port 0 of SW1 and SW3 and validated that there were no Tx and Rx frames during our monitoring

    for atleast an hour. There were no enc_outs, crc errors, loss of sync, etc that increased counters from zero.

     

    sw1 porterrshow(1hour)

     

    frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs
    tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err
    0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

     

    sw2 porterrshow(1hour)

     

    frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs
    tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err
    0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

     

     

    Going through individual physical SFP attributes(Longwave) for port 0 of SW1 and SW3 though sfpshow,

    the attributes seems normal as shown below.

     

    SW1 sfpshow

     

    sfpshow 1/0
    Identifier: 3 SFP
    Connector: 7 LC
    Transceiver: 5401001200000000 2,4,8_Gbps SM lw Long_dist
    Encoding: 1 8B10B
    Baud Rate: 85 (units 100 megabaud)
    Length 9u: 10 (units km)
    Length 9u: 100 (units 100 meters)
    Length 50u (OM2): 0 (units 10 meters)
    Length 50u (OM3): 0 (units 10 meters)
    Length 62.5u:0 (units 10 meters)
    Length Cu: 0 (units 1 meter)
    Vendor Name: BROCADE
    Vendor OUI: 00:05:1e
    Vendor PN: 57-1000027-01
    Vendor Rev: A
    Wavelength: 1310 (units nm)
    Options: 001a Loss_of_Sig,Tx_Fault,Tx_Disable
    BR Max: 0
    BR Min: 0
    Serial No: UDA111391003891
    Date Code: 111022
    DD Type: 0x68
    Enh Options: 0xf0
    Status/Ctrl: 0x82
    Alarm flags[0,1] = 0x0, 0x0
    Warn Flags[0,1] = 0x0, 0x0
    Alarm Warn
    low high low high
    Temperature: 38 Centigrade -40 90 -10 85
    Current: 28.150 mAmps 2.000 95.000 2.000 80.000
    Voltage: 3272.3 mVolts 2700.0 3800.0 2970.0 3630.0
    RX Power: -5.9 dBm (259.6uW) 4.4 uW 2240.0 uW 11.2 uW 1122.0 uW
    TX Power: -1.7 dBm (674.4 uW) 27.0 uW 2240.0 uW 67.0 uW 1122.0 uW

    State transitions: 17
    Last poll time: 03-26-2019 UTC Tue 07:12:50

     

    SW2 sfpshow

     

    sfpshow 1/0
    Identifier: 3 SFP
    Connector: 7 LC
    Transceiver: 5401001200000000 2,4,8_Gbps SM lw Long_dist
    Encoding: 1 8B10B
    Baud Rate: 85 (units 100 megabaud)
    Length 9u: 10 (units km)
    Length 9u: 100 (units 100 meters)
    Length 50u (OM2): 0 (units 10 meters)
    Length 50u (OM3): 0 (units 10 meters)
    Length 62.5u:0 (units 10 meters)
    Length Cu: 0 (units 1 meter)
    Vendor Name: BROCADE
    Vendor OUI: 00:05:1e
    Vendor PN: 57-1000027-01
    Vendor Rev: A
    Wavelength: 1310 (units nm)
    Options: 001a Loss_of_Sig,Tx_Fault,Tx_Disable
    BR Max: 0
    BR Min: 0
    Serial No: UDA109124001526
    Date Code: 090322
    DD Type: 0x68
    Enh Options: 0xf0
    Status/Ctrl: 0x80
    Alarm flags[0,1] = 0x0, 0x0
    Warn Flags[0,1] = 0x0, 0x0
    Alarm Warn
    low high low high
    Temperature: 50 Centigrade -40 90 -10 85
    Current: 36.126 mAmps 2.000 95.000 2.000 80.000
    Voltage: 3255.1 mVolts 2700.0 3800.0 2970.0 3630.0
    RX Power: -2.0 dBm (630.3uW) 4.4 uW 2240.0 uW 11.2 uW 1122.0 uW
    TX Power: -1.9 dBm (644.0 uW) 27.0 uW 2240.0 uW 67.0 uW 1122.0 uW

     

    Also ran portshow 1/0 on sw1 and sw3

     

    SW1 portshow 0

     

     

     

     

     

     

    SW3 portshow 0

     

    portIndex: 0
    portName: DWDM_ISL0_SW1
    portHealth: HEALTHY

    Authentication: None
    portDisableReason: None
    portCFlags: 0x1
    portFlags: 0x4203 PRESENT ACTIVE G_PORT NOELP LED
    LocalSwcFlags: 0x0
    portType: 24.0
    portState: 1Online
    Protocol: FC
    portPhys: 6In_Sync portScn: 1Online
    port generation number: 3860
    state transition count: 1165

    portId: 0b0000
    portIfId: 43120019
    portWwn: 20:00:50:eb:1a:72:4e:00
    portWwn of device(s) connected:

    Distance: static (desired distance = 120 Km)
    portSpeed: 2Gbps

    FEC: Inactive
    Credit Recovery: Inactive
    LE domain: 0
    Peer beacon: Off
    FC Fastwrite: OFF
    Interrupts: 0 Link_failure: 0 Frjt: 0
    Unknown: 0 Loss_of_sync: 0 Fbsy: 0
    Lli: 0 Loss_of_sig: 0
    Proc_rqrd: 0 Protocol_err: 0
    Timed_out: 0 Invalid_word: 0
    Rx_flushed: 0 Invalid_crc: 0
    Tx_unavail: 0 Delim_err: 0
    Free_buffer: 0 Address_err: 0
    Overrun: 0 Lr_in: 0
    Suspended: 0 Lr_out: 0
    Parity_err: 0 Ols_in: 0
    2_parity_err: 0 Ols_out: 0
    CMI_bus_err: 0

     

     

     

    portshow 0
    portDisableReason: None
    portCFlags: 0x1
    portFlags: 0x4101 PRESENT E_PORT G_PORT LOGIN LED
    LocalSwcFlags: 0x0
    portType: 24.0
    portState: 1Online
    Protocol: FC
    portPhys: 6In_Sync portScn: 1Online
    port generation number: 2716
    state transition count: 1165

    portId: 0d0000
    portIfId: 43120019
    portWwn: 20:00:50:eb:1a:72:35:00
    portWwn of device(s) connected:
    20:00:50:eb:1a:72:4e:00
    Distance: static (desired distance = 120 Km)
    portSpeed: 2Gbps

    FEC: Inactive
    Credit Recovery: Active
    LE domain: 0
    Peer beacon: Off
    FC Fastwrite: OFF
    Interrupts: 0 Link_failure: 0 Frjt: 0
    Unknown: 0 Loss_of_sync: 0 Fbsy: 0
    Lli: 0 Loss_of_sig: 0
    Proc_rqrd: 0 Protocol_err: 0
    Timed_out: 0 Invalid_word: 0
    Rx_flushed: 0 Invalid_crc: 0
    Tx_unavail: 0 Delim_err: 0
    Free_buffer: 0 Address_err: 0
    Overrun: 0 Lr_in: 0
    Suspended: 0 Lr_out: 0
    Parity_err: 0 Ols_in: 0
    2_parity_err: 0 Ols_out: 0
    CMI_bus_err: 0

     

    and portstatsshow/portstats64show 0 on both sw1 and sw3

     

    sw1

     

    portstatsshow 1/0
    stat_wtx 0 4-byte words transmitted
    stat_wrx 0 4-byte words received
    stat_ftx 0 Frames transmitted
    stat_frx 0 Frames received
    stat_c2_frx 0 Class 2 frames received
    stat_c3_frx 0 Class 3 frames received
    stat_lc_rx 0 Link control frames received
    stat_mc_rx 0 Multicast frames received
    stat_mc_to 0 Multicast timeouts
    stat_mc_tx 0 Multicast frames transmitted
    tim_rdy_pri 0 Time R_RDY high priority
    tim_txcrd_z 0 Time TX Credit Zero (2.5Us ticks)
    tim_txcrd_z_vc 0- 3: 0 0 0 0
    tim_txcrd_z_vc 4- 7: 0 0 0 0
    tim_txcrd_z_vc 8-11: 0 0 0 0
    tim_txcrd_z_vc 12-15: 0 0 0 0
    tim_latency_vc 0- 3: 1 1 1 1
    tim_latency_vc 4- 7: 1 1 1 1
    tim_latency_vc 8-11: 1 1 1 1
    tim_latency_vc 12-15: 1 1 1 1
    fec_cor_detected 0 Count of blocks that were corrected by FEC
    fec_uncor_detected 0 Count of blocks that were left uncorrected by FEC
    er_enc_in 0 Encoding errors inside of frames
    er_crc 0 Frames with CRC errors
    er_trunc 0 Frames shorter than minimum
    er_toolong 0 Frames longer than maximum
    er_bad_eof 0 Frames with bad end-of-frame
    er_enc_out 0 Encoding error outside of frames
    er_bad_os 0 Invalid ordered set
    er_pcs_blk 0 PCS block errors
    er_rx_c3_timeout 0 Class 3 receive frames discarded due to timeout
    er_tx_c3_timeout 0 Class 3 transmit frames discarded due to timeout
    er_unroutable 0 Frames that are unroutable
    er_unreachable 0 Frames with unreachable destination
    er_other_discard 0 Other discards
    er_type1_miss 0 frames with FTB type 1 miss
    er_type2_miss 0 frames with FTB type 2 miss
    er_type6_miss 0 frames with FTB type 6 miss
    er_zone_miss 0 frames with hard zoning miss
    er_lun_zone_miss 0 frames with LUN zoning miss
    er_crc_good_eof 0 Crc error with good eof
    er_inv_arb 0 Invalid ARB
    er_single_credit_loss0 Single vcrdy/frame loss on link
    er_multi_credit_loss0 Multiple vcrdy/frame loss on link
    phy_stats_clear_ts 03-26-2019 UTC Tue 07:13:20Timestamp of phy_port stats clear
    lgc_stats_clear_ts 03-26-2019 UTC Tue 07:13:20Timestamp of lgc_port stats clear
    DCX8510_SW1:TIM> portstats64show 1/0
    stat64_wtx 0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_wrx 0 top_int : 4-byte words received
    0 bottom_int : 4-byte words received
    stat64_ftx 0 top_int : Frames transmitted
    0 bottom_int : Frames transmitted
    stat64_frx 0 top_int : Frames received
    0 bottom_int : Frames received
    stat64_c2_frx0 top_int : Class 2 frames received
    0 bottom_int : Class 2 frames received
    stat64_c3_frx0 top_int : Class 3 frames received
    0 bottom_int : Class 3 frames received
    stat64_lc_rx0 top_int : Link control frames received
    0 bottom_int : Link control frames received
    stat64_mc_rx0 top_int : Multicast frames received
    0 bottom_int : Multicast frames received
    stat64_mc_to0 top_int : Multicast timeouts
    0 bottom_int : Multicast timeouts
    stat64_mc_tx0 top_int : Multicast frames transmitted
    0 bottom_int : Multicast frames transmitted
    tim64_rdy_pri0 top_int : Time R_RDY high priority
    0 bottom_int : Time R_RDY high priority
    tim64_txcrd_z0 top_int : Time BB_credit zero
    0 bottom_int : Time BB_credit zero
    er64_enc_in 0 top_int : Encoding errors inside of frames
    0 bottom_int : Encoding errors inside of frames
    er64_crc 0 top_int : Frames with CRC errors
    0 bottom_int : Frames with CRC errors
    er64_trunc 0 top_int : Frames shorter than minimum
    0 bottom_int : Frames shorter than minimum
    er64_toolong0 top_int : Frames longer than maximum
    0 bottom_int : Frames longer than maximum
    er64_bad_eof0 top_int : Frames with bad end-of-frame
    0 bottom_int : Frames with bad end-of-frame
    er64_enc_out0 top_int : Encoding error outside of frames
    0 bottom_int : Encoding error outside of frames
    er64_disc_c30 top_int : Class 3 frames discarded
    0 bottom_int : Class 3 frames discarded
    er64_pcs_blk0 top_int : PCS block errors
    0 bottom_int : PCS block errors
    stat64_rateTxFrame0 Tx frame rate (fr/sec)
    stat64_rateRxFrame0 Rx frame rate (fr/sec)
    stat64_rateTxPeakFrame0 Tx peak frame rate (fr/sec)
    stat64_rateRxPeakFrame0 Rx peak frame rate (fr/sec)
    stat64_rateTxWord0 Tx Word rate (words/sec)
    stat64_rateRxWord0 Rx Word rate (words/sec)
    stat64_rateTxPeakWord0 Tx peak Word rate (words/sec)
    stat64_rateRxPeakWord0 Rx peak Word rate (words/sec)
    stat64_PRJTFrames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_PBSYFrames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_inputBuffersFull0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_rxClass1Frames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_aveTxFrameSize0 Average Tx Frame size
    stat64_aveRxFrameSize0 Average Rx Frame size

     

     

     

    SW3

     

     

     

    portstatsshow 0
    stat_wtx 0 4-byte words transmitted
    stat_wrx 0 4-byte words received
    stat_ftx 0 Frames transmitted
    stat_frx 0 Frames received
    stat_c2_frx 0 Class 2 frames received
    stat_c3_frx 0 Class 3 frames received
    stat_lc_rx 0 Link control frames received
    stat_mc_rx 0 Multicast frames received
    stat_mc_to 0 Multicast timeouts
    stat_mc_tx 0 Multicast frames transmitted
    tim_rdy_pri 0 Time R_RDY high priority
    tim_txcrd_z 0 Time TX Credit Zero (2.5Us ticks)
    tim_txcrd_z_vc 0- 3: 0 0 0 0
    tim_txcrd_z_vc 4- 7: 0 0 0 0
    tim_txcrd_z_vc 8-11: 0 0 0 0
    tim_txcrd_z_vc 12-15: 0 0 0 0
    tim_latency_vc 0- 3: 0 1 1 1
    tim_latency_vc 4- 7: 1 1 1 1
    tim_latency_vc 8-11: 1 1 1 1
    tim_latency_vc 12-15: 1 1 1 1
    fec_cor_detected 0 Count of blocks that were corrected by FEC
    fec_uncor_detected 0 Count of blocks that were left uncorrected by FEC
    er_enc_in 0 Encoding errors inside of frames
    er_crc 0 Frames with CRC errors
    er_trunc 0 Frames shorter than minimum
    er_toolong 0 Frames longer than maximum
    er_bad_eof 0 Frames with bad end-of-frame
    er_enc_out 0 Encoding error outside of frames
    er_bad_os 0 Invalid ordered set
    er_pcs_blk 0 PCS block errors
    er_rx_c3_timeout 0 Class 3 receive frames discarded due to timeout
    er_tx_c3_timeout 0 Class 3 transmit frames discarded due to timeout
    er_unroutable 0 Frames that are unroutable
    er_unreachable 0 Frames with unreachable destination
    er_other_discard 0 Other discards
    er_type1_miss 0 frames with FTB type 1 miss
    er_type2_miss 0 frames with FTB type 2 miss
    er_type6_miss 0 frames with FTB type 6 miss
    er_zone_miss 0 frames with hard zoning miss
    er_lun_zone_miss 0 frames with LUN zoning miss
    er_crc_good_eof 0 Crc error with good eof
    er_inv_arb 0 Invalid ARB
    er_single_credit_loss0 Single vcrdy/frame loss on link
    er_multi_credit_loss0 Multiple vcrdy/frame loss on link
    phy_stats_clear_ts 03-26-2019 UTC Tue 08:51:58Timestamp of phy_port stats clear
    lgc_stats_clear_ts 03-26-2019 UTC Tue 08:51:58Timestamp of lgc_port stats clear

    portstats64show 0
    stat64_wtx 0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_wrx 0 top_int : 4-byte words received
    0 bottom_int : 4-byte words received
    stat64_ftx 0 top_int : Frames transmitted
    0 bottom_int : Frames transmitted
    stat64_frx 0 top_int : Frames received
    0 bottom_int : Frames received
    stat64_c2_frx0 top_int : Class 2 frames received
    0 bottom_int : Class 2 frames received
    stat64_c3_frx0 top_int : Class 3 frames received
    0 bottom_int : Class 3 frames received
    stat64_lc_rx0 top_int : Link control frames received
    0 bottom_int : Link control frames received
    stat64_mc_rx0 top_int : Multicast frames received
    0 bottom_int : Multicast frames received
    stat64_mc_to0 top_int : Multicast timeouts
    0 bottom_int : Multicast timeouts
    stat64_mc_tx0 top_int : Multicast frames transmitted
    0 bottom_int : Multicast frames transmitted
    tim64_rdy_pri0 top_int : Time R_RDY high priority
    0 bottom_int : Time R_RDY high priority
    tim64_txcrd_z0 top_int : Time BB_credit zero
    0 bottom_int : Time BB_credit zero
    er64_enc_in 0 top_int : Encoding errors inside of frames
    0 bottom_int : Encoding errors inside of frames
    er64_crc 0 top_int : Frames with CRC errors
    0 bottom_int : Frames with CRC errors
    er64_trunc 0 top_int : Frames shorter than minimum
    0 bottom_int : Frames shorter than minimum
    er64_toolong0 top_int : Frames longer than maximum
    0 bottom_int : Frames longer than maximum
    er64_bad_eof0 top_int : Frames with bad end-of-frame
    0 bottom_int : Frames with bad end-of-frame
    er64_enc_out0 top_int : Encoding error outside of frames
    0 bottom_int : Encoding error outside of frames
    er64_disc_c30 top_int : Class 3 frames discarded
    0 bottom_int : Class 3 frames discarded
    er64_pcs_blk0 top_int : PCS block errors
    0 bottom_int : PCS block errors
    stat64_rateTxFrame0 Tx frame rate (fr/sec)
    stat64_rateRxFrame0 Rx frame rate (fr/sec)
    stat64_rateTxPeakFrame0 Tx peak frame rate (fr/sec)
    stat64_rateRxPeakFrame0 Rx peak frame rate (fr/sec)
    stat64_rateTxWord0 Tx Word rate (words/sec)
    stat64_rateRxWord0 Rx Word rate (words/sec)
    stat64_rateTxPeakWord0 Tx peak Word rate (words/sec)
    stat64_rateRxPeakWord0 Rx peak Word rate (words/sec)
    stat64_PRJTFrames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_PBSYFrames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_inputBuffersFull0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_rxClass1Frames0 top_int : 4-byte words transmitted
    0 bottom_int : 4-byte words transmitted
    stat64_aveTxFrameSize0 Average Tx Frame size
    stat64_aveRxFrameSize0 Average Rx Frame size

     

    And lastly , the Diagshow displays no Tx and Rx frames both sw1 and sw3. But port 1/0 on SW3 state is down(DN).

     

     

    SW!

     

    diagshow 1


    Diagnostics Status: Tue Mar 26 07:20:21 2019

    Slot: 1 UPORTS
    Port BPort Diag Active Speed FrTX FrRX LLI Errs Loopback
    1/0 25 OK UP 2Gbps 0 0 0

     

    SW3

     

    diagshow 1


    Diagnostics Status: Tue Mar 26 10:14:36 2019

    Slot: 1 UPORTS
    Port BPort Diag Active Speed FrTX FrRX LLI Errs Loopback
    1/0 25 OK DN 2Gbps -- -- --

     

     

    as of this moment, our port 0 ISL is still down. BTW we still have port 1 up as our redundant ISL port.

    May i seek for your kind advice.

     

     

     

     

     

     


    #BrocadeFibreChannelNetworkingCommunity


  • 2.  Re: isl port 0 is down

    Posted 03-29-2019 07:46 AM

    SW1 sfpshow

    sfpshow 1/0
    RX Power: -5.9 dBm (259.6uW) 4.4 uW 2240.0 uW 11.2 uW 1122.0 uW
    TX Power: -1.7 dBm (674.4 uW) 27.0 uW 2240.0 uW 67.0 uW 1122.0 uW

    State transitions: 17
    Last poll time: 03-26-2019 UTC Tue 07:12:50

     

    SW2 sfpshow

    sfpshow 1/0
    RX Power: -2.0 dBm (630.3uW) 4.4 uW 2240.0 uW 11.2 uW 1122.0 uW
    TX Power: -1.9 dBm (644.0 uW) 27.0 uW 2240.0 uW 67.0 uW 1122.0 uW

     

    RX Power on SW1 is low. But, not so low that it should be causing issues. That typed, I would recommend looking at the cabling just be certain. There are a lot of things that could be impacting the difference between TX and RX. Distance. Cable bends. Connection points. WDM. Etc.

     

    As this is in a G-Port state, it is most likely not passing any kind of traffic, so errors on the port may not be showing (such as ENC_OUT).

     

     


    #BrocadeFibreChannelNetworkingCommunity


  • 3.  Re: isl port 0 is down

    Posted 03-30-2019 06:37 AM

    Hi

    We manage to replace cable and SFP on the SW2 side and bring back the port 0 to E port state. How ever, porterrshow counters goes up such as enc_in,crc_err,crc_g_eof,too_short,bad_eof,enc_out and disc_c3.


    porterrshow |more
    frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs
    tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err
    0: 214.8k 1.3m 17 39 1 1 0 38 30 74 1 0 0 0 0 74 0 0




    On SW1 side, there where no errors logged on porterrshow pertaining to port 0. We havent replaced the SPF on port 0 that you indicated have low power

     

    There is a DWDM in between BTW


    #BrocadeFibreChannelNetworkingCommunity


  • 4.  Re: isl port 0 is down

    Posted 03-29-2019 07:49 AM

    Also, I noticed these are long wave SFPs. Are you certain that the cabling is LW cabling throughout (Single Mode Fiber 9 micron)


    #BrocadeFibreChannelNetworkingCommunity


  • 5.  Re: isl port 0 is down

    Posted 03-30-2019 06:23 AM