Brocade Fibre Channel Networking Community

Expand all | Collapse all

Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

  • 1.  Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-23-2013 07:17 PM

    Hi All,

     

    I want to connect two Brocade 300 SAN switches over 2km distance using 'single-mode' fibre. We have Brocade LW SFPs at both ends in the repective switch ports.

    However, at one end, the port state reads as "No Light" while other end it is read as "In_Sync". Here I have included 'switchshow','portshow','portcfgshow','porterrshow' outputs for your reference. Can someone kindly tell me what has gone wrong here? What configuration changes do I need to do in order to make it work?

    Note that I even set 'portcfglongdistance' value to LE at both ends but, still there is no improvement. Is there any media error? I mean something like the fibre being too noisy etc.


    admin> switchshow

    switchName: DR_B24_SW2

    switchType: 71.2

    switchState: Online  

    switchMode: Native

    switchRole: Principal

    switchDomain: 2

    switchId: fffc02

    switchWwn: 10:00:00:05:1x:xx:xx:xx

    zoning:  ON (

    switchBeacon: OFF

     

    Area Port Media Speed State     Proto =====================================

     

     22  22   id    N4   No_Light        
     23  23   id    N4   In_Sync         

    admin> portshow 23

    portName:

    portHealth: No Fabric Watch License

    Authentication: None

    portDisableReason: None

    portCFlags: 0x1

    portFlags: 0x1  PRESENT U_PORT

    portType:  18.0

    POD Port: Port is licensed

    portState: 2 Offline 

    portPhys:  6 In_Sync 

    portScn:   0          

    port generation number:    0

    portId:    021700

    portIfId:    43020000

    portWwn:   20:17:00:05:1e:d4:40:17

    portWwn of device(s) connected:

     

     

    Distance:  normal portSpeed: N4Gbps

    LE domain: 0

    FC Fastwrite: OFF

    Interrupts:        0          Link_failure: 253        Frjt:         0         

    Unknown:           0          Loss_of_sync: 47         Fbsy:         0         

    Lli:               771        Loss_of_sig:  93        

    Proc_rqrd:         5          Protocol_err: 0         

    Timed_out:         0          Invalid_word: 7871990   

    Rx_flushed:        0          Invalid_crc:  0         

    Tx_unavail:        0          Delim_err:    0         

    Free_buffer:       0          Address_err:  0         

    Overrun:           0          Lr_in:        0         

    Suspended:         0          Lr_out:       0         

    Parity_err:        0          Ols_in:       0         

    2_parity_err:      0          Ols_out:      0         

    CMI_bus_err:       0         

     

    Port part of other ADs: No

     

    admin> portcfgshow 23

    Area Number:    23
    Speed Level:    AUTO(HW)
    Fill Word:    0(Idle-Idle)
    AL_PA Offset 13:   OFF
    Trunk Port    ON
    Long Distance    OFF
    VC Link Init    OFF
    Locked L_Port    OFF
    Locked G_Port    OFF
    Disabled E_Port    OFF
    ISL R_RDY Mode    OFF
    RSCN Suppressed    OFF
    Persistent Disable   OFF
    NPIV capability    ON
    QOS E_Port    ON
    Port Auto Disable:   OFF
    Rate Limit    OFF
    Mirror Port    OFF
    Credit Recovery    ON
    F_Port Buffers    OFF



    admin> porterrshow


            frames  enc  crc  crc   too  too  bad  enc disc link loss loss frjt fbsy
           tx   rx   in  err g_eof shrt long  eof  out   c3 fail sync  sig
         ============================================================================
      0:   23   25    0    0    0    0    0    0   14    0    0    1    2    0    0
      1:   25   27    0    0    0    0    0    0   32    0    0    1    2    0    0
      2:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      3:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      4:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      5:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      6:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      7:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      8:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
      9:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     10:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     11:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     12:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     13:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     14:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     15:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     16:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     17:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     18:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     19:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     20:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     21:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     22:    0    0    0    0    0    0    0    0    0    0    0    0    2    0    0
     23:    0    0    0    0    0    0    0    0  7.8m   0  260   49   97    0    0



    Once I set long distance mode to 'LE' in port 23

    admin> portcfglongdistance 23 LE


    The 'switchshow' output still shows In_sync at one end while other end says 'No_Light'

     23  23   id    N4   In_Sync          LE



    Can someone kindly tell me what has gone wrong here? Is it a media error?


    Thank you in avanced


     


    #BrocadeFibreChannelNetworkingCommunity


  • 2.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-24-2013 01:30 AM

    > LE mode is right for over 2 KM of Distance

    > looks Like you have used the port speed @ 4Gbps on both san switch

    > one thing what I think can be an issue is the SFP, LWL for longdistance up to 10 KM & ELWL for more than 10 KM

     

    Regards,

     

    Bhaskar


    #BrocadeFibreChannelNetworkingCommunity


  • 3.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-24-2013 03:33 AM

    Hi Bhaskar,

     

    The actual distance is little less than 2km. (1.8km)

    I have not specifically set the port speed to a fix value instead, I have configured it to take via auto negotiation.
    We have LW SFPs at both ends.

    Do you see any indications of media error? Loss of signal etc.?

    Thanks
    Srangama


    #BrocadeFibreChannelNetworkingCommunity


  • 4.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-24-2013 10:58 PM

    I have attached some photos of the SFPs and Single-Mode cables that we use.

     

    I doubt whether there is a huge / considerable sinal loss due to the cables that we use. Could you kindly verify and tell me..

    Thanks

     

    The LW SFP

     

    The Single Mode Fibre Path code


    #BrocadeFibreChannelNetworkingCommunity


  • 5.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 01:21 AM

    Pictures are not showing (at least for me)


    #BrocadeFibreChannelNetworkingCommunity


  • 6.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 04:57 AM

    I too cannot see Pictures

     

    You can try this steps:

     

    Since it is with in 2 KM

     

    >use L0 which is default mode for E_port

    >set port speed at 4G which is not required, but recommended for ISL

    >If the SFP has problem, try use different SFP (regular one which is available in the switch- free ones, instead of using LWL) on both switch ports

    >change the port back to #portcfgeport port# to set the port to E_ port

     

    as per the document, L0 is default for < 5KM & for >5KM use LE both does not require longdistance license (extended fabric license)

     

    refer FOS  admin guide 7.0 under longdistance which is 1 page notes.

    let us know the outcome

     

    Bhaskar  


    #BrocadeFibreChannelNetworkingCommunity


  • 7.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 08:30 AM


  • 8.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 08:31 AM

    I re-attached the photos.. Hopefully you can see them now.

     

    Thanks


    #BrocadeFibreChannelNetworkingCommunity


  • 9.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 08:32 AM

    HI


    I re-attached the photos.. Hopefully you can see them now.

     

    Thanks


    #BrocadeFibreChannelNetworkingCommunity


  • 10.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 08:48 AM

    Nope again no picture, well a default picture from the lithium platform perhaps, but no usefull image.

    I inserted the old Brocade logo, just as a test i image uploading actually works

    But more to the point did you try Bhaskar suggestions?

     

     

    test


    #BrocadeFibreChannelNetworkingCommunity


  • 11.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 08:49 AM

    My own attached image works for me, did you perhaps select the private checkbox?


    #BrocadeFibreChannelNetworkingCommunity


  • 12.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 09:43 AM

    You should be able to see images now. We had a permissions/approval issue. Apologies for the inconvenience


    #BrocadeFibreChannelNetworkingCommunity


  • 13.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 09:58 AM


  • 14.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-25-2013 10:33 AM

    Hi,

     

    unfortunateley, 2km of cable could mean everything. Usually you have  a loss of 1dB but this highly is dependand of how many patch panels are inbetween, how  the fiber routed, perhaps some bend radius violations .. and so on...

    could you check the sfpshow, if you have any light at least and if the light levels are ok?

    Also if you have an optical powermeter, pls check the lightlevels..

     

    If nothing helps, you should consider doing an OTDR measurement.

     

    Hope this helps

    Uli

     


    #BrocadeFibreChannelNetworkingCommunity


  • 15.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-26-2013 03:26 AM

    Hi

     

    Below is the 'sfpshow' output taken from two respective switches that need to connect

     

     

    :admin> sfpshow
    Port  0: id (sw) Vendor: BROCADE          Serial No: VAF209460000GV5  Speed: 100
    ,200,400_MB/s
    Port  1: id (sw) Vendor: BROCADE          Serial No: VAF209460000GV7  Speed: 100
    ,200,400_MB/s
    Port  2: id (sw) Vendor: BROCADE          Serial No: VAF209460000GTJ  Speed: 100
    ,200,400_MB/s
    Port  3: id (sw) Vendor: BROCADE          Serial No: VAF209460000GUN  Speed: 100
    ,200,400_MB/s
    Port  4: id (sw) Vendor: BROCADE          Serial No: VAF209440000M78  Speed: 100
    ,200,400_MB/s
    Port  5: id (sw) Vendor: BROCADE          Serial No: VAF209460000F8J  Speed: 100
    ,200,400_MB/s
    Port  6: id (sw) Vendor: BROCADE          Serial No: VAF209460000F8G  Speed: 100
    ,200,400_MB/s
    Port  7: id (sw) Vendor: BROCADE          Serial No: VAF209440000JK6  Speed: 100
    ,200,400_MB/s
    Port  8: id (sw) Vendor: BROCADE          Serial No: VAF209440000JK9  Speed: 100
    ,200,400_MB/s
    Port  9: id (sw) Vendor: BROCADE          Serial No: VAF209440000JJ0  Speed: 100
    ,200,400_MB/s
    Port 10: id (sw) Vendor: BROCADE          Serial No: VAF209440000SM6  Speed: 100
    ,200,400_MB/s
    Port 11: id (sw) Vendor: BROCADE          Serial No: VAF209440000JW9  Speed: 100
    ,200,400_MB/s
    Port 12: id (sw) Vendor: BROCADE          Serial No: VAF209460000GPZ  Speed: 100
    ,200,400_MB/s
    Port 13: id (sw) Vendor: BROCADE          Serial No: VAF209440000M70  Speed: 100
    ,200,400_MB/s
    Port 14: id (sw) Vendor: BROCADE          Serial No: VAF209440000JJ2  Speed: 100
    ,200,400_MB/s
    Port 15: id (sw) Vendor: BROCADE          Serial No: VAF209440000JJE  Speed: 100
    ,200,400_MB/s
    Port 16: id (sw) Vendor: BROCADE          Serial No: VAF209460000GVC  Speed: 100
    ,200,400_MB/s
    Port 17: id (sw) Vendor: BROCADE          Serial No: VAF209460000GUT  Speed: 100
    ,200,400_MB/s
    Port 18: id (sw) Vendor: BROCADE          Serial No: VAF209460000GVA  Speed: 100
    ,200,400_MB/s
    Port 19: id (lw) Vendor: BROCADE          Serial No: WCA109314405360  Speed: 100
    ,200,400_MB/s
    Port 20: id (sw) Vendor: BROCADE          Serial No: VAF209440000JJL  Speed: 100
    ,200,400_MB/s
    Port 21: id (sw) Vendor: BROCADE          Serial No: VAF209440000JJF  Speed: 100
    ,200,400_MB/s
    Port 22: id (sw) Vendor: BROCADE          Serial No: VAF209460000ALC  Speed: 100
    ,200,400_MB/s
    Port 23: id (lw) Vendor: BROCADE          Serial No: WCA109314405316  Speed: 100
    ,200,400_MB/s
    :admin>



    :admin> sfpshow                            
    Port  0: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVN  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  1: id (sw) Vendor: BROCADE          Serial No: VAF209460000AW5  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  2: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVT  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  3: id (sw) Vendor: BROCADE          Serial No: VAF209460000AW0  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  4: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVK  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  5: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVH  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  6: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVD  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  7: id (sw) Vendor: BROCADE          Serial No: VAF209460000AVR  Speed: 100                                                                               
    ,200,400_MB/s            
    Port  8: id (sw) Vendor: BROCADE                                                  
    ,200,400_MB/s            
    Port  9: id (sw) Vendor: BROCADE          Serial No: VAF20946000070V  Speed: 100                                                                               
    ,200,400_MB/s            
    Port 10: id (sw) Vendor: BROCADE          Serial No: VAF2094600006ZJ  Speed: 100                                                                               
    ,200,400_MB/s            
    Port 11: id (sw) Vendor: BROCADE          Serial No: VAF209460000710  Speed: 100                                                                               
    ,200,400_MB/s            
    Port 12: id (sw) Vendor: BROCADE          Serial No: VAF20946000071R  Speed: 100
    ,200,400_MB/s
    Port 13: id (sw) Vendor: BROCADE          Serial No: VAF20946000071V  Speed: 100
    ,200,400_MB/s
    Port 14: id (sw) Vendor: BROCADE          Serial No: VAF20946000070T  Speed: 100
    ,200,400_MB/s
    Port 15: id (sw) Vendor: BROCADE          Serial No: VAF20946000070N  Speed: 100
    ,200,400_MB/s
    Port 16: --
    Port 17: id (sw) Vendor: BROCADE          Serial No: VAF209430000S30  Speed: 100
    ,200,400_MB/s
    Port 18: id (sw) Vendor: BROCADE          Serial No: VAF209440000FBW  Speed: 100
    ,200,400_MB/s
    Port 19: id (lw) Vendor: BROCADE          Serial No: WCA109314405304  Speed: 100
    ,200,400_MB/s
    Port 20: id (sw) Vendor: BROCADE          Serial No: VAF209450000B7Z  Speed: 100
    ,200,400_MB/s
    Port 21: id (sw) Vendor: BROCADE          Serial No: VAF209430000PNE  Speed: 100
    ,200,400_MB/s
    Port 22: id (sw) Vendor: BROCADE          Serial No: VAF209440000SMM  Speed: 100
    ,200,400_MB/s
    Port 23: id (lw) Vendor: BROCADE          Serial No: WCA109314405342  Speed: 100
    ,200,400_MB/s
    :admin>

    Thanks


    #BrocadeFibreChannelNetworkingCommunity


  • 16.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 11-26-2013 05:47 AM

    Hi,

     

    sorry for not beeing precise enough: you would need to execute the sfphsow just for your ISL's like:

     the output should look like:

    Identifier: 3 SFP
    Connector: 7 LC
    Transceiver: 540c404000000000 2,4,8_Gbps M5,M6 sw Short_dist
    Encoding: 1 8B10B
    Baud Rate: 85 (units 100 megabaud)
    Length 9u: 0 (units km)
    Length 9u: 0 (units 100 meters)
    Length 50u: 5 (units 10 meters)
    Length 62.5u:2 (units 10 meters)
    Length Cu: 0 (units 1 meter)
    Vendor Name: BROCADE
    Vendor OUI: 00:05:1e
    Vendor PN: 57-1000012-01
    Vendor Rev: A
    Wavelength: 850 (units nm)
    Options: 003a Loss_of_Sig,Tx_Fault,Tx_Disable
    BR Max: 0
    BR Min: 0
    Serial No: UAF108050000240
    Date Code: 080130
    DD Type: 0x68
    Enh Options: 0xfa
    Status/Ctrl: 0xb0
    Alarm flags[0,1] = 0x5, 0x0
    Warn Flags[0,1] = 0x5, 0x0
    Alarm Warn
    low high low high
    Temperature: 35 Centigrade -10 90 -5 85
    Current: 7.306 mAmps 1.000 17.000 2.000 14.000
    Voltage: 3293.4 mVolts 2900.0 3700.0 3000.0 3600.0
    RX Power: -3.0 dBm (506.4uW) 10.0 uW 1258.9 uW 15.8 uW 1000.0 uW
    TX Power: -3.3 dBm (467.3 uW)125.9 uW 631.0 uW 158.5 uW 562.3 uW

     

     

    then, you can see if your powerlevels are correct  (RX Power nad TX power values)

     

    Cheers

     

    Uli

     


    #BrocadeFibreChannelNetworkingCommunity


  • 17.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 12-01-2013 09:30 AM

    1. from the picture I can see the patch cord is of 10 mtr length which will be used only for 50/62.5 micron cable

     

    In this case you need to use LWL SFP with single mode cable 9 microns

     

    SFP is correct, connect with single mode

     

    I guess with your existing LE mode it should work, if you do not intend to use L0

     


    #BrocadeFibreChannelNetworkingCommunity


  • 18.  Re: Connecting Two Bracade 300 SAN switches over 2km distance via Single-Mode Fibre?

    Posted 12-02-2013 06:14 PM

    HI All,

     

    Very sorry for replying in late.

     

    Thank you very much for your valuable suggestions and ideas. We were finally able to 'online' the respective switchports. The problem was some impurity thus high noise & signal loss in patch pannels (where "dark fibre" connects).

    Regards,
    srangama

     


    #BrocadeFibreChannelNetworkingCommunity