Fabric change from time to time

[rh1]

Hi
I have problem with Brocade 300 switch. Switch works ok but from time to time I'm receiving below errors on console:

2012/05/07-23:35:59, , 3, FID 128, INFO, br300, The last fabric change happened at: Mon May  7 21:35:54 2012
2012/05/07-23:36:09, , 4, FID 128, INFO, br300,  The last device change happened at : Mon May  7 21:36:00 2012
2012/05/07-23:36:19, , 5, FID 128, INFO, br300, The last fabric change happened at: Mon May  7 21:36:18 2012
2012/05/07-23:36:24, , 6, FID 128, INFO, br300,  The last device change happened at : Mon May  7 21:36:19 2012
2012/05/07-23:39:49, , 7, FID 128, INFO, br300, The last fabric change happened at: Mon May  7 21:39:45 2012
2012/05/07-23:39:54, , 8, FID 128, INFO, br300,  The last device change happened at : Mon May  7 21:39:51 2012
2012/05/07-23:40:09, , 9, FID 128, INFO, br300, The last fabric change happened at: Mon May  7 21:40:03 2012
2012/05/07-23:40:09, , 10, FID 128, INFO, br300,  The last device change happened at : Mon May  7 21:40:05 2012

And I see loss sync & loss sig + increasing enc out at ports.

Do you think it can be hardware problem ?

Do you suggest any commands for futher diagnostics ?

Reboot of switch or power off and power on didn't help.

I'm running v6.4.2b

SFPs and patchcord are just fine - I tested them for 5 days in different Brocade switch and they work without any problem as described.

br300:admin> porterrshow
          frames      enc    crc    crc    too    too    bad    enc   disc   link   loss   loss   frjt   fbsy
       tx     rx      in    err    g_eof  shrt   long   eof     out   c3    fail    sync   sig
     =========================================================================================================
  0:   24     25      0      0      0      0      0      0     85.5m   0      0      4      8      0      0
  1:    3.9k   3.0k   0      0      0      0      0      0     84.0m   0      0      6     12      0      0
  2:   10.1k  16.8k   0      0      0      0      0      0     45.6m   0      0      3      6      0      0
  3:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  4:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  5:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  6:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  7:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  8:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
  9:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
10:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
11:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
12:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
13:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
14:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
15:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
16:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
17:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
18:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
19:  102.5k   8.3k   0      0      0      0      0      0    675.3k   0      2      2      0      0      0
20:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
21:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
22:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0
23:    0      0      0      0      0      0      0      0      0      0      0      0      0      0      0

[Anonymous]

Looks like symptoms described in .

But that would only be true if these ports are running 8G.

The errors in the console are a result of the signal/sync loss events.

[Anonymous]

portCfgShow output would be helpful in this context. If it is the fillword you could simply test lower link speed or different fill words.

enc_out means non-data frames, i.e. ordered set encoding, of which >80% is current fill word on the wire (so IDLE's or ARBff ordered sets). Other ordered sets you'd typically see are R_RDY's for instance. When loosing too many fill words (which aren't per se important for processing at all) you end up loosing bit / word synchronization leading to loss of sync. As long as your loss of sync > R_T_TOV (100ms) the port will invoke the LF protocol (link fault). Since the LF counter is zero in your case, you can assume that the issue isn't really critical.

[rh1]

On port 0 and 1 I have connected 2Gbps HBA in servers
On port 3 I have connected 16Gbps HBA (Brocade 1860) with 8Gbps SFP+
On port 19 I have connected 2Gbps Brocade 3250 switch running latest FabOS 5.x

I attached all configuration below + type of SFP in my BR300 switch

What setting of fillword you recommends for these ports ?

>switchshow
  0   0   010000   id    N2   Online      FC  F-Port  21:00:00:e0:xx:xx:xx:xx
  1   1   010100   id    N2   Online      FC  F-Port  10:00:00:00:xx:xx:xx:xx
  3   3   010300   id    N8   Online      FC  F-Port  10:00:8c:7c:xx:xx:xx:xx
19  19   011300   id    N2   Online      FC  E-Port  10:00:00:05:xx:xx:xx:xx "SW3250" (upstream)

>sfpshow
Port  0: id (sw) Vendor: BROCADE          Serial No: WAA1095xxxxxxxx  Speed: 100,200,400_MB/s
Port  1: id (sw) Vendor: BROCADE          Serial No: WAA1094xxxxxxxx  Speed: 100,200,400_MB/s
Port  3: id (sw) Vendor: BROCADE          Serial No: UAA2101xxxxxxxx  Speed: 200,400,800_MB/s
Port 19: id (sw) Vendor: BROCADE          Serial No: UAA2101xxxxxxxx  Speed: 200,400,800_MB/s

> portcfgshow 0
Area Number:              0
Speed Level:              AUTO(HW)
Fill Word:                0(Idle-Idle)
AL_PA Offset 13:          OFF
Trunk Port                ON
Long Distance             OFF
VC Link Init              OFF
Locked L_Port             OFF
Locked G_Port             OFF
Disabled E_Port           OFF
Locked E_Port             OFF
ISL R_RDY Mode            OFF
RSCN Suppressed           OFF
Persistent Disable        OFF
LOS TOV enable            OFF
NPIV capability           ON
QOS E_Port                AE
Port Auto Disable:        OFF
Rate Limit                OFF
Mirror Port               OFF
Credit Recovery           ON
F_Port Buffers            OFF
Fault Delay:              0(R_A_TOV)
NPIV PP Limit:            126
CSCTL mode:               OFF

> portcfgshow 1
Area Number:              1
Speed Level:              AUTO(HW)
Fill Word:                0(Idle-Idle)
AL_PA Offset 13:          OFF
Trunk Port                ON
Long Distance             OFF
VC Link Init              OFF
Locked L_Port             OFF
Locked G_Port             OFF
Disabled E_Port           OFF
Locked E_Port             OFF
ISL R_RDY Mode            OFF
RSCN Suppressed           OFF
Persistent Disable        OFF
LOS TOV enable            OFF
NPIV capability           ON
QOS E_Port                AE
Port Auto Disable:        OFF
Rate Limit                OFF
Mirror Port               OFF
Credit Recovery           ON
F_Port Buffers            OFF
Fault Delay:              0(R_A_TOV)
NPIV PP Limit:            126
CSCTL mode:               OFF

> portcfgshow 3
Area Number:              3
Speed Level:              AUTO(HW)
Fill Word:                0(Idle-Idle)
AL_PA Offset 13:          OFF
Trunk Port                ON
Long Distance             OFF
VC Link Init              OFF
Locked L_Port             OFF
Locked G_Port             OFF
Disabled E_Port           OFF
Locked E_Port             OFF
ISL R_RDY Mode            OFF
RSCN Suppressed           OFF
Persistent Disable        OFF
LOS TOV enable            OFF
NPIV capability           ON
QOS E_Port                AE
Port Auto Disable:        OFF
Rate Limit                OFF
Mirror Port               OFF
Credit Recovery           ON
F_Port Buffers            OFF
Fault Delay:              0(R_A_TOV)
NPIV PP Limit:            126
CSCTL mode:               OFF

> portcfgshow 19
Area Number:              19
Speed Level:              AUTO(HW)
Fill Word:                0(Idle-Idle)
AL_PA Offset 13:          OFF
Trunk Port                ON
Long Distance             OFF
VC Link Init              OFF
Locked L_Port             OFF
Locked G_Port             OFF
Disabled E_Port           OFF
Locked E_Port             OFF
ISL R_RDY Mode            OFF
RSCN Suppressed           OFF
Persistent Disable        OFF
LOS TOV enable            OFF
NPIV capability           ON
QOS E_Port                AE
Port Auto Disable:        OFF
Rate Limit                OFF
Mirror Port               OFF
Credit Recovery           ON
F_Port Buffers            OFF
Fault Delay:              0(R_A_TOV)
NPIV PP Limit:            126
CSCTL mode:               OFF

> porterrshow
          frames      enc    crc    crc    too    too    bad    enc   disc   link   loss   loss   frjt   fbsy
       tx     rx      in    err    g_eof  shrt   long   eof     out   c3    fail    sync   sig
     =========================================================================================================
  0:    0      0      0      0      0      0      0      0      3      0      0      0      0      0      0
  1:  510    495      0      0      0      0      0      0      5      0      0      0      0      0      0
  3:    6.9k   3.0k   0      0      0      0      0      0    312      0      0      0      0      0      0
19:    4.1k   8.0k   0      0      0      0      0      0      2      0      0      0      0      0      0

On BR3250 to BR300:

  0   0   id    N2   Online           E-Port  10:00:00:05:xx:xx:xx:xx "BR300" (downstream)

Port  0: id (sw) Vendor: BROCADE          Serial No: WAA10948xxxxxxx

>portcfgshow 0
Area Number:              0
Speed Level:              AUTO
Trunk Port                ON
Long Distance             OFF
VC Link Init              OFF
Locked L_Port             OFF
Locked G_Port             OFF
Disabled E_Port           OFF
ISL R_RDY Mode            OFF
RSCN Suppressed           OFF
Persistent Disable        OFF
NPIV capability           OFF

3250> porterrshow
        frames  enc  crc  too  too  bad  enc disc link loss loss frjt fbsy
       tx   rx   in  err shrt long  eof  out   c3 fail sync  sig
     =====================================================================
  0:   82k 122k  18    8    0    0    1   17m   0   23  5.6k   1    0    0

[felipon]

Hi,

Those kind of messages appear because there is a modification in the Fabric topology.

2012/05/07-23:35:59, , 3, FID 128, INFO, br300, The last fabric change happened at: Mon May  7 21:35:54 2012

Since, there is only one ISL between the two switchs, if the link fail you'll get that message.

Try to set the port speed of the ISL port manually instead of using autonegotiation, check the Rx Signal level in both ends, clear the port stats and monitor the errors.

A FOS upgrade will also be recommendable, since I see that the crc_g_eof counter does not appear in porterrshow.