7800 Router Configuration Challenges

[Anonymous]

I'm configuring two 7800 routers for storage replication across a network for DR. I've followed the instructions to create the EX ports, set the fabric IDs for the EX ports, set unique domain IDs for all participating switches, configured the GE0 ports and created the circuit. I'm able to ping from both ends and it seems everything is setup correctly. The problem I have is the FCIP tunnel is bouncing up and down. I'm not able to see the fabric information for the peer switch. Here's some information about the configuration from one router. Any suggestions on why it's bouncing?

ACMODRFCIP01:admin> fabricshow

Switch ID   Worldwide Name           Enet IP Addr    FC IP Addr      Name

-------------------------------------------------------------------------

  3: fffc03 10:00:00:27:f8:38:36:4f

  6: fffc06 10:00:00:27:f8:38:3e:6f 172.29.200.225  0.0.0.0        >"ACMODRFCIP01"

The Fabric has 2 switches

ACMODRFCIP01:admin> switchshow

switchName:     ACMODRFCIP01

switchType:     83.3

switchState:    Online

switchMode:     Native

switchRole:     Principal

switchDomain:   6

switchId:       fffc06

switchWwn:      10:00:00:27:f8:38:3e:6f

zoning:         OFF

switchBeacon:   OFF

FC Router:      ON

FC Router BB Fabric ID: 10

Address Mode:   0

Index Port Address Media Speed State     Proto

==============================================

   0   0   060000   id     N8   Online      FC  EX-Port  10:00:00:27:f8:20:77:32 "DRSW1_2498B24" (fabric id = 4 )

   1   1   060100   id     N8   Online      FC  EX-Port  10:00:00:27:f8:25:14:14 "DRSW2_2498B24" (fabric id = 5 )

   2   2   060200   id     N8   No_Light    FC

   3   3   060300   id     N8   No_Light    FC

   4   4   060400   id     N8   No_Light    FC  Disabled

   5   5   060500   id     N8   No_Light    FC  Disabled

   6   6   060600   id     N8   No_Light    FC  Disabled

   7   7   060700   id     N8   No_Light    FC  Disabled

   8   8   060800   --     N8   No_Module   FC  Disabled

   9   9   060900   --     N8   No_Module   FC  Disabled

  10  10   060a00   --     N8   No_Module   FC  Disabled

  11  11   060b00   --     N8   No_Module   FC  Disabled

  12  12   060c00   --     N8   No_Module   FC  Disabled

  13  13   060d00   --     N8   No_Module   FC  Disabled

  14  14   060e00   --     N8   No_Module   FC  Disabled

  15  15   060f00   --     N8   No_Module   FC  Disabled

  16  16   061000   --    --   Online      VE  VE-Port  10:00:00:27:f8:38:36:4f "" (downstream)

  17  17   061100   --    --   Offline     VE

  18  18   061200   --    --   Offline     VE

  19  19   061300   --    --   Offline     VE

  20  20   061400   --    --   Offline     VE

  21  21   061500   --    --   Offline     VE

  22  22   061600   --    --   Offline     VE

  23  23   061700   --    --   Offline     VE

     ge0  cu     1G   Online    FCIP  Copper

     ge1  cu     1G   Online    FCIP  Copper

     ge2  --     1G   No_Module FCIP  Disabled

     ge3  --     1G   No_Module FCIP  Disabled

     ge4  --     1G   No_Module FCIP  Disabled

     ge5  --     1G   No_Module FCIP  Disabled

ACMODRFCIP01:admin> portshow 0

portIndex:   0

portName: port0

portHealth: No Fabric Watch License

Authentication: None

EX_Port Mode:   Enabled

Fabric ID:      4

Front Phantom:  State: OK       Cur Dom ID: 160 WWN: 50:02:7f:83:83:e6:fe:04

Pr Switch Info:         Dom ID: 4       WWN: 10:00:00:27:f8:20:77:32

Fabric params:  R_A_TOV: 10000  E_D_TOV: 2000   PID fmt: core

Authentication Type: None

Hash Algorithm: N/A

DH Group: N/A

Edge fabric's primary wwn: N/A

Edge fabric's version stamp: N/A

portDisableReason: None

portCFlags: 0x1

portFlags: 0x10000903    PRESENT ACTIVE G_PORT U_PORT EX_PORT PHY_EX_PORT LOGICAL_ONLINE LOGIN

LocalSwcFlags: 0x0

portType:  18.0

portState: 1    Online

Protocol: FC

portPhys:  6    In_Sync         portScn:   1    Online

port generation number:    6028

state transition count:    15

portId:    060000

portIfId:    43020017

portWwn:   20:00:00:27:f8:38:3e:6f

portWwn of device(s) connected:

Distance:  normal

portSpeed:  N8Gbps

LE domain: 0

FC Fastwrite: OFF

Interrupts:        0          Link_failure: 0          Frjt:         0

Unknown:           0          Loss_of_sync: 1          Fbsy:         0

Lli:               10         Loss_of_sig:  2

Proc_rqrd:         1125       Protocol_err: 0

Timed_out:         0          Invalid_word: 2564752

Rx_flushed:        0          Invalid_crc:  0

Tx_unavail:        0          Delim_err:    0

Free_buffer:       0          Address_err:  0

Overrun:           0          Lr_in:        1

Suspended:         0          Lr_out:       1

Parity_err:        0          Ols_in:       0

2_parity_err:      0          Ols_out:      1

CMI_bus_err:       0

Port part of other ADs: No

CMODRFCIP01:admin> portshow fciptunnel 16  -c

-------------------------------------------

Tunnel ID: 16

   Tunnel Description:

   Admin Status: Enabled

   Oper Status: Up

   Compression: On (Standard)

   Fastwrite: On

   Tape Acceleration: Off

   TPerf Option: Off

   IPSec: Disabled

   QoS Percentages: High 50%, Med 30%, Low 20%

   Remote WWN: Not Configured

   Local WWN: 10:00:00:27:f8:38:3e:6f

   Peer WWN: 10:00:00:27:f8:38:36:4f

   Circuit Count: 1

   Flags: 0x00000000

   FICON: Off

   -------------------------------------------

   Circuit ID: 16.0

      Circuit Num: 0

      Admin Status: Enabled

      Oper Status: Up

      Connection Type: Default

      Remote IP: 172.28.95.5

      Local IP: 172.29.200.226

      Metric: 0

      Min Comm Rt: 40000

      Max Comm Rt: 40000

      SACK: On

      Min Retrans Time: 100

      Max Retransmits: 8

      Keepalive Timeout: 10000

      Path MTU Disc: 0

      VLAN ID: (Not Configured)

      L2CoS: (VLAN Not Configured)

      DSCP:  F:  0 H:  0 M:  0 L:  0

      Flags: 0x00000000

[Anonymous]

Hi harold,

you told you are able to ping both ends. Which cli exactly are you using ?

Could you provide portshow ipif all on both routers ?

I would not configure FCIP interface's IP in the same subnet of LAN interface's IP, I think it may be confusing.

Kind regards

--

david

[Anonymous]

Turn off EX_Port back to a standard port and let the 7800 do its thing and your fabrics will merge, I think this should solve your issue

[Anonymous]

Harold

You don't need to turn off routing to troubleshoot this type of problem. The 'flapping' GbE port could be due to several factors. One of those is marginal physical layer or an invalid commit rate. Please refer to the FCIP Admin guide for suggestions regarding commit rate configuration. There are several commands to display the port level physical layer errors. Make sure they are clean. Also, when you performed the ping command were there any lost packets?

[Anonymous]

Thanks you all for your suggestions. I changed the IP address and subnet on the management interface as I thought that was the problem. I am still have problems though. Here are the latest configurations:

ACMODRFCIP01:admin> ipaddrshow

SWITCH
Ethernet IP Address: 172.29.201.2
Ethernet Subnetmask: 255.255.255.0
Gateway IP Address: 172.29.201.1
DHCP: Off
IPv6 Autoconfiguration Enabled: Yes
Local IPv6 Addresses:
IPv6 Gateways:
ACMODRFCIP01:admin>

ACMODRFCIP01:admin> portshow ipif ge0

Port: ge0
Interface IPv4 Address    NetMask         Effective MTU  Flags
--------------------------------------------------------------
    0     172.29.200.226  255.255.255.0   1500           U R M

Flags: U=Up B=Broadcast D=Debug L=Loopback P=Point2Point R=Running
       N=NoArp PR=Promisc M=Multicast S=StaticArp LU=LinkUp
ACMODRFCIP01:admin> portshow iproute ge0

Port: ge0
IP Address      Mask            Gateway        Metric   Flags
-------------------------------------------------------------
172.28.95.0     255.255.255.0   172.29.200.1     0      U G S
172.29.200.0    255.255.255.0   *                0      U C
172.29.200.1    255.255.255.255 *                0      U H L

Flags: U=Usable G=Gateway H=Host C=Created(Interface) S=Static L=LinkLayer(Arp)

ACMODRFCIP01:admin> portshow fciptunnel all -c
-------------------------------------------------------------------------------
Tunnel Circuit  OpStatus  Flags    Uptime  TxMBps  RxMBps ConnCnt CommRt  Met
-------------------------------------------------------------------------------
16     -         Up      cf-----       7s    0.00    0.00 23689      -      -
16     0 ge0     Up      ---4--s       6s    0.00    0.00 23689    40/40    0
-------------------------------------------------------------------------------
Flags:  tunnel: c=compression m=moderate compression a=aggressive compression
                 A=Auto compression f=fastwrite t=Tapepipelining F=FICON
                 T=TPerf i=IPSec l=IPSec Legacy
Flags: circuit: s=sack v=VLAN Tagged x=crossport 4=IPv4 6=IPv6
                 L=Listener I=Initiator

ACMODRFCIP01:admin> portshow fciptunnel all --circuit --ip-address
-------------------------------------------------------------------------------
Tunnel Circuit   Local IP Address               Remote IP Address
-------------------------------------------------------------------------------
16     -
16     0 ge0     172.29.200.226                 172.28.95.5
-------------------------------------------------------------------------------

ACMODRFCIP01:admin> portshow fciptunnel all --detail
-------------------------------------------
Tunnel ID: 16
   Tunnel Description:
   Admin Status: Enabled
   Oper Status: Up
   Compression: On (Standard)
   Fastwrite: On
   Tape Acceleration: Off
   TPerf Option: Off
   IPSec: Disabled
   QoS Percentages: High 50%, Med 30%, Low 20%
   Remote WWN: Not Configured
   Local WWN: 10:00:00:27:f8:38:3e:6f
   Peer WWN: 10:00:00:27:f8:38:36:4f
   Circuit Count: 1
   Flags: 0x00000000
   FICON: Off

ACMODRFCIP01:admin> portcmd --ping ge0 -s 172.29.200.225 -d 172.29.200.226
Source IP address not configured on port ge0
ACMODRFCIP01:admin> portcmd --ping ge0 -s 172.29.200.226 -d 172.28.95.5
Pinging 172.28.95.5 from ip interface 172.29.200.226 on 0/ge0 with 64 bytes of data
Reply from 172.28.95.5: bytes=64 rtt=69ms ttl=17
Reply from 172.28.95.5: bytes=64 rtt=71ms ttl=17
Reply from 172.28.95.5: bytes=64 rtt=69ms ttl=17
Reply from 172.28.95.5: bytes=64 rtt=71ms ttl=17

Ping Statistics for 172.28.95.5:
        Packets: Sent = 4, Received = 4, Loss = 0 ( 0 percent loss)
        Min  RTT = 69ms, Max RTT = 71ms Average = 70ms

ACMODRFCIP01:admin> portshow fcipcircuit all
-------------------------------------------------------------------------------
Tunnel Circuit  OpStatus  Flags    Uptime  TxMBps  RxMBps ConnCnt CommRt  Met
-------------------------------------------------------------------------------
16     0 ge0     Up      ---4--s       8s    0.00    0.00 23715    40/40    0
-------------------------------------------------------------------------------
Flags: circuit: s=sack v=VLAN Tagged x=crossport 4=IPv4 6=IPv6
                 L=Listener I=Initiator

ACMODRFCIP01:admin> portshow fcipcircuit 16 0 --perf
   -------------------------------------------
   Circuit ID: 16.0
      Circuit Num: 0
      Admin Status: Enabled
      Oper Status: Up
      Connection Type: Default
      Remote IP: 172.28.95.5
      Local IP: 172.29.200.226
      Metric: 0
      Min Comm Rt: 40000
      Max Comm Rt: 40000
      SACK: On
      Min Retrans Time: 100
      Max Retransmits: 8
      Keepalive Timeout: 10000
      Path MTU Disc: 0
      VLAN ID: (Not Configured)
      L2CoS: (VLAN Not Configured)
      DSCP:  F:  0 H:  0 M:  0 L:  0
      Flags: 0x00000000
      Flow Ctrl State: On
      Connected Count: 23716
      Circuit Duration: 1 second
      Performance Statistics: Overall Throughput
         1188 Output Bytes
            0 Bps 30s Avg, 51 Bps Lifetime Avg
         10 Output Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         1344 Input Bytes
            0 Bps 30s Avg, 58 Bps Lifetime Avg
         10 Input Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         TCP Stats:
            2030474544 Output Bytes
            17629276 Output Packets
            1034478300 Input Bytes
            17454973 Input Packets
            Retransmits: 179746
            Round Trip Time: 71 ms
            Out Of Order: 102926
            Slow Starts: 30378

ACMODRFCIP01:admin>

ACMODRFCIP01:admin> portshow fcipcircuit 16 0 --perf --qos
   -------------------------------------------
   Circuit ID: 16.0
      Circuit Num: 0
      Admin Status: Enabled
      Oper Status: Up
      Connection Type: Default
      Remote IP: 172.28.95.5
      Local IP: 172.29.200.226
      Metric: 0
      Min Comm Rt: 40000
      Max Comm Rt: 40000
      SACK: On
      Min Retrans Time: 100
      Max Retransmits: 8
      Keepalive Timeout: 10000
      Path MTU Disc: 0
      VLAN ID: (Not Configured)
      L2CoS: (VLAN Not Configured)
      DSCP:  F:  0 H:  0 M:  0 L:  0
      Flags: 0x00000000
      Flow Ctrl State: On
      Connected Count: 23719
      Circuit Duration: 0 seconds
      Performance Statistics - Priority: F-Class
         Oper Status: Up
         Flow Ctrl State: On
         Connected Count: 23719
         Duration: 0 seconds
         1188 Output Bytes
            0 Bps 30s Avg, 1188 Bps Lifetime Avg
         10 Output Packets
            0 pkt/s 30s Avg, 10 pkt/s Lifetime Avg
         1344 Input Bytes
            0 Bps 30s Avg, 1344 Bps Lifetime Avg
         10 Input Packets
            0 pkt/s 30s Avg, 10 pkt/s Lifetime Avg
         TCP Stats:
            842879256 Output Bytes
            5231993 Output Packets
            348041148 Input Bytes
            5123401 Input Packets
            Retransmits: 177478
            Round Trip Time: 75 ms
            Out Of Order: 102227
            Slow Starts: 28129
      Performance Statistics - Priority: High
         Oper Status: Up
         Flow Ctrl State: On
         Connected Count: 23719
         Duration: 24 seconds
         0 Output Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Output Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         0 Input Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Input Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         TCP Stats:
            412988776 Output Bytes
            4314446 Output Packets
            238636240 Input Bytes
            4292694 Input Packets
            Retransmits: 820
            Round Trip Time: 71 ms
            Out Of Order: 252
            Slow Starts: 804
      Performance Statistics - Priority: Medium
         Oper Status: Up
         Flow Ctrl State: On
         Connected Count: 23717
         Duration: 23 seconds
         0 Output Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Output Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         0 Input Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Input Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         TCP Stats:
            395921476 Output Bytes
            4132614 Output Packets
            228848268 Input Bytes
            4110648 Input Packets
            Retransmits: 744
            Round Trip Time: 73 ms
            Out Of Order: 240
            Slow Starts: 732
      Performance Statistics - Priority: Low
         Oper Status: Up
         Flow Ctrl State: On
         Connected Count: 23717
         Duration: 22 seconds
         0 Output Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Output Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         0 Input Bytes
            0 Bps 30s Avg, 0 Bps Lifetime Avg
         0 Input Packets
            0 pkt/s 30s Avg, 0 pkt/s Lifetime Avg
         TCP Stats:
            378942220 Output Bytes
            3952526 Output Packets
            219084680 Input Bytes
            3930514 Input Packets
            Retransmits: 726
            Round Trip Time: 74 ms
            Out Of Order: 221
            Slow Starts: 717

ACMODRFCIP01:admin>

cmofcip01:admin> portcmd --ping ge0 -s 172.28.95.5 -d 172.29.200.226
Pinging 172.29.200.226 from ip interface 172.28.95.5 on 0/ge0 with 64 bytes of data
Reply from 172.29.200.226: bytes=64 rtt=74ms ttl=17
Reply from 172.29.200.226: bytes=64 rtt=67ms ttl=17
Reply from 172.29.200.226: bytes=64 rtt=70ms ttl=17
Reply from 172.29.200.226: bytes=64 rtt=69ms ttl=17

Ping Statistics for 172.29.200.226:
        Packets: Sent = 4, Received = 4, Loss = 0 ( 0 percent loss)
        Min  RTT = 67ms, Max RTT = 74ms Average = 70ms
acmofcip01:admin>

ACMODRFCIP01:admin> switchshow
switchName:     ACMODRFCIP01
switchType:     83.3
switchState:    Online
switchMode:     Native
switchRole:     Subordinate
switchDomain:   6 (unconfirmed)
switchId:       fffc06
switchWwn:      10:00:00:27:f8:38:3e:6f
zoning:         OFF
switchBeacon:   OFF
FC Router:      ON
FC Router BB Fabric ID: 10
Address Mode:   0

Index Port Address Media Speed State     Proto
==============================================
   0   0   060000   id     N8   Online      FC  EX-Port  10:00:00:27:f8:20:77:32 "DRSW1_2498B24" (fabric id = 4 )
   1   1   060100   id     N8   Online      FC  EX-Port  10:00:00:27:f8:25:14:14 "DRSW2_2498B24" (fabric id = 5 )
   2   2   060200   id     N8   No_Light    FC
   3   3   060300   id     N8   No_Light    FC
   4   4   060400   id     N8   No_Light    FC  Disabled
   5   5   060500   id     N8   No_Light    FC  Disabled
   6   6   060600   id     N8   No_Light    FC  Disabled
   7   7   060700   id     N8   No_Light    FC  Disabled
   8   8   060800   --     N8   No_Module   FC  Disabled
   9   9   060900   --     N8   No_Module   FC  Disabled
  10  10   060a00   --     N8   No_Module   FC  Disabled
  11  11   060b00   --     N8   No_Module   FC  Disabled
  12  12   060c00   --     N8   No_Module   FC  Disabled
  13  13   060d00   --     N8   No_Module   FC  Disabled
  14  14   060e00   --     N8   No_Module   FC  Disabled
  15  15   060f00   --     N8   No_Module   FC  Disabled
  16  16   061000   --    --   Online      VE  VE-Port  10:00:00:27:f8:38:36:4f "" (upstream)
  17  17   061100   --    --   Offline     VE
  18  18   061200   --    --   Offline     VE
  19  19   061300   --    --   Offline     VE
  20  20   061400   --    --   Offline     VE
  21  21   061500   --    --   Offline     VE
  22  22   061600   --    --   Offline     VE
  23  23   061700   --    --   Offline     VE
     ge0  cu     1G   Online    FCIP  Copper
     ge1  cu     1G   Online    FCIP  Copper
     ge2  --     1G   No_Module FCIP  Disabled
     ge3  --     1G   No_Module FCIP  Disabled
     ge4  --     1G   No_Module FCIP  Disabled
     ge5  --     1G   No_Module FCIP  Disabled

ACMODRFCIP01:admin> fabricshow
Switch ID   Worldwide Name           Enet IP Addr    FC IP Addr      Name
-------------------------------------------------------------------------
  3: fffc03 10:00:00:27:f8:38:36:4f                               >
  6: fffc06 10:00:00:27:f8:38:3e:6f 172.29.201.2    0.0.0.0         "ACMODRFCIP01"

The Fabric has 2 switches

ACMODRFCIP01:admin>

ACMODRFCIP01:admin> portshow fciptunnel all -perf -params
portshow: invalid option -- e
portshow: invalid option -- r
portshow: invalid option -- f
portshow: invalid option -- a
portshow: invalid option -- r
portshow: invalid option -- a
portshow: invalid option -- m
-------------------------------------------------------------------------------
Tunnel Circuit    TxMBps  RxMBps ComRatio RTT(ms)    ReTx  OutOfSeq SlowStart
-------------------------------------------------------------------------------
16     -            0.00    0.00  1.00:1     -        131       198        27
16     0 ge0        0.00    0.00    -       72        131       198        27
-------------------------------------------------------------------------------

ACMODRFCIP01:admin>

ACMODRFCIP01:admin> portshow fciptunnel all -perf -tpc -c
portshow: invalid option -- e
portshow: invalid option -- r
portshow: invalid option -- f
-------------------------------------------
Tunnel ID: 16
   Tunnel Description:
   Admin Status: Enabled
   Oper Status: Up
   Compression: On (Standard)
   Fastwrite: On
   Tape Acceleration: Off
   TPerf Option: Off
   IPSec: Disabled
   QoS Percentages: High 50%, Med 30%, Low 20%
   Remote WWN: Not Configured
   Local WWN: 10:00:00:27:f8:38:3e:6f
   Peer WWN: 10:00:00:27:f8:38:36:4f
   Circuit Count: 1
   Flags: 0x00000000
   FICON: Off
   Oper Status: Up
   Flow Ctrl State: Off
   Connected Count: 21
   Tunnel Duration: 6 seconds
      Compression Statistics:
         0 Uncompressed Bytes
         0 Compressed Bytes
           1.00 : 1 Compression Ratio
      Performance Statistics: Overall Throughput
         1604 Output Bytes
            53 Bps 30s Avg, 267 Bps Lifetime Avg
         14 Output Packets
            0 pkt/s 30s Avg, 2 pkt/s Lifetime Avg
         1760 Input Bytes
            58 Bps 30s Avg, 293 Bps Lifetime Avg
         14 Input Packets
            0 pkt/s 30s Avg, 2 pkt/s Lifetime Avg
         TCP Stats:
            18468 Output Bytes
            183 Output Packets
            10928 Input Bytes
            181 Input Packets
            Retransmits: 146
            Round Trip Time: 73 ms
            Out Of Order: 219
            Slow Starts: 30
   -------------------------------------------
   Circuit ID: 16.0
      Circuit Num: 0
      Admin Status: Enabled
      Oper Status: Up
      Connection Type: Default
      Remote IP: 172.28.95.5
      Local IP: 172.29.200.226
      Metric: 0
      Min Comm Rt: 40000
      Max Comm Rt: 40000
      SACK: On
      Min Retrans Time: 100
      Max Retransmits: 8
      Keepalive Timeout: 10000
      Path MTU Disc: 0
      VLAN ID: (Not Configured)
      L2CoS: (VLAN Not Configured)
      DSCP:  F:  0 H:  0 M:  0 L:  0
      Flags: 0x00000000
      Flow Ctrl State: Off
      Connected Count: 21
      Circuit Duration: 6 seconds
      Performance Statistics: Overall Throughput
         1604 Output Bytes
            53 Bps 30s Avg, 267 Bps Lifetime Avg
         14 Output Packets
            0 pkt/s 30s Avg, 2 pkt/s Lifetime Avg
         1760 Input Bytes
            58 Bps 30s Avg, 293 Bps Lifetime Avg
         14 Input Packets
            0 pkt/s 30s Avg, 2 pkt/s Lifetime Avg
         TCP Stats:
            1714740 Output Bytes
            15278 Output Packets
            895004 Input Bytes
            15134 Input Packets
            Retransmits: 146
            Round Trip Time: 73 ms
            Out Of Order: 219
            Slow Starts: 30
      -------------------------------------------
      TCP Connection 16.0:7931587
         Priority: F-Class
         Flags: 0x00000000
         Duration: 7 seconds
         Local Port: 49232
         Remote Port: 3225
         Max Seg Size: 1420
         Adaptive Rate Limiting Statistics:
            None (F-Class)
         Sender Statistics:
            Bytes Sent: 8088
            Packets Sent: 78
            Round Trip Time 69 ms, HWM 0 ms, Variance 2, HWM 3
            Send Window: 20971008 bytes, scale: 9
            Slow Starts: 0
            Slow Start Threshold: 16777216
            Congestion Window: 16778636
            TCP Op State: slow start
            Next Seq: 0x1d8cd25e, Min: 0x1d8cd25e, Max: 0x1d8cd25e
            Unacked data: 0
            Retransmit Timeout: 100 ms, Duplicate ACKs 0
            Retransmits: 0, max: 0
            Fast ReTx: 0, HWM 0, Slow ReTx: 0
         Receiver Statistics:
            Bytes Received: 4940
            Packets Received: 79
            Receive Window: 20971520 Bytes, max: 20971520
            Negotiated Window Scale: 9
            RecvQ Bytes: 0
            RecvQ Next: 0xfdc22fc6 Min: 0xfdc22fc6 Max: 0xff022fc6
            Out Of Sequence Pkts: 0, HWM 0, Total 0
         Keepalive:
            Keepalive Timeout: 60 s
            Keepalive Interval: 15 s
            Inactivity: 120 s
      -------------------------------------------
      TCP Connection 16.0:7932177
         Priority: Low
         Flags: 0x00000000
         Duration: 4 seconds
         Local Port: 49235
         Remote Port: 3225
         Max Seg Size: 1420
         Adaptive Rate Limiting Statistics:
            Min Rate: 8000 kbps
            Max Rate: 40000 kbps
            Cur Rate: 8000 kbps
            Soft Limit: 40000 kbps
         Sender Statistics:
            Bytes Sent: 2728
            Packets Sent: 27
            Round Trip Time 70 ms, HWM 0 ms, Variance 1, HWM 0
            Send Window: 999936 bytes, scale: 9
            Slow Starts: 0
            Slow Start Threshold: 16777216
            Congestion Window: 16778636
            TCP Op State: slow start
            Next Seq: 0x1dae90a8, Min: 0x1dae90a8, Max: 0x1dae90a8
            Unacked data: 0
            Retransmit Timeout: 100 ms, Duplicate ACKs 0
            Retransmits: 0, max: 0
            Fast ReTx: 0, HWM 0, Slow ReTx: 0
         Receiver Statistics:
            Bytes Received: 1584
            Packets Received: 26
            Receive Window: 999936 Bytes, max: 999936
            Negotiated Window Scale: 9
            RecvQ Bytes: 0
            RecvQ Next: 0xf39a0b14 Min: 0xf39a0b14 Max: 0xf3a94d14
            Out Of Sequence Pkts: 0, HWM 0, Total 0
         Keepalive:
            Keepalive Timeout: 60 s
            Keepalive Interval: 15 s
            Inactivity: 120 s
      -------------------------------------------
      TCP Connection 16.0:7931980
         Priority: Medium
         Flags: 0x00000000
         Duration: 5 seconds
         Local Port: 49234
         Remote Port: 3225
         Max Seg Size: 1420
         Adaptive Rate Limiting Statistics:
            Min Rate: 12000 kbps
            Max Rate: 40000 kbps
            Cur Rate: 12000 kbps
            Soft Limit: 12000 kbps
         Sender Statistics:
            Bytes Sent: 3460
            Packets Sent: 35
            Round Trip Time 72 ms, HWM 0 ms, Variance 3, HWM 0
            Send Window: 999936 bytes, scale: 9
            Slow Starts: 0
            Slow Start Threshold: 16777216
            Congestion Window: 16778636
            TCP Op State: slow start
            Next Seq: 0x1da327e5, Min: 0x1da327e5, Max: 0x1da327e5
            Unacked data: 0
            Retransmit Timeout: 100 ms, Duplicate ACKs 0
            Retransmits: 0, max: 0
            Fast ReTx: 0, HWM 0, Slow ReTx: 0
         Receiver Statistics:
            Bytes Received: 1996
            Packets Received: 34
            Receive Window: 999936 Bytes, max: 999936
            Negotiated Window Scale: 9
            RecvQ Bytes: 0
            RecvQ Next: 0xc937389b Min: 0xc937389b Max: 0xc9467a9b
            Out Of Sequence Pkts: 0, HWM 0, Total 0
         Keepalive:
            Keepalive Timeout: 60 s
            Keepalive Interval: 15 s
            Inactivity: 120 s
      -------------------------------------------
      TCP Connection 16.0:7931784
         Priority: High
         Flags: 0x00000000
         Duration: 6 seconds
         Local Port: 49233
         Remote Port: 3225
         Max Seg Size: 1420
         Adaptive Rate Limiting Statistics:
            Min Rate: 20000 kbps
            Max Rate: 40000 kbps
            Cur Rate: 20000 kbps
            Soft Limit: 20000 kbps
         Sender Statistics:
            Bytes Sent: 4192
            Packets Sent: 43
            Round Trip Time 73 ms, HWM 0 ms, Variance 4, HWM 4
            Send Window: 999936 bytes, scale: 9
            Slow Starts: 0
            Slow Start Threshold: 16777216
            Congestion Window: 16778636
            TCP Op State: slow start
            Next Seq: 0x1d986941, Min: 0x1d986941, Max: 0x1d986941
            Unacked data: 0
            Retransmit Timeout: 100 ms, Duplicate ACKs 0
            Retransmits: 0, max: 0
            Fast ReTx: 0, HWM 0, Slow ReTx: 0
         Receiver Statistics:
            Bytes Received: 2408
            Packets Received: 42
            Receive Window: 999936 Bytes, max: 999936
            Negotiated Window Scale: 9
            RecvQ Bytes: 0
            RecvQ Next: 0xbceb213d Min: 0xbceb213d Max: 0xbcfa62c5
            Out Of Sequence Pkts: 0, HWM 0, Total 0
         Keepalive:
            Keepalive Timeout: 60 s
            Keepalive Interval: 15 s
            Inactivity: 120 s

ACMODRFCIP01:admin>

[Anonymous]

Could you provide (only) portshow fciptunnel all and portshow fcipcircuit all on both sides ?

Did you try first without enabling Fastwrite ? BTW if you are not sure you need Fastwrite and don't know what it's done for, don't use it.

Kind regards,

--

david

[Anonymous]

Harold,

What version of FOS are you using? Additionally, has your array manufacturer recommended the Fast Write configuration? What are the messages displaying in the error log when the ports are 'flapping'? Lastly - do you have a case open with your support provider TAC?