I have recently set up a FCIP tunnel between a couple 7800's and am seeing the tunnel bouncing quite frequently.
Setup is as follows
Site 1 - 2 7800's - site1_FCIP_A and Site1_FCIP_B routers - V7000 connected directly to the FC ports - V7K port 7's on FCIP_A switch and V7K port 8's on FCIP_B switch. Ge0 ports on each switch defined with an ipinterface, iproutes defined also.
Site 2 - 2 7800's - site2_FCIP_A and site2_FCIP_B routers - V7000 connected directly to the FC ports - V7K port 7's on FCIP_A switch and V7K port 8's on FCIP_B switch. Ge0 ports on each switch defined with an ipinterface, iproutes defined also.
On FCIP_A switches - tunnel 16 created with compression and ipsec, because advanced extension not licensed, commrate was set to -b 1000000 -B 1000000 (think this may be wrong, maybe should have set to 0 without the -b and -B switches). Same config applies on the FCIP_B switches - tunnel 16 created
The WAN between sites is 10Gb - so with this setup, the aggregate is potentially 2Gb for the tunnels - 1Gb on FCIP_A and 1 Gb on FCIP_B.
the tunnel created perfectly fine and the fabrics merged as expected, but i cannot figure out why the tunnel keeps dropping.
looking at the portshow fciptunnel 16 on both switches on one site, i am seeing one switch doing way more than the other with alot of retransmits, out of order and slow starts - which i do not quite know what those last two indicate in terms of troubleshooting.
the network team has indicated they are not seeing any issues with the WAN link, so i suspect something with the tunnel configuration or something of the sort causing the issues.
Any suggestions/hints on where to start troubleshooting?
What is the FOS release installed on the 7800?
Are you running Ficon or Open systems?
running 7.4.1b, no ficon - it is all Open Systems replication between v7000 arrays.
Please find attached fcip admin guide that is providing troubleshooting guidance starting on page 109.
I have actually got that and have read it through multiple times.
I am wondering if i did something wrong in my setup.
basically i just create a single ipif on the Ge0 port on both sides, created the route, verified everything was pingable from ip to ip, gateway to gateway etc and created a tunnel on both sides with the same parameter.
now the only thing i did with the tunnel creation is i specified the -b and -B parameters the same - 1000000Kbps . Advanced Extension license is not used, so it would not let me set a lower -b and higher -B commrate.
Wondering if i should have set that to 0 and not specify a -b and -B parameter.
Also the FC ports are negotiating at 8Gbps, the Ge ports are set to auto. do i need to do anything to the FC ports to dial them down? is it trying to send too much through the tunnel and it cannot handle it?
There are some defects in the current release that you are using relative to IPsec and compression.
Attached is the FOS release note of 7.4.2b.
As a guidance, potentially upgrade to the latest release (not 100% sure that is the issue), or try to remove compression and/or IPsec and see if better as a test.
Found out that the IP's initially provisioned for the FCIP switches ipif's on one site were routed through an encrypting appliance. So we reconfigrued the FCIP IPIF's to a subnet that bypasses this appliance, and the tunnels have been stable, still seeing lots of other issues now, but as far as the tunnel staying up, it looks ok.
I'm not sure if the issue is still presented, but we have the very similar issue with our 7800 switches between sites. The tunnel bouncing, flapping was visible well while monitoring them with INA. Our issue was solved, after we upgraded to FOS lvl 8.1.2a, so probably it's a deffect that was fixed in this release. We ran many circlees due to this issue.Regards, Tamas.