Brocade Fibre Channel Networking Community

Expand all | Collapse all

DIstance limits for FABRICS

  • 1.  DIstance limits for FABRICS

    Posted 11-19-2018 07:53 AM

    Hi, can anyone point me to documention containing Brocade distance limits for a SAN fabric spanning two Datacentres?

    We may be looking at creating a 3rd datacentre 300 miles away.


    Our current fabric spans 15 km via a WDM.


    thanks rob


  • 2.  Re: DIstance limits for FABRICS

    Posted 11-20-2018 11:02 AM

    Search on for: SOLUTION DESIGN GUIDE Brocade Extension Solution Design Guide 53-1004573-01 9 September 2016 2016


    This is the latest regarding distance solutions. If you are planning to go native FC over DWDM, there are a lot of tests that would be implemented before turning it over to production. It's a big subject, but I would strongly recommend knowing a lot about the traffic type, and payload size of the data being sent. Small data payloads of 100-800 bytes is not going to work well over distance due to the buffers required, and the distance latency. Brocade uses buffers for flow control. Getting the right setup may take several trial and refinement jobs.


  • 3.  Re: DIstance limits for FABRICS

    Posted 11-21-2018 09:57 AM

    Regarding buffer flow over long distant links. Brocade switch ports configured in "default" mode provide a default number of buffers which is typically sufficient for local data center i/o. Long distance links can often require an adjustment to quantity of buffers allocated. The number of buffers required to keep a long distant link full when operating at max link speed capacity is dependent on: link speed, distance and frame size.


    Every fibre channel frame requires a buffer. The smaller the average frame
    size the more frames can be "fitted" onto the long distant link to keep it full and therefore,

    putting more (smaller) frames on a link requires more buffers.

    As an example, a 10km 8Gb link requires 40 buffers to keep the link fully utilized when using full size frames (2048 payload bytes). If the same 10km 8Gb link has average frame size of 512 bytes, the number of configured buffers would be 160.

    You can find the tx and rx average frame size from the portstats64show output or by calculating the number of tx frames vs.
    tx 4-byte words from portstatshow command. Suggest you first clear the port counter and capture new output while the expected
    i/o is running over the port to ensure the measurement represents the actually characteristic of the i/o.

    Other thoughts;
    If your long distance equipment (ie., DWDM) is passive, which is most common, you may want to explore the Brocade portlongdistance and portbuffercalc commands to aid in setting up the number of buffers for
    the long distance port.

    If your long distance equipment is "spoofing" (providing it's own credits for the long distant link), which is rare in my experience,
    then a default port config should be fine.

    You can find how many buffers are available for the Brocade switch using portbuffershow. Directors will have output for each line

    Lastly, Brocade has an Excel spreadsheet to aid in Buffer Credit calculations:


  • 4.  Re: DIstance limits for FABRICS

    Posted 11-22-2018 02:49 AM