Rapid increasing er_bad_os at 8 Gbit speed

[Anonymous]

Andreas,

Hitachi arrays follow the standard. Very strict. End of Story. The problem with the Brocade switches (or rather the administrators) have, is that they have options. :-) If you choose the incorrect one it doesn't work with any vendors array.

According to the standard during init state the ports still have to use IDLE primitives irrespective of speed. Upon entering the Active state both ports have to switch to ARBff primitives after sending at least 6 IDLE and recieving at least 2. (Also normal process in the FC-FS part). So only one mode (2) adheres to that. The reason Brocade came up with mode 3 is because some vendors had 8G implementations while some issues in the standard weren't totally fleshed out. There was one issue were there could be a deadlock during init phase in which both ports could never come online. One port would send a NOS primitive in which the OLS/LR/LRR primitive sequence started off. This then would still end up in this deadlock situation. For this reason mode 3 was implemented where the Brocade switch waits for a NOS and then switches to ARBff.

As you can see all these 4 modes serve their purpose because some vendors had deviations in their implementations during the creation of this part of the standard. Brocade just gave them the options. Only mode 2 adheres strictly to the standard.

Hope this explains some of the reasons behind these options as well as some internals.

Cheers

E

[Anonymous]

Mahendran,

Any update on if you were able to resolve your issues?

Thanks

[Anonymous]

I have the same issue, but the issue is on the 8G switch port where the VMWare host is connected. Does this problem have a fix?

Can I try to peg down the speed of the port to 4G and see if it resolves the issue?

Thanks,

SK

[Anonymous]

Depends on your FOS code.

Check the fill word setting on the SAN switch port and have a play with it.

Try option mode 2 or 1. A change on this settings will disrupt the IO due to a link reset.

Andreas

[Anonymous]

Thank you for the prompt reply, Andreas.

The interesting point I note is , this is happening on the switch ports on the two fabrics where the host is connected to.

One switch is at 6.4.1b and other side is on 6.3.0b.

Ok. Does this change prove to solve the issue?

[Anonymous]

Currently I would say this is not a real issue. 

Traffic will flow through the switch without a big performance issue.

I assume that on FOS 6.3.0b this mode is not present.

So change it on FOS 6.4.1b and check if everything is OK on the server side.

Andreas

[Anonymous]

v6.3.0b has only 2 modes

0 -idle-idle

1 -arbff-arbff

v6.4.1b has 4 modes

0 -idle-idle

1 -arbff-arbff

2 -idel-arbff

3- aa-then-ia

Technically I believe that this problem can be fixed, if the host can be moved to some existing 4G connections. Am I assuming it right?

[Anonymous]

And..to add to that regarding performance. The VM team is seeing an impact in the form of backups running slow. I'm not completely sure, if this would be the reason, but at the moment, I cannot find any other errors.

Thanks,

SK

[Anonymous]

In my case it doesn't had an impact on the performance.

The switch expect other fill words than the HBA is sending and this causes the error counter to increase.

Between each data frame the switch expect arbff frames but the HBA is sending idle frames. This is due to different implementations of 8GB FC standard.

Brocade found out that abrff are better than idle on 8gbit speed. If you slow down the port speed to 4Gbit everything is fine because both devices are sending idles.

Idles frames or arbff frames should not have a performance issue.

I suggest to look somewhere else to find the performance issue.

Andreas