Interface Traffic Probe 5.01 and Max Interface Speed Alerts

[Anon Anon]

I am wondering if others are seeing this issue with the latest version or previous versions of interface_traffic  - periodically we get criticial alerts indicating that the max interface speed cannot be determined yet when you go into the probe and select monitor you see the correct speed and traffic flowing - see screenshot.  Support seems to believe that this might related to certain types of cisco equipment yet from my perspective this is something to do with version 5.01 since I did not see this issue (at least to the degree I am seeing now)  in version 4.94 of the probe. Since 99% of my equipment is Cisco - I can't really disprove that it's related to Cisco. I also can't downgrade the probe since it fixed a different more major bug. 

 

I am curious if others are seeing issues like this and with what type of equipment and prove version number.

[Scott Stuessi]

We have been seeing these alerts also since updating to 5.01. All seems to be working fine, then the automatic interface speed detection stops working thus the alarm. To clear the alarms we have gone the route of Temporarily changing the detection to manual and hardcoded the port speed.  PIA.

 

We've only been on 5.01 for three days.

 

Happens across all Cisco models.

[Anon Anon]

Thank you for your reply sstuessi - yes I was given that hardcoded approach/work around as well - This is just not feasaible in our env. 

[Anon Anon]

When this happens, does the alarm continuously repeat? Does it come in once and clear? Does it come in once and then never repeat?

 

Just wondering what to expect if/when we encounter this issue. That seems likely. 

 

Thanks!

[Anon Anon]

It's very random - at least in my env. They come in then clear. Some times they repeat, sometime I get some that don't clear yet the count does not increments. Cold Starts seem to clear the current alarms only to introduce different ones. 

[Scott Stuessi]

For us the alarm will continuously repeat. Will only clear if interface speed is set manually.

[Anon Anon]

Thanks to both of you for sharing. Good to know what we can look forward to.

[Anon Anon]

Having the same issue here, but not with Cisco. This problem happens in 3Com switches too, fiber channel and Ethernet. 

 

Interface_traffic version : 5.11

 

Any idea about why this happens?

[Anon Anon]

When this happens I get the following in the logfile (this was from a debug version provided by support) 

 

pr 13 15:36:15:684 ift: 10.129.177.148/40701->10.129.177.148/48001 (573): 
Apr 13 15:36:15:684 ift: 10.129.177.148/40701<-10.129.177.148/48001 (84): 
Apr 13 15:36:15:684 [1116055872] ift: RREPLY: status=OK(0) <-10.129.177.148/48001 h=38 d=28 
Apr 13 15:36:15:684 [1116055872] ift: ciClose - [C3F6E2688A15C0029A2D3CEAE7B9CA58C] 
Par 13 15:36:15:684 [1116055872] ift: *************CheckOctets Interface Speed found 0! 
Apr 13 15:36:15:684 [1116055872] ift: ************Admin Status 1 
Apr 13 15:36:15:684 [1116055872] ift: ************Alias INSTRUCTIONAL VLAN 
Apr 13 15:36:15:684 [1116055872] ift: ************Operator State 2 
Apr 13 15:36:15:684 [1116055872] ift: ************IfSpeed 0.000000 
Apr 13 15:36:15:684 [1116055872] ift: ciOpen - cache path: /opt/nimsoft/niscache 
Apr 13 15:36:15:684 [1116055872] ift: ciGetDeviceIdentifiers - 10.0.0.184 found (remote device) [DBDB8504269BD43276A189883F1157ECC] 
Apr 13 15:36:15:684 [1116055872] ift: SendCiAlarm - source=51196-m,sev=5,subsys=1.1.3, The max interface speed (ifSpeed) of 'GigabitEthernet1/14' on '10.0.0.184' could not be determined! Traffic alarms in percent of max speed cannot be issued. Please override interface speed or set alarm options on actual values. 
Apr 13 15:36:15:684 [1116055872] ift: sockWrite: first 20 bytes of buf = <nimbus/1.0 255 299^M 

 

Support is thinking that at that point in time the speed could not be determined, although almost immediately after getting this alert, I can either snmpwalk the device and get the max speed value or use the monitor feature inside the probe itself to get the max speed value. It is possible that something on the device side of things is causing this, however this began occurring with a much greater degree of frequency with probe version 5.01 and greater.  

 

Support has just provided me with a version of the probe that may address this issue, but I just received this about 2 hours ago and have not tested yet.  I would encourage you to open a support ticket if you have not already. This might help expedite a resolution. You can reference my case 72923 

[Anon Anon]

I encountered the same issue as well with several Cisco devices. A case opened with Nimsoft support and we have tried to capture from log level 5 and Wireshark. We found that when the probe is requesting for ifSpeed (so that it can be compared and calculated upon flowing packets) - there is no reply from the network device.

It could be due discarded packet or anything else since that the protocol itself is UDP. Temporary fix is to set pre-defined interface speed until Nimsoft engineering has figured out the mechanism to have retries or any other method before it conclude on generating the alarm.

FYI, this is tested using v5.11

[Anon Anon]

Thanks Slash that is great info. I can't hardcode the speed on all my devices so I was hoping that at the very least the probe would re-attempt and/or clear the critical alert that is generated on the next polling cycle so that I could build a suppression rule in the NAS for these alerts if the count was less than say 5. As it stands these alerts only seem to clear if I restart the probe. 

[Anon Anon]

That's the problem that we encountered currently as well. Over hundreds of Cisco devices that we monitored, luckily only several that has this issue and we have no choice but to set it as static and put alarm suppression in nas. Looking forward to Nimsoft to provide an enhancement.

[Anon Anon]

I've noticed this same random max speed behavior going back to 4.70. It seemed to have started when i had set the hidden cfg for 4.70 value /setup/only_kbits = 1 which is available in GUI around 4.90+.  The option allows you to turn off the KBytes qos and just send bits and % speeed.

 

Courious to know if others having this max speed issue have this set in raw config.  It can also be found in Advanced Tab under setup as checkbox if you have 4.90+

[Anon Anon]

I am being told from support that the following will be introduced in a beta release of the probe (June timeframe) to address this issue:

 

 

1) We have implemented a retry mechanism during retrieval of IfSpeed during interfaceDiscover (at start of probe).

 

2) In rare scenarios above mechanism may also fail. For this we are checking IfSpeed OID at each interval. If IfSpeed is 0 then we are again querying device for it.

 

 

Thought I would share

[Anon Anon]

Thanks for the info. I got the beta probe as well and inquiring for more info on how the fix is done. Would you happen to know how many retries and is it configurable either using probe GUI or raw configure?

[Anon Anon]

I do not - in fact I was not actually provided a version of this probe to test either.  I have the same questions as well and was going to see if I could determine the answer once I obtained a copy of the probe. Sorry I don't have a better answer