THE COMMUNICATION LINK IS DOWN Alarm

[Aurimas Plunge]

Hi,

I get minor alarms THE COMMUNICATION LINK IS DOWN even though in Event configuration manager severity by default is None.

 

 

How does that happen? and What is the difference between  BAD LINK DETECTED (Critical 0x00010d11) and THE COMMUNICATION LINK IS DOWN (Minor 0x00220001)?

[ackjo04]

Could you please double check the event id again? I believe the actual event id for the "THE COMMUNICTION LINK DOWN" alarm is 0x220101 and the assciated probable cause id is 0x220001.

 

 

The "BAD LINK ALARM" critical is asserted on the interface model and can be enabled by one of the following:

 

1. Link down trap if the AlarmOnLinkDownTrap attribute 0x11fc2 is set to Check Status (1) on the interface model

2. PollPortStatus attribute 0x1280a set to Yes on the interface model

3. Live Links (Pipes) is enabled on the link associated with the interface which sets the ok_to_poll attribute 0x11dd8 attribute to Yes on the interface model

4. Port Fault Correlation is enabled at the Fault Isolation model

 

The "THE COMMUNICTION LINK DOWN" minor alarm is asserted on the device model and can be enabled by setting the AssertLinkDownAlarm attribute 0x12957 to Yes on the interface model. This tells Spectrum to assert the "THE COMMUNICTION LINK DOWN" minor alarm on the associated device model for this interface if a link down trap is received.

 

Joe

[Aurimas Plunge]

Thanks Joe,

all are clear to now.

[John OMahony]

Hi Joe... ackjo04

I wonder could you help me with something related to this or point me at the docs for an explanation please ...

 

We have a number of devices that we are getting link up/down traps for. On those devices certain interfaces are configured with AlarmOnLinkDownTrap to Check Status and AssertLinkDownAlarm, GeneratePortStatusAlarms, ok_to_poll and PollPortStatus all set to Yes. 

For those interfaces when a Link Down trap is received we get a 0x220001 event followed by a 0x220101 event and a Communication Link Down alarm is created. 

When the Link Up trap is later received I see the event 0x220002, however I don't always see any alarm clear event, 0x220102. As a consequence we have lots of Comm Link Down alarms that are not being cleared, although I have seen cases where they are cleared by polling.

 

First off I wonder could you confirm if my understanding of what's happening is correct? 

I expect the link down trap to create a 0x220001 event followed by a 0x220101 event which created the alarm. When the link up is received I'm expecting a 0x220002 event to be followed by a 0x220102 event to clear the alarm. Is that what you would expect?

 

And can you explain why I don't get the 0x220102 event, despite seeing the 0x220002 event? Is there some setting in Spectrum that might prevent the 0x220102 event being created?

 

Hopefully you can help. If you need me to I can create a support call.

 

Thanks, John

[Jason Meader]

Hi John,

  As long as the interface isn't flapping and you haven't customized the events then it should work as you noted.  Please open a case for us to investigate.

Cheers

Jay

[John OMahony]

Hi Jason

I will open a case as we had a specific case where this occurred in testing earlier in the week.

 

I was also curious how it works with a flapping interface. I can understand why you would not want the alarm raising, clearing, etc.... but can you explain what happens in the case of a flapping interface as I have also seen examples of this?

 

Regards, John

[Jason Meader]

Hi John,

 

The EventDisp probably describes it best:

 

0x00220001 E 50
0x00220002 E 30 R { 1 } CA.EventSequence, \
"0x00220006 1:1,3-6:3-6", 60, "0x00220001 "
# The following rule is to detect flapping interfaces
# When customizing it please be aware that events 0x220007 and
# 0x220008 should still be generated in appropriate pairs, as these
# are used internally in determining if device configuration should
# happen on link-up traps or not
# When we see at least 15 flaps in 5 mins, generate excessive flap
# event 0x220007. The excessive flap clear event 0x220008 will
# afterwards be generated when not a single flap has been seen anymore
# for at least 10 minutes
0x00220006 E 50 R CA.InterfaceFlapRule, 15, 300, 600, "0x00220007 -:-", "0x00220008 -:-"
0x00220007 E 50 A 1,0x00220007,1
0x00220008 E 50 C 0x00220007,1

 

Let me know if that doesn't answer your question.

Cheers!

Jay

[John OMahony]

Hi Jay

I'm afraid that doesn't answer my question fully. I was wondering what happens to the 0x220102 event in the case of flapping. The EventDisp doesn't seem to explain that. Is that event not raised when there is a flapping event raised? 

 

In truth I don't really understand how the 0x220102 event is raised. Is that down to internal Spectrum logic? I also see instances where the event 0x220103 is used to clear the Link Down alarm. What determines which of those is used to clear the alarm?

 

I guess if there was some documentation or knowledge base article that explains exactly what happens for link up/down trap processing that would be useful. Is there anything like that?

 

 

Regards, John

[sirha02]

Hi John,

 

We are seeing this because, in Spectrum, traps are always processed at the device level and not at the interface level.

 

In order to process the link up/down traps on the interface model, when the traps are received, Spectrum reads the value of the ifIndex varbind sent with the trap, checks the status of the associated interface model and if ifOper down ifAdmin up, will assert the critical “BAD LINK DETECTED” alarm on the interface model. Now, if the AssertLinkDownAlarm attribute id 0x12957 is set to Yes on that interface model (as in your environment), Spectrum asserts the minor “THE COMMUNICATION LINK IS DOWN” alarm on the associated device model.

 

This minor alarm should be cleared when either the 0x00220102 or 0x00220103 events are generated on the device model.

 

You could also have a look at the below interesting documents:

 

https://support.ca.com/us/knowledge-base-articles.TEC602286.html

https://support.ca.com/us/knowledge-base-articles.TEC517801.html

 

Thanks

[John OMahony]

Thanks for that explanation, which is pretty much what I thought.

 

So back to my original question, which is why don't we see the 0x220102 event even though we get the 0x220002 event. Do you know of something that might prevent the creation of the 0x220102 event when we receive a Link Up trap for an interface that had previously had a Link Down trap? 

 

Thanks, John

[sirha02]

Hi,

 

Am not sure about 0x220002 but perhaps it is asserted on the interface.

 

There is an underlying code used to process the link down and link up traps. Spectrum looks for device model with IP Address not interface to process the traps. As the events 0x00220102 or 0x00220103 events are generated on the device model, you would not see them on the interface.

[Olaf Schalenberg]

Hi

 

One thing what will prevent the creation of 0x00220102  are outstandig Link Up traps or positive polling for additional different ports.

That means there is only one "Communication Link Is Down" alarm on the device, with possible multiple occurrences.

The alarm will only be cleared if ALL alarmed instances are up again.

 

In addition it is very confusing that the alarm always shows the Interface information/Instancenumber of the first occurrence, even this instance maybe has already sent the LInk Up trap.

So far I haven't found a way to figure out which occurence of which instance is blocking 0x00220102 or how to reset the alarm table of the underlying Inferencehandler. This is required if one interface instance number will never come up again by configuration. In this case the Link Up trap or positive polling will never happen and this blocks further automatically clearing of the alarm for any new occurrence.

 

Regards,

Olaf

[John OMahony]

Hi Olaf

That is exactly what I was missing ... I can make some sense of it now. Thanks for your input 

 

Regards, John

[Olaf Schalenberg]

For the history:

I had a very good session with CA Support about the current behavior and CA will think about the following improvements

-to clear the internal Inferencehandler alarm table for all outstanding alarmed instances, if the single CLID alarm will be cleared in Spectrum manually

-to clear all outstanding alarmed instances, if a Reconfigure Action will be done on the device or Interface table

-to provide information to the operator about all outstanding alarmed instances in OneClick

we may see a change in behavior in a future release (>= 10.2.3).

 

Regards,

Olaf

[PDKSWAN KSWAN]

Hi All,

am looking to assert this  communication link alarm on interface.

did anybody configured like this,

[Jason Meader]

Hi,

All you need to do is set AlarmOnLinkDownTrap to Check Status on the Interface models that you want to alarm on when a link down trap is received.  When the trap is received Spectrum will check the status of the port and alarm if it’s ifadmin up and ifoper down.

Cheers

Jay

 

Further info:

https://comm.support.ca.com/kb/portinterface-alarming-in-ca-spectrum-when-a-portinterface-model-goes-down/kb000039093