Correlate Events and generate Single Alarm - CA Spectrum

[Rajashekar Allala]

Hi All,

We have rubrik cluster forwarding traps from multiple appliances to Spectrum for the same issue and getting multiple alarms.

Example: Backup Failed for one of the SQL DB, rubrik manages the status of the same from multiple nodes in the rubrik cluster and generates the traps to Spectrum.

Is it possible to correlate all of the events generated from different sources in the cluster and generate only one alarm? Please suggest.

------------------------------
Thank you.
Rajashekar
------------------------------

[Joseph Ackley]

You should be able to use Condition Correlation to correlate to a single alarm.

That alarm would be the root cause alarm and the other would be symptom alarms.

Then use the Alarms tab to filter and display only root cause alarms.

Joe

[Rajashekar Allala]

Thank you Joe. I observed that, the trap/event/alarm generation from two different sources is having a delay of min 1 hour as they check the status in different intervals. In that case, will the condition correlation help?

------------------------------
Thank you.
Rajashekar
------------------------------

[Joseph Ackley]

That much time between alarms will probably not work for you with the Condition Correlation.

Is there any way to sync the checking of the status?

Joe

[Rajashekar Allala]

Also, is it possible to make a single event as a symptom and cause as we have only one event that generates the alarm with no clear event. When I enabled condition correlation, even if there is a single alarm, it is hiding with the below event message.

Please provide some insight.

------------------------------
Thank you.
Rajashekar
------------------------------

[Rajashekar Allala]

Hi Joseph Ackley, any help here please.

------------------------------
Thank you.
Rajashekar
------------------------------

[Rajashekar Allala]

@Joseph Ackley​ please clarify the below.

Is it possible to make a single event as a symptom and cause as we have only one event that generates the alarm with no clear event? When I enabled condition correlation on this event as symptom and cause, even if there is a single alarm, it is hiding the alarm with the below event message.

Please provide some insight.

------------------------------
Thank you.
Rajashekar
------------------------------

[Rajashekar Allala]

Hi All,

Please help me on this. Correcting the logic the Rubrik follow.

We have different backup jobs scheduled in Rubrik. It manages the job queue and each job is picked up by one of the available nodes in the cluster. If the job fails, it is put back to queue and picked up by another node to process and continues if fails again.

For each failure at respective node, we get a trap. That means, if the job is picked up by 3 nodes and failed to complete, we get 3 traps and in turn 3 alarms.

We have one event mapped to the rubrik trap to generate the alarm. As the trap is coming from multiple nodes for same issue at different intervals when the respective job runs, we are getting the same alarm on all the nodes at respective trap generation intervals. Is it possible to correlate them and generate the alarm on correlation domain or one of the nodes. Also, please clarify me on the below.

Is it possible to make a single event as a symptom and cause as we have only one event that generates the alarm with no clear event? When I enabled condition correlation on this event as symptom and cause, even if there is a single alarm, it is hiding the alarm with the below event message.

Please provide some insight.

@Joseph Ackley @Matthew_Gay @Jason Meader @Michael Poller @Marcelo Zacchi @Catalin Farcasanu @Stephen_Warne

​​​​​​​​

------------------------------
Thank you.
Rajashekar
------------------------------

[Stephen_Warne]

Hi Rajashekar
the way I would deal with this is to create a spectrum service manager service model.
create resource groups for the backup nodes. One to monitor the contact status of the nodes (good for root cause analysis) and another to monitor their condition but ONLY for the events created by the failure traps.
You then define the policy/rules to determine the state of the service.eg if all nodes are contactable but one or two fail to run the backup but the third one works ok, then the service is degraded. If all three nodes fail to back up then its a service down situation.
hope that helps.
Regards
Stephen.

[Rajashekar Allala]

Thank you All for your help.

I am able to achieve the required solution with the below simple code. I have created an Event Model to represent the Rubrik Cluster and whenever the Spectrum receives a trap from any one of the cluster nodes, alarm will be generating on EventModel with the model handle 0x4a3a72 and I am using the event discriminator as varbind 200 (this is the unique value in the trap value). So, all traps will be generating the alarm event on Event Model and generates unique alarm based on discriminator.

Event Procedure

0xfff0096c P "CreateEventWithVariables({H 0x4a3a72}, {H 0xfff0096d}, SetEventVariable(GetEventVariableList(), {U 200}, GetRegexp(GetEventAttribute({U 100}), {S \"(?<=objectName\=)(.*)(?= eventId)\"}, {U 0})))"

------------------------------
Thank you.
Rajashekar
------------------------------