traps are not attaching to SBG event model or the spectrum model for which they were created

[Rohit Kumar]

Hi All,
We have used SBG to forward the trap from third party system such Oracle enterprise manager (OEM), Splunk etc to spectrum. but traps are not attaching to SBG event model or the spectrum model for which they were created, instead they are attaching to management server model (OEM, Splunk servers discovered in spectrum) in spectrum. what could be the reason for the same?

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

SBG uses 3 options to map the trap to an existing model or a new EventModel:

1. Combination of variables 1-6 as the EventModel name (if it does not exist it is created)
2. Variable 7 as the target model name
3. Variable 8 as the target model ip address

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/10-4-3/integrating/southbound-gateway-toolkit/the-event-data-template.html

When using IP address to map to an existing model, have in mind there are 3 IP address in a trap:

1. The IP address the trap is sent from (this is part of the IP header. ex: 1.1.1.1)
2. The IP address of the agent sending the trap (usually this one matches the first one, but let's consider different in our example: 2.2.2.2)
3. Any IP address included in the trap varbinds (ex: 3.3.3.3)

The second IP (2.2.2.2) is used to map the trap to EventAdmin model. That IP address must match the IP address assigned to EventAdmin at creation

Then, when you map your varbinds in the AlertMap file, if your third IP (3.3.3.3) is mapped to variable 8, SBG will assert the resulting event to the model with that IP address.

Maybe your AlertMap file and one trap example will help to understand the issue

Regards

[Rohit Kumar]

Hi Jose,
Thanks for response!
only target hostname is coming in the trap which i tried to map it to variable ID 7 or 16 as noted in that tech doc. but it did not work.
https://knowledge.broadcom.com/external/article?articleId=44103


Below is content of alert map file. and sample screenshot. in current mapping hostname is showing variable ID 24

#oraEMNGEvent oraEMNGEventNotifType

1.3.6.1.4.1.111.15.2.0.3 0xfff00788 1.3.6.1.4.1.111.15.3.1.1.2(1,2)\
#oraEMNGEventMessage
1.3.6.1.4.1.111.15.3.1.1.3(3,0)\
#oraEMNGEventMessageURL
1.3.6.1.4.1.111.15.3.1.1.4(4,0)\
#oraEMNGEventSeverity
1.3.6.1.4.1.111.15.3.1.1.5(5,0)\
#oraEMNGEventSeverityCode
1.3.6.1.4.1.111.15.3.1.1.6(6,0)\
#oraEMNGEventRepeatCount
1.3.6.1.4.1.111.15.3.1.1.7(7,0)\
#oraEMNGEventActionMsg
1.3.6.1.4.1.111.15.3.1.1.8(8,0)\
#oraEMNGEventOccurrenceTime
1.3.6.1.4.1.111.15.3.1.1.9(9,0)\
#oraEMNGEventReportedTime
1.3.6.1.4.1.111.15.3.1.1.10(10,0)\
#oraEMNGEventCategories
1.3.6.1.4.1.111.15.3.1.1.11(11,0)\
#oraEMNGEventCategoryCodes
1.3.6.1.4.1.111.15.3.1.1.12(12,0)\
#oraEMNGEventType
1.3.6.1.4.1.111.15.3.1.1.13(13,0)\
#oraEMNGEventName
1.3.6.1.4.1.111.15.3.1.1.14(14,0)\
#oraEMNGAssocIncidentId
1.3.6.1.4.1.111.15.3.1.1.15(15,0)\
#oraEMNGAssocIncidentOwner
1.3.6.1.4.1.111.15.3.1.1.16(16,0)\
#oraEMNGAssocIncidentAcked
1.3.6.1.4.1.111.15.3.1.1.17(17,0)\
#oraEMNGAssocIncidentStatus
1.3.6.1.4.1.111.15.3.1.1.18(18,0)\
#oraEMNGAssocIncidentPriority
1.3.6.1.4.1.111.15.3.1.1.19(19,0)\
#oraEMNGAssocIncidentEscLevel
1.3.6.1.4.1.111.15.3.1.1.20(20,0)\
#oraEMNGEventTargetName
1.3.6.1.4.1.111.15.3.1.1.21(21,0)\
#oraEMNGEventTargetNameURL
1.3.6.1.4.1.111.15.3.1.1.22(22,0)\
#oraEMNGEventTargetType
1.3.6.1.4.1.111.15.3.1.1.23(23,0)\
#oraEMNGEventHostName
1.3.6.1.4.1.111.15.3.1.1.24(24,0)\
#oraEMNGEventTargetOwner
1.3.6.1.4.1.111.15.3.1.1.25(25,0)\
#oraEMNGEventTgtLifeCycleStatus
1.3.6.1.4.1.111.15.3.1.1.26(26,0)\
#oraEMNGEventTargetVersion
1.3.6.1.4.1.111.15.3.1.1.27(27,0)\
#oraEMNGEventUserDefinedTgtProp
1.3.6.1.4.1.111.15.3.1.1.28(28,0)\

[Jose Vicente Espinosa]

Hi Rohit,

Variables from 1 to 16 are reserved for specific use and 17-9 for future usage of Broadcom.

I guess the problem is you mapped:

• oraEMNGEventActionMsg to variable 8 used as Target Address
• oraEMNGEventRepeatCount used as Target Name
• oraEMNGAssocIncidentOwner to variable 16 user as Target Name (case insensitive)

If you specify a Target Name using fields 7 or 16 and a Target Address and one or both is not found, the appropriate error message is generated, and the event is not sent to any model

The recommendation is that you map your varbinds starting on variable 100 and only map oraEMNGEventHostName to variable 16 to set the Target Model by Name (case insensitive)

This would be my proposal for AlertMap:

#oraEMNGEvent #oraEMNGEventNotifType
1.3.6.1.4.1.111.15.2.0.3 0xfff00788 1.3.6.1.4.1.111.15.3.1.1.2(100,0)\
#oraEMNGEventMessage
1.3.6.1.4.1.111.15.3.1.1.3(101,0)\
#oraEMNGEventMessageURL
1.3.6.1.4.1.111.15.3.1.1.4(102,0)\
#oraEMNGEventSeverity
1.3.6.1.4.1.111.15.3.1.1.5(103,0)\
#oraEMNGEventSeverityCode
1.3.6.1.4.1.111.15.3.1.1.6(104,0)\
#oraEMNGEventRepeatCount
1.3.6.1.4.1.111.15.3.1.1.7(105,0)\
#oraEMNGEventActionMsg
1.3.6.1.4.1.111.15.3.1.1.8(106,0)\
#oraEMNGEventOccurrenceTime
1.3.6.1.4.1.111.15.3.1.1.9(107,0)\
#oraEMNGEventReportedTime
1.3.6.1.4.1.111.15.3.1.1.10(108,0)\
#oraEMNGEventCategories
1.3.6.1.4.1.111.15.3.1.1.11(109,0)\
#oraEMNGEventCategoryCodes
1.3.6.1.4.1.111.15.3.1.1.12(110,0)\
#oraEMNGEventType
1.3.6.1.4.1.111.15.3.1.1.13(111,0)\
#oraEMNGEventName
1.3.6.1.4.1.111.15.3.1.1.14(112,0)\
#oraEMNGAssocIncidentId
1.3.6.1.4.1.111.15.3.1.1.15(113,0)\
#oraEMNGAssocIncidentOwner
1.3.6.1.4.1.111.15.3.1.1.16(114,0)\
#oraEMNGAssocIncidentAcked
1.3.6.1.4.1.111.15.3.1.1.17(115,0)\
#oraEMNGAssocIncidentStatus
1.3.6.1.4.1.111.15.3.1.1.18(116,0)\
#oraEMNGAssocIncidentPriority
1.3.6.1.4.1.111.15.3.1.1.19(117,0)\
#oraEMNGAssocIncidentEscLevel
1.3.6.1.4.1.111.15.3.1.1.20(118,0)\
#oraEMNGEventTargetName
1.3.6.1.4.1.111.15.3.1.1.21(119,0)\
#oraEMNGEventTargetNameURL
1.3.6.1.4.1.111.15.3.1.1.22(120,0)\
#oraEMNGEventTargetType
1.3.6.1.4.1.111.15.3.1.1.23(121,0)\
#oraEMNGEventHostName
1.3.6.1.4.1.111.15.3.1.1.24(16,0)\
#oraEMNGEventTargetOwner
1.3.6.1.4.1.111.15.3.1.1.25(123,0)\
#oraEMNGEventTgtLifeCycleStatus
1.3.6.1.4.1.111.15.3.1.1.26(124,0)\
#oraEMNGEventTargetVersion
1.3.6.1.4.1.111.15.3.1.1.27(125,0)\
#oraEMNGEventUserDefinedTgtProp
1.3.6.1.4.1.111.15.3.1.1.28(126,0)\

Be sure a model with name oraEMNGEventHostName exists in Spectrum.

If the Target Name is not found, the EventAdmin model generates an Event that indicates that Spectrum has no models with the name <Target Name>

Please update event 0xfff00788 with new variables numbers.

Regards

[Rohit Kumar]

Hi Jose,
Thanks for the response!
I tried that too, but alarms are created with blanks. Could you please let me know if this is because of the location of AlertMAP and EventDisp'Files? Which is Set to below location in my configuration?

$Specroot/custom/Events/AlertMap

$Specroot/custom/Events/EventDisp

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

I think you are right

Files locations are:

• In SpectroServer

$SPECROOT/custom/Events/gen_app_gw/EventAdmin/AlertMap
$SPECROOT/custom/Events/gen_app_gw/EventAdmin/EventDisp

• In OneClick

$SPECROOT/custom/Events/CsEvFormat/Event<eventcode>_en_US

Regards

[Rohit Kumar]

Hi Jose,
thanks for the information shared.
I am wondering how it is working if the path is not correct?
the strange thing is everything is working fine except, alerts are not associating on the target models.

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

Can you share your AlertMap and a screenshot of the alarm/event resulting?

Regards

[Rohit Kumar]

Hi Jose,
PFA sample alerts and the content of Alert Map file for OEM alert.
alert MapFile Content.
# oraEMNGEvent oraEMNGEventNotifType
1.3.6.1.4.1.111.15.2.0.3 0xfff00788 1.3.6.1.4.1.111.15.3.1.1.2(1,2)\
#oraEMNGEventMessage
1.3.6.1.4.1.111.15.3.1.1.3(3,0)\
#oraEMNGEventMessageURL
1.3.6.1.4.1.111.15.3.1.1.4(4,0)\
#oraEMNGEventSeverity
1.3.6.1.4.1.111.15.3.1.1.5(5,0)\
#oraEMNGEventSeverityCode
1.3.6.1.4.1.111.15.3.1.1.6(6,0)\
#oraEMNGEventRepeatCount
1.3.6.1.4.1.111.15.3.1.1.7(7,0)\
#oraEMNGEventActionMsg
1.3.6.1.4.1.111.15.3.1.1.8(8,0)\
#oraEMNGEventOccurrenceTime
1.3.6.1.4.1.111.15.3.1.1.9(9,0)\
#oraEMNGEventReportedTime
1.3.6.1.4.1.111.15.3.1.1.10(10,0)\
#oraEMNGEventCategories
1.3.6.1.4.1.111.15.3.1.1.11(11,0)\
#oraEMNGEventCategoryCodes
1.3.6.1.4.1.111.15.3.1.1.12(12,0)\
#oraEMNGEventType
1.3.6.1.4.1.111.15.3.1.1.13(13,0)\
#oraEMNGEventName
1.3.6.1.4.1.111.15.3.1.1.14(14,0)\
#oraEMNGAssocIncidentId
1.3.6.1.4.1.111.15.3.1.1.15(15,0)\
#oraEMNGAssocIncidentOwner
1.3.6.1.4.1.111.15.3.1.1.16(16,0)\
#oraEMNGAssocIncidentAcked
1.3.6.1.4.1.111.15.3.1.1.17(17,0)\
#oraEMNGAssocIncidentStatus
1.3.6.1.4.1.111.15.3.1.1.18(18,0)\
#oraEMNGAssocIncidentPriority
1.3.6.1.4.1.111.15.3.1.1.19(19,0)\
#oraEMNGAssocIncidentEscLevel
1.3.6.1.4.1.111.15.3.1.1.20(20,0)\
#oraEMNGEventTargetName
1.3.6.1.4.1.111.15.3.1.1.21(21,0)\
#oraEMNGEventTargetNameURL
1.3.6.1.4.1.111.15.3.1.1.22(22,0)\
#oraEMNGEventTargetType
1.3.6.1.4.1.111.15.3.1.1.23(23,0)\
#oraEMNGEventHostName
1.3.6.1.4.1.111.15.3.1.1.24(24,0)\
#oraEMNGEventTargetOwner
1.3.6.1.4.1.111.15.3.1.1.25(25,0)\
#oraEMNGEventTgtLifeCycleStatus
1.3.6.1.4.1.111.15.3.1.1.26(26,0)\
#oraEMNGEventTargetVersion
1.3.6.1.4.1.111.15.3.1.1.27(27,0)\
#oraEMNGEventUserDefinedTgtProp
1.3.6.1.4.1.111.15.3.1.1.28(28,0)\
#oraEMNGEventSourceObjName
1.3.6.1.4.1.111.15.3.1.1.29(29,0)\
#oraEMNGEventSourceObjNameURL
1.3.6.1.4.1.111.15.3.1.1.30(30,0)\
#oraEMNGEventSourceObjType
1.3.6.1.4.1.111.15.3.1.1.31(31,0)\
#oraEMNGEventSourceObjSubType
1.3.6.1.4.1.111.15.3.1.1.32(32,0)\
#oraEMNGEventSourceObjOwner
1.3.6.1.4.1.111.15.3.1.1.33(33,0)\
#oraEMNGEventCAJobName
1.3.6.1.4.1.111.15.3.1.1.34(34,0)\
#oraEMNGEventCAJobStatus
1.3.6.1.4.1.111.15.3.1.1.35(35,0)\
#oraEMNGEventCAJobOwner
1.3.6.1.4.1.111.15.3.1.1.36(36,0)\
#oraEMNGEventCAJobStepOutput
1.3.6.1.4.1.111.15.3.1.1.37(37,0)\
#oraEMNGEventCAJobType
1.3.6.1.4.1.111.15.3.1.1.38(38,0)\
#oraEMNGEventRuleSetName
1.3.6.1.4.1.111.15.3.1.1.39(39,0)\
#oraEMNGEventRuleName
1.3.6.1.4.1.111.15.3.1.1.40(40,0)\
#oraEMNGEventRuleOwner
1.3.6.1.4.1.111.15.3.1.1.41(41,0)\
#oraEMNGEventSequenceId
1.3.6.1.4.1.111.15.3.1.1.42(42,0)\
#oraEMNGEventRCADetails
1.3.6.1.4.1.111.15.3.1.1.43(43,0)\
#oraEMNGEventContextAttrs
1.3.6.1.4.1.111.15.3.1.1.44(44,0)\
#oraEMNGEventUserComments
1.3.6.1.4.1.111.15.3.1.1.45(45,0)\
#oraEMNGEventUpdates
1.3.6.1.4.1.111.15.3.1.1.46(46,0)\
#oraEMNGEventTotalOccurrenceCount
1.3.6.1.4.1.111.15.3.1.1.47(47,0)\
#oraEMNGEventCurrOccurrenceCount
1.3.6.1.4.1.111.15.3.1.1.48(48,0)\
#oraEMNGEventCurrFirstOccurDate
1.3.6.1.4.1.111.15.3.1.1.49(49,0)\
#oraEMNGEventCurrLastOccurDate
1.3.6.1.4.1.111.15.3.1.1.50(50,0)\
#oraEMNGEventRCAStatus
1.3.6.1.4.1.111.15.3.1.1.51(51,0)\
#oraEMNGEventReportedState
1.3.6.1.4.1.111.15.3.1.1.52(52,0)\
#oraEMNGEventTypeAttr1
1.3.6.1.4.1.111.15.3.1.1.61(61,0)\
#oraEMNGEventTypeAttr2
1.3.6.1.4.1.111.15.3.1.1.62(62,0)\
#oraEMNGEventTypeAttr3
1.3.6.1.4.1.111.15.3.1.1.63(63,0)\
#oraEMNGEventTypeAttr4
1.3.6.1.4.1.111.15.3.1.1.64(64,0)\
#oraEMNGEventTypeAttr5
1.3.6.1.4.1.111.15.3.1.1.65(65,0)\
#oraEMNGEventTypeAttr6
1.3.6.1.4.1.111.15.3.1.1.66(66,0)\
#oraEMNGEventTypeAttr7
1.3.6.1.4.1.111.15.3.1.1.67(67,0)\
#oraEMNGEventTypeAttr8
1.3.6.1.4.1.111.15.3.1.1.68(68,0)\
#oraEMNGEventTypeAttr9
1.3.6.1.4.1.111.15.3.1.1.69(69,0)\
#oraEMNGEventTypeAttr10
1.3.6.1.4.1.111.15.3.1.1.70(70,0)\
#oraEMNGEventTypeAttr11
1.3.6.1.4.1.111.15.3.1.1.71(71,0)\
#oraEMNGEventTypeAttr12
1.3.6.1.4.1.111.15.3.1.1.72(72,0)

[Jose Vicente Espinosa]

Hi Rohit,

here the files I have in the custom folder and files under Spectrum SpectroServer

custom/Events/gen_app_gw/EventAdmin/AlertMap - we map the trap to event 0xfff00788 and specifically map varbind oraEMNGEventHostName to variable 16. This is a special variable with the objective to assert the event into a model whose name matched (case insensitive) its value
custom/Events/EventDisp - we indicate our event 0xfff00788 will become an alert with severity Critical (3). You can apply conditional based on oraEMNGEventSeverity
custom/Events/gen_app_gw/EventAdmin/EventDisp - We indicate our event 0xfff00788 does not register alarm in the EventAdmin

If you copy these files or their content to your Spectrum, remember to update Event Configuration under VNM model

With this configuration, if we send a trap whose varbind oraEMNGEventHostName is set to ce1.mydomain.com, SBGW will lookup for a model with that name

Note: when the model is not found, there is no alarm but an event in the Event Admin.

I hope this helps
Regards

Attachment(s)

oraEMNGEvent_sbgw.zip   2 KB 1 version

[Rohit Kumar]

Hi Jose,
Thanks a lot for the information shared.
but in my case  I already have a Linux systemedge agent for that same host the traps are coming from, and siting on it.
considering my case do i need to create EventAdmin?
though i will try the steps you have shared.

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

this is the reason why the alerts are not mapped to target models.

When using Host compatible models instead of Event Admin you need to follow this simple step

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/10-4-3/integrating/southbound-gateway-toolkit/use-a-host-model-type-instead-of-the-eventadmin-model-type.html

Regards

[Rohit Kumar]

Hi Jose,

Thanks for the information shared. I will try this step as well.

However, in my configuration the the location of alertMap file and EventDisp file was custom/Events/ AlertMap and custom/Events/EventDisp

So, what I did, commented out the existing alert mapping for OEM "custom/Events/ AlertMap, and copy Alertmap content of OEM mapping to custom/Events/gen_app_gw/EventAdmin/AlertMap. After that reloaded events from VNM>spectroserver control>update event configuration.

Now, new events are coming as unknown without mapping. What could be reason for the same?

I have enclosed the screenshot for the same.

If go and put OEM alert mappings to spectrum/SS/CsVendor/gen_app_gw/EventAdmin/AlertMap, then also I am getting unknow event.

 

Regards

Rohit.

[Jose Vicente Espinosa]

Hi Rohit,

Did you copy the EventDisp files?

Did you remove the Host_SystemEdge model with the IP where the traps are received from?

Regards

[Rohit Kumar]

Hi Jose,
i copy Eventides' file exactly  the same way you mentioned.
custom/Events/EventDisp - we indicate our event 0xfff00788 will become an alert with severity Critical (3). You can apply conditional based on oraEMNGEventSeverity
custom/Events/gen_app_gw/EventAdmin/EventDisp - We indicate our event 0xfff00788 does not register alarm in the EventAdmin
but i did not removed Host_SystemEdge model with the IP where the traps are received from, because this is being monitored in Spectrum.
Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

I was able to make it work as you expect. You need to follow these steps:

1) For SystemEdge model acting as EventAdmin, set attribute Enable_SouthboundGateway (0x116296e) = Yes
2) For SystemEdge model acting as EventAdmin, set attribute EventModelContainerHandle (0x3dc000a) = <model handle of a container where you want to create EventModels>
3) Create folder $SPECROOT/custom/Events/Ctron_Gen_HOST/Host_systemEDGE in your SpectroServer
4) Copy AlertMap and EventDisp files (I sent to you) under $SPECROOT/custom/Events/Ctron_Gen_HOST/Host_systemEDGE in your SpectroServer
5) Remove folder $SPECROOT/custom/Events/gen_app_gw in your SpectroServer (only if files contained are the ones I sent to you)
6) Reload Event Configuration from VNM model

Regards

[Rohit Kumar]

Hi Jose thanks again!
when i put my  files below and create  an eventAdmin with the same IP from where traps are coming(which is discovered as SystemEdge host in spectrum ), it is working, and traps are attaching to correct CI/model.

recommended Steps:
custom/Events/gen_app_gw/EventAdmin/AlertMap - we map the trap to event 0xfff00788 and specifically map varbind oraEMNGEventHostName to variable 16. This is a special variable with the objective to assert the event into a model whose name matched (case insensitive) its value
custom/Events/EventDisp - we indicate our event 0xfff00788 will become an alert with severity Critical (3). You can apply conditional based on oraEMNGEventSeverity
custom/Events/gen_app_gw/EventAdmin/EventDisp - We indicate our event 0xfff00788 does not register alarm in the EventAdmin.

However same traps showing unknow on systemEdge host(from where traps are coming)


When i follow the exactly same steps as you have given in your last update, i dont see any event on syetemEDGE model(from where traps are coming), nether any alarm on any model in spectrum.

am i missing  something here? 

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

if you open the event configuration tool and filter by event 0xfff00788, what do you get?

You should only have 2 entries:

One at Global scope configured as alarm,

and one at Host_SystemEdge scope with severity None

Note: my event 0xfff0001 equals to your 0xfff00788

Regards

[Rohit Kumar]

Hi Jose,
yes, i see two entries:
One with Global scope configured as alarm,
one with Host_SystemEdge scope with severity None

but in my case traps do not process at all if i keep this configuartion. is this because of presence of  folder $SPECROOT/custom/Events/gen_app_gw?
if so, i can not remove it because this is already being used by other integeration.

Regards
Rohit

[Jose Vicente Espinosa]

Hi Rohit,

There should be something missing or mixed across all the changes. Try to review it.

Regards