How can I create a watch with the "fgHaStatsSerial" attribute?

[Hernan Bertel]

Hi,

 

I have CA Spectrum V 10.2, i have a situation when a fortigate device make a failover, the device have the same IP adress and model name the difference is in the serial number.

I was find a OID (1.3.6.1.4.1.12356.101.13.2.1.1.2) who evaluates the change of serial number but i try to create a Watch in Spectrum with this Atribute "fgHaStatsSerial", text string type but it doesn't work.

Can someone help me find some way to create the whatch ?.

 

Thanks.

 

#spectrumversion10.2 #watches #situationstowatch

[Christophe_Sperandio]

Well, before creating a Watch, you need to have the attribute reliying on that OID created on your Model Type. Is that the case already? When you look in the "Attributes" tab of a device modeled with the Model Type you want to create a Watch for, do you see the attribute "fgHaStatsSerial" in the list?

I guess it's not the case. So, basically, you need to import/map this attribute from the MIB onto a Model Type to be able to collect it and use it for a watch. I highly recommend not doing this through the "MIB Tools" for several reasons, and would rather prefer the use of the MTE (Model Type Editor) to do such a thing.

When you have a Model Type with the appropriate attribute, you're good to go.

Some docs which could help: Model Type Editor - CA Spectrum - 10.2 to 10.2.2 - CA Technologies Documentation 

[Hernan Bertel]

The process of import the OID from MIB tools to the attribute on the device has already been done and it looks correctly with its values. The problem is presented specifically when creating the watch because the attribute is a string type but I can not find how to create a rule for the treshold.

Now ... I will keep in mind the MTE, however, I would like to find a way to create the watch with the current procedure, that is, through the MIB Tools.

[Christophe_Sperandio]

Ok, I see. Well, what are you exactly trying to achieve?

 

You want to get an event generated (and potentially an alarm) anytime this attribute changes? If yes, I suggest you create a watch of type "boolean" for example, and you set the watch expression to be "fgHaStatsSerial == "whatever". You set the rule to be "evaluate on change". The "threshold" could be, when FALSE, generate an event (it will always be false, so will generate an event every time it's evaluated, but will be evaluated only when it changes... So basically does what you want if I'm not wrong in my start hypothesis).

 

I'm absolutely not sure it's going to work though, because when I look into the MIB, the value fgHaStatsSerial comes from a table. Which means if you have two member in your cluster, you'll have two lines in the table and both serials from both members. I don't see the point doing that.

 

If I may suggest another thing: this kind of event should rather be caught by a trap handling. If you still want to do it by polling, I think a better way would be by comparing fgHaStatsMasterSerial with fgHaStatsSerial (which should tell you if this is the master or not).

 

I think to finish, the best way to monitor the HA state of Fortinet devices is through the fgHaStatsSyncStatus OID (1.3.6.1.4.1.12356.101.13.2.1.1.12). From the MIB, it should give you the status of each of the member of the cluster, and then, you should be able to know which one is master vs slaves.

 

Hope that helps!

[Hernan Bertel]

Validating the new MIB that you indicate to me I think it is not possible, since it is a single fortigate device that subdivides into 2 and forms the cluster but it has the same IP address and name, the only difference between them is the serial .

The test was performed with the boolean type watch and the expression was as follows:
((fgHaStatsSerial. # == "Serial1") | (fgHaStatsSerial. # == "Serial2")) having as treshold violated == FALSE since the condition implies one of the two.

 

At the moment it is difficult to make real tests of failover and it is necessary to wait for a real event, do you think that this could work?

 

Thank you in advance for your help.

[Christophe_Sperandio]

Well, the watch condition seems correct to me. I'm still struggling with the expected outcome: the condition you state will always return TRUE, am I wrong? Because you will always have one or the other member of the cluster being the master, won't you?

What is your goal, what are you trying to get? You want to get an alarm when you lose one or the other member of the cluster, don't you ?

[Hernan Bertel]

In effect, there will always be an active member (master) and a passive member (slave) and the idea is to get the alarm when the master asset unexpectedly goes into a slave.

[Christophe_Sperandio]

Ok, that confirms my guess. Would you mind sharing here an snmp walk dump of such a device, especially the whole node fgHighAvailability (1.3.6.1.4.1.12356.101.13) if there's nothing confidential in it? That would help...

[Hernan Bertel]

Cristophe I have a snmp walk of device, I asked my boss and we can not share that result in public. Can you tellme whats especific need of that ?
The OID 1.3.6.1.4.1.12356.101.13 is in this snmp walk.

[Christophe_Sperandio]

That was to see the exact type of data and the structure of all tables. If that cannot be shared, and you are stuck, you might have to open a case to have a support engineer assigned to assist?

[Hernan Bertel]

I just attached the snmp walk "Fortigate.txt" but I had to delete all the ip addresses and names, I do not know if I can serve you that way.

I hope your understanding.

[Christophe_Sperandio]

Hello,

 

Of course I understand! No worries. Sorry for the delay by the way. From my quick analysis on your dump, here's what I suggest then:

• You should rely on the comparison of the fgHaStatsMasterSerial (1.3.6.1.4.1.12356.101.13.2.1.1.16.x) and fgHaStatsSerial (1.3.6.1.4.1.12356.101.13.2.1.1.2.x) attributes.
• Those attributes are tables, which means, you have one instance per cluster member (the 'x' in my previous point).
• The fgHaStatsMasterSerial will (as far as I can see) always be the Serial Number of the Master member of the cluster for each instance.

 

If your goal is just to be warned when there's a failover, you should only use the following watch definition:

• Data Type: TEXT STRING
• Expression: fgHaStatsMasterSerial.#
• Instance: 1
• Evaluated: On Change
• Rule: Generate event or alarm (title of your choice)

This first proposition should generate an event/alarm any time the cluster switches (whenever the master role is given to another member of the cluster).

 

If you want to have always the first member of the cluster as the master in a nominal situation, you can configure a watch as follows:

• Data Type: TEXT STRING
• Expression: fgHaStatsMasterSerial.# == fgHaStatsSerial.#
• Instance: 1
• Evaluated: On Poll
• Frequency: 300 seconds (for example)
• Threshold
• Rule: Generate event or alarm (title of your choice)

That second proposition will generate an alarm in case the master is not the first node, and will therefore tell you the cluster failed over.

 

Does that make sense? Does that help?

[Hernan Bertel]

 

Thank you very much for everything, I clarified amount of doubts with your contributions.
Since it is not possible to perform a failover test, I have decided to leave active 2 watches.

 

I would have only one last question, when setting up the watch as you indicate:

 

Data Type: TEXT STRING
Expression: fgHaStatsMasterSerial. #
Instance: 1
Evaluated: On Change
Rule: Generate event or alarm (title of your choice)

 

I understand that you are evaluating the change, but is it necessary to activate a treshold, so that the alarm is generated?

 

 

[Christophe_Sperandio]

Happy to know our conversation has been useful and helped you!

Yes, you need to enable a threshold to be able to select anything in the "Notification" part. The trick here is to make something which is always triggered like saying it's violated if value == "whateveryouwant". Don't forget to "Reset watch upon user clearing alarm" and you should be good to go.

Be aware though that when you will first activate the watch, that might trigger an initial alarm you'll have to clear before the watch to do what you expect.

[Hernan Bertel]

Thank you very much, I'm configured that way.

Greetings, have a good day.