How to create a new metric - related to RESTmon probe

[Glenn Weavind]

Hi: I'm studying the BC examples for the RESTmon probe.  Some of them appear to create new metrics (e.g. Tandberg MXP Video Out jitter) which makes sense.  However, I can't see where Metric Type is derived from, or where I can find a list/schema of all presently defined metric types.
Is there a documented list somewhere?  Is the JSON that's loaded as part of the RESTmon probe configuration enough to create a completely new metric, please?
Thanks.

[David Michel]

A metric type ID  represents the kind of measurement being collected. This ID is a single integer but is typically referenced using its fully-qualified path including the CI type, where it is the number after the colon (for example, the full path of 1.1:12 contains the CI type ID of 1.1 plus the metric type ID of 12, where the combination defines a unique kind of metric that can be collected: "System"."Disk":"Disk Free"). The metric type ID includes the metric unit, for example: 1.1:12 = "System"."Disk"."Disk Free" reported in GB and 1.1:13 = "System"."Disk"."Disk Free" reported in MB. The metric type IDs are defined in the CM_CONFIGURATION_ITEM_METRIC_DEFINITION table and units are defined in the CM_METRIC_UNIT table.

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/ca-unified-infrastructure-management-probes/GA/probe-development-tools/probe-software-developer-kit/probe-sdk-cookbook/declaring-inventory-metrics-and-bulk-configuration.html


------------------------------
Support Engineer
Broadcom
------------------------------

[Glenn Weavind]

Thanks David.
My assumption/guess that the Tandberg_MXP schema example was creating a new metric is incorrect, it seems?  All of the metric types used in the example schema already exist.
New Q: is there in fact a way for the user to create new metrics in UIM, please?  At the moment I probably don't need to do this...

[Chris Klug]

Hi,

You can find the available metric types in Operator Console -> Settings -> Restmon -> Supported_CI_Metric_Types.xlsx.  After downloading, this spreadsheet contains the metrics types as well as the metric units that are available.  

You had mentioned the Tandberg_MXP schema, those metric types are actually listed in this spreadsheet so they were pre-defined.  I'd suggest referencing the ecs_schema.json as well, it's more straightforward in my opinion.

To create new metric types, you can open a support case and ask that they be added.  One thing to note:  the metric types will more than likely only be added if the metric is generic and would be applicable to other customers.

You can also use 9.1.1:x metric type in your schema if needed.  This would allow you to create a new qos that doesn't fall within the provided metric types.  So let's say you have 3 metrics in your new schema.  You'd use 9.1.1:1, 9.1.1:2, and 9.1.1:3 for the metric_type.  Each qos needs it's own qos name and metric type.

Your first qos would something like this:

"uim": {
"defaultpublishing": "true",
"qos_name": "QOS_MY_NEW_METRIC1",
"qos_desc": "Total Free",
"qos_abbr": "PB",
"metric_type": "9.1.1:1",


Hope that helps,
Chris

[Glenn Weavind]

Thanks Chris.  I've got my first schema past all of the checking.  The next part of the puzzle is handling timestamps.  The data source that I want to poll shows a UNIX-style timestamp for the event/information/alarm.  It's certain that we shall end up polling the same information more than once (more than one monitoring tool is being fed from this source), so how do I prevent UIM from creating multiple data points from the same datum?

Thanks.

[Chris Klug]

At this point in time, Restmon only gathers metric/qos, it can't pull alarms or timestamps.

[Glenn Weavind]

Thanks Chris.  So every metric collected via RESTmon gets the timestamp in UIM of the time that we (UIM) polled for it?
I should anticipate duplicate metrics if I poll too quickly?

[Chris Klug]

Correct, keep in mind the shorter the polling interval, the more data you are collecting and storing in the database and that raw data will be retained until the data engine retention settings kick in.  Start with something like 5 minutes for example and see what it gives you.  Also, watch the restmon.log at log level 3 or 5 and you can see the information being printed to the log.  The log will have messages about run.  Each polling cycle is a run in the log and if you pull too much data in too short of a frequency, then you'll see it in the logs that the previous run didn't finish before the next polling cycle/run started.  We see that sometimes when pulling hundreds of metrics related to clusters and node type level metrics.

[Glenn Weavind]

Thanks Chris.  Perhaps a better question for me to ask would be this: which UIM Probe could give me the ability to collect metrics and alarms from a third-party web-based resource, using the timestamp on the resource to detect previously collected data?

I'm keen to avoid writing my own probe if I can as there's a lot to learn for that challenge and quite a lot of core UIM that I'm rusty on.

Thanks.

[Chris Klug]

Hi Glenn,

Unless I'm totally missing it, we don't have a probe for what you're looking for at this time.  I've been told that Restmon will be enhanced in the near future but I don't have details on what the enhancements will be but hopefully it will be able to bring in alarms.

Chris

[Glenn Weavind]

Thanks Chris.  At least I know I'm not missing something obvious.
Glenn.

[Glenn Weavind]

Hi Chris:
I see from other queries about RESTmon probe that Service Account usage is expected (e.g. to avoid the need for MFA/2FA).
Question: does RESTmon support Service Account authentication where our outgoing poll presents a userid and Key (of a key/pair), please?  How should this be presented in the configuring JSON of the schema to be processed?

Thanks.

[Chris Klug]

Glenn, need a bit more detail as to what you're trying to connect to.  Is this for GCP as is being discussed in the other thread?

[Glenn Weavind]

Hi Chris: Yes, for GCP.  Seems the most convenient way to authenticate to GCP APIs is via a Service Account, and login for that uses an ID and key offer (from a previously established key-pair), if I understand correctly.  This is a common mechanism for REST calls - Postman supports this - so I'm hoping that the current UIM RESTmon probe also supports it.
If it does - and in another thread Rich Lankester suggests that it's the most practical way to avoid multifactor authentication challenges - then what's the syntax, please?
We can agree that the documentation for RESTmon probe is thin.
Thanks.

[Chris Klug]

Hi Glenn,

Attached is a sample schema for GCP that was used a couple years ago for a quick test to see if it would work, hopefully it still works. I didn't write it and I don't have access to GCP. 

Note that auth is none as restmon has code built in for Oauth2 and you'll need that in the Authority field.  For the scope section, it's my understanding that the GCP Service Account gives you the Client ID, Client Secret, and Project ID.  You'll need to check the Scope field to make sure those are still the api's that allow you to get to the metrics and give access accordingly.

The schema contains one metric that you can use to test and as a template for adding additional metrics.

Hope this is helpful and works but no guarantee!  Change the filename to .json after download.

Chris

[Chris Klug]

Let's see if this attaches..

Attachment(s)

gcp_uim_schema_sample.txt   2 KB 1 version

[Glenn Weavind]

Hi Chris: Many thanks for this.  I realise now that I've started to muddle up OAuth2.0 and Service Account key presentation.

If I understand this correctly (and I have a working testbed with POSTman), the OAuth2.0 setup requires generation of a limited duration session token, and it is this that's proffered for each access.  Once this token expires it has to be renewed (repeat of the token generation process) and so on.  Trouble is, my reading of the GCP docs is that these tokens can only be renewed 50 times.  Not exactly a long-term solution, although one might be able to make each token last for a very long time, but still not open-ended.

My immediate question is this: do you know if the current UIM RESTmon probe has the built-in ability to generate this access token, and cope with regeneration?  Or does UIM expect us to generate the OAuth2.0 access token manually?

If you don't know this - and this is really detailed code level stuff - are you able to suggest how I might be able to get an answer, please?