Update Enriched origin -> cm_computer_system?

[Luc Christiaens]

Environment: uim 9.20 on w2016 and sql 2016
MSP with several clients, each with their own origin.
But some probes, like vmware, ibmvm and ibm-svc are monitoring devices from several clients. (and not all these vmware guests have a robot installed)
Implemented:
- we created a pre-processing LUA script to set the correct origin on alarms for these 3 probes. 
- We created also a qos_processor enrichment.rb script to set the correct origin on qos for those 3 probes.
- We defined UMP accounts for each Client, and defined that they only can see their origin.
Problem: 
- In USM we created a group for each customer based on the origin.
But: when I login with a client account, I can see all devices in Inventory BUT no vmware device in the USM group.
The devices monitored by the 3 generic probes (vmware, ibmvm and ibm-svc) are defined in cm_computer_system with the old/wrong origin of the hub even if all alarms and qos are enriched correctly.
Question: can we define a rule in discover_server.cfg where we say that for the indicated probes (vmware,..) we want the EnrichedOrigin field set into the cm_computer_system origin field?
(because, as a test, I did a manual override of some devices in the cm_computer_system table and at that moment the USM group is displayed correctly when I logon with a client account)
Did anyone encounter the same problem and found a solution?
Or did somebody has some experience with the correlation in discovery_server?
Thanks, Luc

[Stephen Danseglio]

Hi Luc,

Is there some reason that the hub/robot origin override cannot be used to control the origin names?

Steve

------------------------------
Support Engineer
Broadcom
US
------------------------------

[Luc Christiaens]

Hi Steve,
This robot with vmware/ibmvm is controlling multiple clients guests.
The client guest origin can be assigned thanks to guest naming convention that is used in the alarm and qos enrichment
Thanks, Luc

[Olaf Pape]

Hi Luc,
Hi Stephen,
I ran in exactly the same issue since many years. Centralized monitoring probes (like vmware, sql_response, net_connect, icmp etc.) monitors hundred devices of several customers. For alarm messages I can manipulate the origin by pre-processing scripts within the NAS. For qos messages I can manipulte the origin by qos_processor. But how i can do this with discovery server?

Some years ago there was a topic on the roadmap that one should be able to define the origins in the probe profiles. This would solve this isuue. However, this topic has not been present anywhere for some time.

Maybe someone of the Broadcom staff can say something about this.
Thanks

------------------------------
-------------------------------------------------------------------------------------
SInce 2006 experience with:
NimBUS | SDP | UMP | UIM | DX Infrastructure Management
API | SDK | Probe Development
Nimsoft | CA | Broadcom
-------------------------------------------------------------------------------------
Germany
------------------------------

[Luc Christiaens]

I have an issue open that is with dev. for the moment.
As a bypass (because not sure if we will receive a solution in time) I wrote a Perl script that will search all devices that are in need of enrichment in cm_computer_system and creates the needed sql update statements

[Luc Christiaens]

At this moment there is a test fix available: discovery_server_20.33T6 that will update cm_computer_system with the enriched origin
(could be included in the upcoming 20.4 CU2 base)

[Luc Christiaens]

-- removed update --

[Luc Christiaens]

I encountered a problem with clients performing qos_processor origin enrichment.
When looking deeper into the created qos metrics I remarked that there was a huge amount of duplicate qos metrics.
If YOU are running origin enrichment via qos_processor, could you run following MSSQL query? It will list all qos metrics that are duplicate in your environment.  
-----
SELECT origin, nim_origin, robot, host, probe, qos, source, target, r_table, h_table, table_id, created
FROM (SELECT origin, robot, probe, qos, source, target, r_table, h_table, table_id,host, nim_origin, created ,ROW_NUMBER()OVER(PARTITION BY [qos], [source], [target]
ORDER BY [created] desc ) AS RN FROM s_qos_data with(nolock))S
WHERE s.rn <> 1
ORDER BY qos, robot, source, target, created
----
And if possible let me know what are the results in your environment?
Thanks

[Garin Walsh]

Luc, it may not be helpful but I figured I'd add the data point anyway.

We don't use enriched origins so consider this a zero use case. We do "use" the default qos_processor behavior which is supposed to track and react to origin updates/changes for data and propagate that to the database.

That "feature" just doesn't work - or it only works sometimes - hard to tell. Regardless we are constantly deleting from s_qos_data and related tables records for systems that are duplicated - where in my understanding they should have been updated with new data rather than creating a new record in the table. This has been a "it's always done this" sort of thing so I don't think that's specifically related to a new behavior you are seeing.

Anyway, your query resulted in 40 rows of results. My s_qos_data table hovers at around 900,000 records so 40 isn't that egregious - granted if a customer is going to call to complain about something it'll be about the 0.001% issue, not the 99.999%. 

I then ran our process to remove the refuse that's left behind after renaming systems and reran your script and it resulted in zero rows.

[Luc Christiaens]

With 20.40.6 (=cu6) the enriched origin ideas are both implemented.

1 - have the correct enriched origin in cm_computer_system (this was done in a previous 20.40.x update)

2 - have the correct enriched origin information in device detail screens. Here it's implemented with a new field: "Latest Origin"