QOS data not in OC

[Steve B]

Hi again,

Following on from my previous post, I've upgraded our lab UIM from 8.40 -> 8.51 -> 9.02 -> 20.3 which went mostly well. I had some minor issues connecting to the cabi instance but switching that over to HTTPS solved that.

After using UMP for so long, the OC interface feels like quite a departure. I've been trying to find the QOS data for the hosts but the interface comes up blank. The View the Metrics Palette page suggests where I need to go, but the Metrics palette comes up empty:

• I've followed the Troubleshooting Missing Data in UIM page to ensure there's actually QOS data in the database, which there is.
• The primary hub queues look good.
• The data_engine probe is putting qos data into the db

Any ideas or advice would be greatly appreciated.

Cheers!

[Luc Christiaens]

I have the same problem and opened a ticket 14 days ago
Update: that ticket is going nowhere, support tries to put this problem now on my layout of my test system (all functions on 1 vmware system) and even closed the case

[Steve B]

Good to know I'm not the only one...

Further to the above, when I click on a device from inventory view - there's nothing in here either:

Under key performance indicators, there's a help link which takes me to this page. I've followed it to configure a couple of monitoring profiles but I never see any data in the above view.

Something definitely seems broken here.

[Luc Christiaens]

The inventory view fields are correctly filled in with me, if I select here "metrics", the list will be empty

[Steve B]

Thanks @Luc Christiaens​, which probes do you have on that host? Assumingly CDM to grab the CPU / memory / disk info?

I've had been using cdm 6.50 but just noticed cdm-MC in the archive...

[Luc Christiaens]

On that host I have: cdm and ntevl via MCS (thus 6.50-MC)
Manually: logmon, dirscan, email_response,  emailgtw, dns_response, jvm_monitor, ...
The host where my metrics are displayed has also cdm and ntevl via MCS and logmon and snmpcollector manually

[Matthias Gruber]

Yes... same Upgrade-Path starting 8.4 :-> but with 9.02->20.1->20.3 and same problem no QoS

Well, Well... we know Flash is dying on 1.1.2021 since 2017 and CA got "something" buggy running  2 Month ahead of the deadline... cant belive all the wasted time and bugs and shortcomings we have to handle now. Very sad...

cheers
Matthias

[Luc Christiaens]

If you have this problem, please open an issue , so that support is aware that I'm not the only one

[David Michel]

looks like two different problems since Luc's lab is collecting metrics and yours is not. 
Configure cdm to collect some data. It's auto deployed during install to collect system stats for the OC robot. 
Then check to see if metrics show up at Inventory, and if so then check the Metrics option. 

Luc,
had your problem in my lab and in that case it turned out to be discovery problem with an old robot moved into the new 20.3 lab. 
If the problem exists for all robots then clearly there is a different cause.

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

Thanks David - i already have CDM on all hosts publishing data which I can see in the database. Just don't see any of it in OC - and there doesn't seem to be any troubleshooting documentation on where to start fault finding

[Luc Christiaens]

David,
One of the robots that is not displaying metrics in OC (edit metric) is my main hub (= the server from my screen shots)
Support suggested to remove the MCS deployment and change to a manual cmd, but this didn't change the behaviour

[Steve B]

thats interesting... all our hosts have the manual cmd, but i've changed them to the -MC variant with no change

[Christopher Moreland]

So I have found the same issue with the Metric Palette, and I have data showing in CABI for it.
Interestingly though, if you double click on the top bar that has the device name, then I get the metrics palette? ( I have opened a case for this :))

------------------------------
Chris
------------------------------

[Luc Christiaens]

Hi Christopher, thanks for the tip, indeed, when selecting the top bar the metrics palette is filled in.
This seems to work for a device, but not for a group.

[Christopher Moreland]

Thanks Luc. I tested that for Groups and got the same result as you, that it does not work. I will add that to the case.

------------------------------
Chris
------------------------------

[Steve B]

hey nice find @Christopher Moreland!

After spotting your post, I originally had no luck getting the same metrics. However, I followed the uninstall guide and then ran through the cabi installation from scratch and now too can see metrics when double-clicking on a host up the top:
​

I'm still not sure why I don't see any metrics in these areas though:

Does anyone have any advice?

[Steve B]

hello again!

i decided to have a play with the cabijs interface and may have found the issue. if we open any of the default dashboards, go to properties and check out where the data is coming from - we can see the following report sources:

However - if i check through the list of reports in the /public/ca/uim/reports/common/qos/ "folder" within cabijs, the above reports are missing:

I've re-read the Install or Upgrade for a Bundled CA Business Intelligence JasperReports Server page again and can confirm that the report packages mentioned have all been installed on the cabi robot:

Any chance someone would be able to do me a massive favour by posting:

• a screenshot of the installed packages on the cabi robot
• a screenshot of the reports in the /public/ca/uim/reports/common/qos/ folder cabijs

Thanks in advance. This has been doing my head in!

edit: maybe ive got all that wrong. looks like the reports are there if i go into edit:

[Luc Christiaens]

In my test environment (fresh install) the content of that ../common/qos directory is the same

[Steve B]

Thanks Luc,

You replied as I was adding my edit above :)

After trying to load the /public/ca/uim/dashboards/common/device_summary dashboard within cabijs with the chrome developer tools on, looks like something with the sql query:

which is odd, as it's able to pull in all the qos data as per my post above. need to dig deeper!

[David Michel]

redeploy these packages to the cabi robot
uim_core_dashboards_pack
uim_unified_reporter_pack

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

Thanks David,

I had already tried that but tried again anyway... same result. The problem seems to be within jasper itself. If I click on the UIM Summary report directly in cabijs, the \probes\service\wasp\webapps\cabijs\WEB-INF\logs\jasperserver.log shows the following:

2020-11-06T08:31:24,979 ERROR SecureExceptionHandlerImpl,pool-15-thread-81:116 - null
2020-11-06T08:31:24,980 ERROR SecureExceptionHandlerImpl,pool-15-thread-88:116 - null
2020-11-06T08:31:25,159 ERROR SecureExceptionHandlerImpl,pool-15-thread-84:116 - net.sf.jasperreports.engine.JRException: Error executing SQL statement for: KPI_List_Report.
2020-11-06T08:31:25,160 ERROR SecureExceptionHandlerImpl,pool-15-thread-89:116 - net.sf.jasperreports.engine.JRException: Error executing SQL statement for: KPI_List_Report.
2020-11-06T08:31:25,382 ERROR SecureExceptionHandlerImpl,pool-15-thread-85:116 - Error executing SQL statement for: QOS_Detail_Report.
2020-11-06T08:31:25,383 ERROR SecureExceptionHandlerImpl,pool-15-thread-91:116 - Error executing SQL statement for: QOS_Detail_Report.
2020-11-06T08:31:25,508 ERROR SecureExceptionHandlerImpl,pool-15-thread-92:116 - net.sf.jasperreports.engine.JRException: Error executing SQL statement for: device_available_metrics_tableDataset_1604611574990_293330.
2020-11-06T08:31:25,510 ERROR SecureExceptionHandlerImpl,pool-15-thread-88:116 - net.sf.jasperreports.engine.JRException: Error executing SQL statement for: device_available_metrics_tableDataset_1604611574990_293330.​

I've tried increasing the log level from ERROR to DEBUG for SQL query executer in cabijs -> Server Settings -> Log Settings but it doesn't provide any further info in the log file.

I've also confirmed the UIM Datasource and UIM JNDI Datasouce within cabijs work (ie Test Connection gives a Connection passed on both).

[David Michel]

Since several upgrades were done, at what point was cabi first deployed? 
What packages are deployed to the cabi robot? 
Admin Console > select the robot > Installed Packages > click the export icon, the square with the arrow > save
If cabi was deployed prior to 20.3 was it confirmed working? 
Were there problems with the deployment of cabi 4.30?

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

Cabi was only deployed after the 20.3 upgrade after we hit the "This requires CABI" message on the OC homepage.

Installed packages as per the attached csv.

There were no issues with the deployment and followed the installation documentation to the letter.

Attachment(s)

InstalledPackage_20201106_090729_data.csv   727 B 1 version

[David Michel]

Ah sorry, was reviewing this thread and saw you already provided the installed packages and yes they look spot on. 
Also my repository /reports/common/qos looks the same. 
Doubtful cabi probe log will have anything helpful on this. 
didn't find anything on the SecureExceptionHandlerImpl. 
out of ideas.

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

Those SQL query errors don't occur when watching the chrome console while looking at a device within OC. I'm assuming that I'm seeing those errors when looking at reports directly through cabijs as the variable for which group or device isn't being provided to the query. When the query is being sent via OC, it fills in the query with the particular ID and sql is happy.

I have noticed this in the console though:

I'm not sure why they're appearing as https://cbihost//webPageView.cs rather than https://cbihost/cabijs/webPageView.cs

The cabi_url is set in the raw config of the cabi probe:

edit: it's none of this either. All these missing files are in the \probes\service\wasp\webapps\cabijs\themes\default folder and for whatever reason, isn't being referred to. If I copy the files chrome is complaining about into \probes\service\wasp\webapps\ROOT, the errors go away - but still no data in the groups and device dashboards.

I think I've gone as far as I can go. Might sit out 20.3 or look at another product.

[David Michel]

perhaps this KB will help: 
https://knowledge.broadcom.com/external/article?articleId=202842

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

Thanks David - just gave that a go even though we're using the "normal" cabi (not external cabi).

 

I think @Luc Christiaens is using cabi_external and this parts of his dashboard appear to be working for him, maybe i should give that a go...​

[Luc Christiaens]

In 20.3 I'm using the bundled cabi (in  previous version I used the external)

[David Michel]

we have seen it help with both external and bundled cabi. 
think at this point it is best to open a support case.

------------------------------
Support Engineer
Broadcom
------------------------------

[Steve B]

I just installed 20.3.1 but now looks like double clicking on the hostname like @Christopher Moreland found​ doesn't show the list of available metrics anymore... 

anyone else tried it?

[Luc Christiaens]

After upgrade to 20.3.1 same behaviour, the bypass to see the metrics by clicking on the top device name is not working anymore!!
(please open issues for this)

[Luc Christiaens]

The reason, in my case, for this empty metric list was that we created QoS entries by custom probes that "did not" have the corresponding entries in the CA_UIM table: CM_CONFIGURATION_ITEM_METRIC_DEFINITION
Example: we had the custom probe: availability_monitor (community: https://community.broadcom.com/communities/community-home/digestviewer/viewthread?MID=778589) but didn't have the entries in CM_CONFIGURATION_ITEM_METRIC_DEFINITION.
Note: this seems only to happen if you create QoS metrics via the Perl calls: ciOpenLocalDevice or ciOpenRemoteDevice and the ci_type and met_id are not defined correctly in the tables CM_CONFIGURATION_ITEM_DEFINITION and CM_CONFIGURATION_ITEM_METRIC_DEFINITION