Operator Console version 20.3.3 Performance

[Henrik Vick]

Hello

We have just installed UIM version 20.3.3 (upgrade from 20.2) and it seem that Operator Console performance is hit by that.

Especially when looking into the inventory - it takes about 65 seconds to get the list... (about 7300 devices).

Does anyone have a good idea for performance improvements ?

Regards
Henrik Vick
Nets Denmark A/S

[Stephen Danseglio]

Hi Henrik,

7300 devices is not that large.

Check the row counts (select count(*) from... <tablename>), and level of fragmentation for these tables:

CM_COMPUTER_SYSTEM
CM_DEVICE
CM_COMPUTER_SYSTEM_ATTR
CM_DEVICE_ATTRIBUTE
CM_CONFIGURATION_ITEM
CM_CONFIGURATION_ITEM_METRIC
CM_CONFIGURATION_ITEM_DEFINITION
CM_CONFIGURATION_ITEM_METRIC_DEFINITION
CM_NIMBUS_ROBOT
CM_DEVICE
CM_COMPUTER_SYSTEM_ORIGIN
CM_CONFIGURATION_ITEM_ATTRIBUTE
CM_RELATIONSHIP_CI_CI
CM_RELATIONSHIP_CI_CS
CM_RELATIONSHIP_CS_CI
CM_DISCOVERY_NETWORK
S_QOS_DATA REBUILD;
S_QOS_SNAPSHOT
S_QOS_DEFINITION
NAS_TRANSACTION_SUMMARY
NAS_ALARMS
NAS_TRANSACTION_LOG

You can break it up and get results faster by using a where clause.

e.g., Check CM_* table index fragmentation first (fragmentation should be less than 30%), e.g., 

SELECT dbschemas.[name] as 'Schema',
dbtables.[name] as 'Table',
dbindexes.[name] as 'Index',
indexstats.avg_fragmentation_in_percent,
indexstats.page_count
FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS indexstats
INNER JOIN sys.tables dbtables on dbtables.[object_id] = indexstats.[object_id]
INNER JOIN sys.schemas dbschemas on dbtables.[schema_id] = dbschemas.[schema_id]
INNER JOIN sys.indexes AS dbindexes ON dbindexes.[object_id] = indexstats.[object_id]
AND indexstats.index_id = dbindexes.index_id
WHERE indexstats.database_id = DB_ID() AND dbtables.[name] like 'CM_%'
ORDER BY indexstats.avg_fragmentation_in_percent desc

Probably best to open a support case.

Steve

------------------------------
Support Engineer
Broadcom
US
------------------------------

[ahd ahd]

Hi Henrick,

we upgraded from 20.3.2 to 20.3.3 and immediately recognized the same behaviour.
The loading time of inventory is significantly longer than before.

Do you already have a solution for this?

Best Regards
Nico Müller
AHD

[Henrik Vick]

Hello Nico

Not found a solution yet... looking into provided SQL statement but due to vacation we will not look further into this before after Easter.

Best Regards
Henrik Vick
Nets Denmark A/S

[Garin Walsh]

I also have a similar issue - inventory went from sub second to open and paint to 60-100 seconds with the only significant change the upgrade from 20.3.2 to 20.3.3.

[Jason Mensing]

I'm seeing the same thing in our dev environment.  CPU on the OC spikes when a single user logs in and the inventory page is nearly unresponsive.  I have an order open with our server team to add more CPU.  I was hoping that's all it was but if others are having the issue it sounds like there's more to it.  Once the page did load I noticed an inventory count over 500k when the dev environment only has a few dozen robots.  The inventory breakdown shows over 500k virtual machines when we don't even have that many endpoints in our production environment.  I'm hoping to dig into this and check the SQL tables later today.  I'll likely open a support case as well.

[Etienne Winnubst]

Hi Nico,

We upgraded also from 20.3.2 to 20.3.3 and I'm seeing the opposite.
The loading time time has been highly improved.
Everything really feels smoothly now.
We have 5000 devices and the Database has been cleaned up from devices that hasn't been seen for a while.....
Upgrading from 20.1.0 to 20.3.2 was already an improvement in performance getting rid of Flash, but now it's even much better in version UIM20.3.3.
So I'm not sure what the difference could be, but with an upgrade you expect improvements, and that's what we are experiencing.

Regards,

Etienne

------------------------------
Support Consultant
Proact Netherlands BV
------------------------------

[Danilo Melo]

Hello everyone!

I just upgraded our production environment and the situation is similar.

The Groups tab now is loading really faster, at least compared to the way it was before.

However, the inventory is still slow to load, it takes at least 3 minutes to display everything, we have over 10 thousand devices. The behavior was the same before the upgrade and it persists now with 20.3.3.

One other thing, I thought this release was supposed to fix the SLA History not being able to select past periods, it is still not working.

Regards,
Danilo Melo

[Ashish Aggarwal]

SLA history is WIP and shall be reintroduced in the next UIM release.

------------------------------
Product Manager - ESD - Infrastructure monitoring solutions
CA Technologies (A Broadcom Company)
------------------------------

[Jason Allen]

I have asked our development team to look into this as multiple customers are experiencing similar issues after the upgrade to 20.3.3.

If anyone who is experiencing this would like to open a support case, and in the support case ask to be attached to defect DE498283 then it will help us keep track of this and we can notify you with any solution.  I will also post here once I have more details but it looks like something has changed in the way this page is populated.

[Henrik Vick]

Hello

I have now created a support case where I have mentioned defect DE498283.

I will keep you updated.

Regards
Henrik Vick
Nets Denmark A/S

[Ashish Aggarwal]

DE498283  impacts inventory management webapp only on Operator Console 20.3.3. It is related loading time of inventory view for Large and XL deployments and not a generic problem across operator console webapps. 
Fix for this defect shall be released as an hotfix by 3rd week on April, 2021.

------------------------------
Product Manager - ESD - Infrastructure monitoring solutions
CA Technologies (A Broadcom Company)
------------------------------

[Ashish Aggarwal]

Operator Console 20.3.3 HF1 has been published and is made available for download from the hotfix site - https://support.broadcom.com/external/content/release-announcements/CA-Unified-Infrastructure-Management-Hotfix-Index/7233

Some of the Key issues addressed in this patch include

1. Addition of golden metrics for Availability and Reachability QOS from pollagent, and allow Customers to configure any QOS/Metric as golden metric applicable for the environment.
2. Improved response time in rendering the Inventory view in Operator Console.
3. Enhanced Metric Viewer with line chart breaks for representing null data values.
4. ACLs to view/create/update MCS profiles in Operator Console.
5. Fixed the issue on device list in Group Config and Metric Viewer to adhere to Origins
6. Enhanced Group widget to display all the groups in the device details in OperatorConsole
7. Removed the references of external library and dependencies used in OperatorConsole

This patch is a set of webapps updated for addressing ~21 issues reported by the early adopters of 20.3.3.

1. mcs-ui-app-portlet-1.3.9-HF1.zip
2. uim_reportscheduler-20.3.3-HF1.zip
3. ump_cabi-4.2.4-HF1.zip
4. ump_operatorconsole-2.1.0-HF1.zip
5. ump_slm-20.3.3-HF1.zip
6. wasp_alarmviewer_api-2.1.7-HF1.zip

------------------------------
Product Manager - ESD - Infrastructure monitoring solutions
CA Technologies (A Broadcom Company)
------------------------------

[David Michel]

ump_operatorconsole-2.1.0-HF1.zip was replaced by ump_operatorconsole-2.1.0-HF2.zip

------------------------------
Support Engineer
Broadcom
NY
------------------------------

[Henrik Vick]

When I try to download the patch I get the following error in my browser (both edge and chrome) :


Can anybody help ?

Regards
Henrik Vick
Nets Denmark