UIM - Spectrum integration

[Ivan Gasparec]

Hi all,

I have issues with my UIM-Spectrum integration. I have installed UIM 9.20 and Spectrum 10.4 and I have enabled integration over spectrumgtw 8.67HF4 probe. 

Let's say that integration is working partially, I can see devices and alarms from Spectrum in the UIM, but in the other way integration does not work as expected. Many alarms and devices from UIM is missing in Spectrum. Alarm incremental sync job is set to 30 sec, but sometimes this time is much more (up to 30 minutes), but again some alarms are missing (they are not only late). Also, sometimes integration stops working, then I need to delete cache folder of the spectrumgtw probe and restart the probe. 

Any ideas?

I have a case for my issue, but it is open for some time and I don't see any progress so I'm asking for community help.

[Silvio Okamoto]

Hi Ivan,

Engineering is currently investigating your issue. But the expected information was not found in the Tomcat log file ($SPECROOT/tomcat/logs/stdout.log or catalina.out).

a) If you have multiples OneClick web servers, did you gather the Tomcat log file from the OC where the integration is enabled?
You can verify that on either in Spectrum OneClick Console or UIM Admin Console.

b) Did you enable the following modules in debug mode on the OneClick web server where the integration is enabled?
Note that if the Spectrum Tomcat service is restarted, the debug mode is disabled, you have to enable them back.

c) Let the debug mode run for a while based on your synchronization interval, then gather the Tomcat log file.

d) Is the Global Collection used to synchronize the devices from Spectrum to UIM, a dynamic GC? or Static?
If it is dynamic, can you please share the search rule/criteria.

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

Hi Silvio,

a) Yes I have gathered the Tomcat log file from the OC where the integration is enabled.
b) Only UIM integration logs were enabled, but during the morning I have enabled both options. 
c) OK, done that. I have noticed for some alarms that I can see them in spectrumgtw log (searched by UIM alarm ID), but not in the tomcat log. That alarms are not visible in the OC. 
d) GC is dynamic, but this is not the issue, this direction is working ok and there is not much alarms from Spectrum. 

Regards,

Ivan

[Silvio Okamoto]

c) Besides the Tomcat log file, also upload the spectrumgtw.log.* files to the case.
d) Please double check in your dynamic Global Collection if there is any UIMHost model.

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

Logs with the details example alarm are attached to the case.

UIM GC is set to search "Devices only" and "Models: UIM Bidirectional Search"

[Silvio Okamoto]

There was a known issue in UIM Bidirectional Search. Please review this KB article:
https://knowledge.broadcom.com/external/article?articleId=93507

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

Ok, will try to do this.
But I do not think that this will resolve my issue because I have Spectrum 10.4.1 and this know issue is for 10.0; 10.1.x; 10.2.x, 10.3.

[Silvio Okamoto]

If you have UIMHost models in the GC, then you have to fix it.

I did not find the spectrumgtw creating the UIM alarm ID FM40560606-29534 in Spectrum side via RESTful - POST(ing) in the attached log file.
May 21 2020 13:42:22,496 [twScheduler_Worker-4] INFO UimAlarmConsumer - Alarm added to uim cacheFM40560606-29534
And the log end at May 21 2020 13:46:12

When this alarm is created in Spectrum side?
Can you share the spectrumgtw.log.* and stdout.log files after the alarm is created in Spectrum side?

Here an example when the spectrumgtw successfully creates an UIM alarm in Spectrum side:

Here is the snip from the Tomcat log file for the same UIM alarm ID:

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

Great, we are moving forward :-)

What could be the reason for that behavior of the spectrumgtw probe. 
We can see that some alarms are created and some not, but I can't find any pattern in the logs

I have checked and there is no UIM models in the GC, only models monitored by Spectrum. 

[Silvio Okamoto]

Once the UIM alarm ID FM40560606-29534 is generated in Spectrum side, please share the spectrumgtw.log and stdout.log files

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

I have noticed new information regarding this alarm (FM40560606-29534) so I have attached new tomcat log to the case. There is no anything new regarding this alarm in the spectrumgtw probe log, so I didn't share it. 
I can see that alarm was late. Unfortunately it is not active anymore, so I don't know if it was visible in OC.

[Silvio Okamoto]

Did you check the timestamp of spectrugtw.log.* files. I suspect the spectrumgtw.log.* files were overwritten.
You can increase the number of spectrumgtw.log.* files before them got overwritten.

Edit the log4j.xml file located in the spectrumgtw folder.
Increase the MaxBackupIndex from 5 to 50.

Also ensure the Maximum Log File Size is set to 100000.

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------

[Ivan Gasparec]

You are right, spectrumgtw logs were overwritten.

I have new, fresh example. Alarm UIM id is AJ12112954-49776.
I have attached tomcat and spectrum logs with the screenshots from UIM and Spectrum to the case.
You can see that alarm is 8 minutes late.

[Silvio Okamoto]

Thanks Ivan for example for delayed alarm in Spectrum. Let's await engineering's feedback on this.

You also mentioned about missing devices from UIM. Are they regular devices such as routers, switches?
What is the value of dedicated column for those devices when you run this SQL query?
select * from CM_COMPUTER_SYSTEM

------------------------------
Technical Support Engineer IV
Broadcom Inc
------------------------------