Data not being inserted into the database

[Alex Marques]

Folks
I have the following problem when performing the installation of version 9.20 in our environment with a mysql database.

Everything has been successfully installed.

Even after 30 days, data is no longer being entered into the database.

I have already examined the data_engine logs and there are no flaws with the data being added to the database.

data_engine 9.20
mysql 5.7

[Gregg Stillwell]

SO when you say no data, do you mean no new data after 30 days or you only capture the last 30 days? You can check your data engine retention section, and the mysql logs.   Also I believe there is a HF for 9.2 DE

https://support.broadcom.com/external/content/release-announcements/CA-Unified-Infrastructure-Management-Hotfix-Index/7233
https://knowledge.broadcom.com/external/article?articleId=33592

[Alex Marques]

Hi Gregg

When I say data is new, that data comes from a secondary hub, I see all the traffic coming. But when it arrives at the bank, it looks like it is not inserted, but I don't see an error in the data_engine probe logs.

example of part of the log

Aug  4 10:52:02:222 [140169700001536] 5 de: RREQUEST: hubpost_bulk <-10.15.100.231/48002  h=296 d=1134 fd=42 Aug  4 10:52:02:222 [140169700001536] 5 de: RREQUEST: hubpost_bulk <-10.15.100.231/48002  h=296 d=1134 fd=42 Aug  4 10:52:02:222 [140169700001536] 5 de:  head  mtype=100 cmd=hubpost_bulk seq=1101 ts=1596549122 frm=10.15.100.231/48002 tout=0 addr= sid=***** Aug  4 10:52:02:222 [140169700001536] 5 de:  data  bulk_list=PPDS(1083) bulk_size=2 qsize=2 Aug  4 10:52:02:222 [140169700001536] 2 de: DataCallback - processed 2 messages, sending reply: OK (0) Aug  4 10:52:02:222 [140169700001536] 5 de: SREPLY: status = 0(OK) ->10.15.100.231/48002 Aug  4 10:52:02:222 [140169700001536] 5 de:  head  mtype=200 status=0 seq=1101 Aug  4 10:52:02:259 [140169132893952] 2 de: InsertDefinition - QOS_POWER_STATE is already defined, qos def id: 1 Aug  4 10:52:02:259 [140169132893952] 2 de: HandleMessage: Insert Data - nimid: GV12025920-43952     (2020-08-04 10:50:22) table_id:        146 value=         1.00 time: 2020-08-04 10:50:22 qos=QOS_POWER_STATE      source=000hub_vacli         target=000hub_vacli            Aug  4 10:52:03:223 [140169700001536] 5 de: lockStatisticsFile LOCK Aug  4 10:52:03:223 [140169700001536] 5 de: lockStatisticsFile UNLOCK Aug  4 10:52:04:224 [140169700001536] 4 de: qos_check - Check Queue Loop.  Connected: 1 vbQueue: 1 Aug  4 10:52:04:224 [140169700001536] 5 de: lockStatisticsFile LOCK Aug  4 10:52:04:224 [140169700001536] 5 de: lockStatisticsFile UNLOCK Aug  4 10:52:05:225 [140169700001536] 5 de: lockStatisticsFile LOCK Aug  4 10:52:05:225 [140169700001536] 5 de: lockStatisticsFile UNLOCK Aug  4 10:52:06:226 [140169700001536] 5 de: lockStatisticsFile LOCK Aug  4 10:52:06:226 [140169700001536] 5 de: lockStatisticsFile UNLOCK Aug  4 10:52:07:227 [140169700001536] 2 de: SwapBuffers - swapping 1 rows to output buffer (RN_QOS_DATA_0001) Aug  4 10:52:07:227 [140169700001536] 5 de: lockStatisticsFile LOCK Aug  4 10:52:07:227 [140169700001536] 5 de: lockStatisticsFile UNLOCK Aug  4 10:52:07:245 [140169143383808] 3 de: [QoSData] thread_id: 51, commit, exec bulk query... Aug  4 10:52:07:248 [140169143383808] 3 de: infile_init_bigbuffer(data), total: 5001, buffer size: 50 Aug  4 10:52:07:248 [140169143383808] 5 de:  ----------------------------------- 146,2020-08-04 10:50:22,1.00,0.00,300.00,10800,\N Aug  4 10:52:07:248 [140169143383808] 3 de: infile_read_bigbuffer, bufsize: 8192, tot: 50, read: 50, left: 0 Aug  4 10:52:07:248 [140169143383808] 3 de: infile_end_bigbuffer Aug  4 10:52:07:249 [140169143383808] 3 de: [QoSData] thread_id: 51, query exec: ok -> LOAD DATA CONCURRENT LOCAL INFILE 'data' INTO TABLE RN_QOS_DATA_0001 FIELDS TERMINATED BY ',' LINES TERMINATED BY ' Aug  4 10:52:07:249 [140169143383808] 1 de: [QoSData] thread_id: 51, committed 1 rows to RN_QOS_DATA_0001 in 4 ms (ms/r:4) Aug  4 10:52:07:249 [140169143383808] 3 de: [QoSData] thread_id: 51, commit completed Aug  4 10:52:07:249 [140169143383808] 5 de: commit_thread - (COMMIT_THREAD_OK) commited OK 1 rows in 4 ms

[Gregg Stillwell]

Ah secondary hub may be a clue. do you have queues between secondary and primary hubs? Check the primary hub probe GUI Status tab. Lets see if the DE and queue flow quantity matches and is even working and configured. 

I only see powerstate QoS in your log no probe data but this may be your snapshot limitations.

[Alex Marques]

Gregg

I have queues between the hubs, yes, but I believe that is expected.
What I expected in the log would be the QOS of the net_connect probe.

You can see that between the hubs comes little information but looking at the data_engine queue it is retaining 26000 QOS for a 2000 queu

[Gregg Stillwell]

It looks OK in face value. DO you have all 3 QOS subjects on your attach queue. You are not sending very much. Can you see any qos for the secondary hub robot? Check with dr nimbus to see they are completing the journey. Don't think it is a db issue.

[Alex Marques]

Gregg

Through nimldr, I saw that only the net_connect probe QOS is not coming.
I did a test with the cdm probe and was able to enter information.

You can see only this alarm that comes

[Gregg Stillwell]

So you dont see cdm qos data? Are you using MCS? have Dr Nimbus sniff directly on secondary hub to verify robot not sending qos or is. You ahouls have enough to go on now to figure this out. Do you qos for cdm on primary hub and UMP robots?

[Alex Marques]

Gregg

I do see the qm of the cdm, I'm not using MCS, but this is having a strange behavior. I am re-deploying the net_connect probe to do a test.