U02000383 Web Socket connection closed with status code 1011

[Olgun Onur Ozmen]

Hi Community,

for the last few days we have been seeing down/up in our random random agents for no reason. it goes down/up instantly and continues its work. i did not open a case directly because there are 4 lines of log that make sense. i share the relevant part of the error log for the sample rest agent below.  it seems to be an error code specific to automic. any ideas? 

note: all components is 21.0.9

JCP log:

20241226/185922.986 - 89     U00003412 Agent 'agentname_WEBREST' logged on (Client connection='CP006#00000053').
20241226/185923.078 - 37     U00029416 Agent Challenge authorization successful : 'agentname_WEBREST' 

Agent log:

20241226/185922.794 - U02000383 Web Socket connection closed with status code '1011 (Coordination "com.automic.message.queue" with ID "27766417" has failed.)'.
20241226/185922.798 - U02000379 Initiating connection to server 'enginehost:8443' using WebSocket URI: 'wss://enginehost:8443/agent'.

Thanks.

------------------------------
Olgun Onur Ozmen
https://www.linkedin.com/in/olgunonurozmen/
------------------------------

[Shannon Hebert]

Hello Olgun,

If you don't find a solution here, our support teams will be happy to help you investigate further.

Thank you,

-Shannon

------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
------------------------------

[Eric Lontz]

I searched through our Agent logs, which we are ingesting with Splunk. Unfortunately we are not ingesting the webservicerest agents; however we are with RA FTP. I see a number of these messages in my logs (status code 1011), which I can directly correlate to server restarts due to our internal patching schedule. These also usually accompany, in my case, a report file stuck in status Transferring in Process Monitoring. This doesn't appear to cause any fail conditions on the JOBS's objects or stop processing. 

Here is one example from our log

20241209/024929.066 - U02000383 Web Socket connection closed with status code '1011 (INSERT INTO MQ1WP (MQWP_BAcv,MQWP_BTable,MQWP_CAcv,MQWP_CAddr,MQWP_CSRName,MQWP_DRole,MQWP_FAddr,MQWP_Len,MQWP_Msg,MQWP_Pri)'

1011 is an unexpected close condition. This could be a failed insert due to db locks or the port shutting down. 

In our case it is not disruptive and the JOBS will queue while the agent reconnects and begin processing when it reconnects.

I have not been able to correlate these messages in my logs to anytime other than servers restarts.