PWP contention, zombie WPs, and a defunct MQSRV entries

[Michael A. Lowry]

This morning we upgraded an AE system from v12.0.8 to v12.3.2. When we were bringing the server up, WPs on two nodes began contending to become the PWP. At the same time, many other WPs were starting up but dying right away. As we watched, the WP numbers skyrocketed well beyond the normal range. It seemed that when the WPs were dying, their entries were not being removed from MQSRV, and their WP numbers were not getting re-used when a new WP started up.

We ended up doing a complete shutdown followed by a cold start of the system, first one one node, and then on the other. This appeared to resolve the PWP contention problem, and WPs began to come up normally. However, once all the AE server processes were up, there were still hundreds of entries in MQSRV corresponding to all of those abortive WP start attempts.

Will these defunct WP entries be cleaned out automatically, or do we need to do something to delete them?

[Tim Quakulinsky]

Hi Michael,
I am surprised: I thought that a cold start should also empty the table MQSRV. I can't tell you whether the entries are now automatically deleted. But I don't think so.
So far, I have only deleted orphaned server objects in client 0 after there were problems with WPs.
Good luck
Tim

------------------------------
Automation Evangelist
Fiducia & GAD IT AG
---
Mitglied des deutschsprachigen Automic-Anwendervereins FOKUS e.V.
Member of the German speaking Automic user association FOKUS e.V.
------------------------------

[Keld Mollnitz]

​Hi @Michael A. Lowry,
those orphaned WP entries will be automatically removed from MQSRV table after 10 minutes.​ The orphaned entries you see in Processes and Utilization must be removed manually..("work as design" according to Support (SURPRISE??!)

[Michael A. Lowry]

The defunct WP entries (going up to ~250) are still there one day after they first appeared. I see that it's possible to delete AE server processes in the Administration perspective of client 0, but I'm going to wait for official guidance from Broadcom Support. ​

[Timothy Yanosko]

Hi Michael,
  Just wondering if you ever get a response from Broadcom and if so can you share?

[Michael A. Lowry]

Thanks for the reminder. The SQL query I was using was doing an OUTER JOIN and so it was showing entries that I thought were in MQSERV. It turns out they had been removed already from MQSRV, and remained only in HOST. Deleting these the host entries in client 0 and running a DB reorg made them go away.