Automic Workload Automation

 View Only
Expand all | Collapse all

WP with role "O" on 100% all day long

  • 1.  WP with role "O" on 100% all day long

    Posted Feb 24, 2021 10:45 AM
    Hi guys,

    any Idea why the WP with output- Role is permanetly on 100% despite all others are bored?


    Interestingly its only one client that uses it excessively..



    at 9:30 we switched the WP thats why the usage before is low...

    Any Idea from far away what can be looked for or what could cause this?

    many THX 
    Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------


  • 2.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 01:42 AM
    Hi Wolfgang,

    Do you find anything on the MQOWP table?

    ------------------------------
    Regards,
    Carlos
    ------------------------------



  • 3.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 02:31 AM
    Hi Carlos

    THX for your Input.

    Nothing unusual in my opinion 28 entries from today...

    Cheers, Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 4.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 02:40 AM
    Found a Bunch of Quarantine Messages, will delete them & check if its persinting.

    Thank you so far :-)

    cheers, Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 5.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 03:50 AM
    Damn, removed all Quarantines, no change...

    KR Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 6.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 02:30 AM
    Hi Wolfgang,

    You may also want to check if the execution plan is wrong. I have seen this to be the problem on a number of occasions...

    /Keld.


  • 7.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 03:50 AM
    Hi Keld,

    THX will take a look with DBAs.

    cheers Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 8.  RE: WP with role "O" on 100% all day long
    Best Answer

    Broadcom Employee
    Posted Feb 25, 2021 04:18 AM
    Hi,
    please activate trace ip=2 and db=3 for a couple of seconds. Then check the trace file of the WP with Role "O" and search for a repeating DEACT message and the attached eh_ah_idnr. Then search for this RunID in AWI. Looks like there is an object in the activities with an incorrect status (most probably due to system restart). 
    Then use "modify status manually" for this task and set it to "ENDED_CANCEL" or "ENDED_OK". This should solve the issue.
    Regards, Markus



  • 9.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 05:42 AM
    Hi Markus,

    THX a lotfor the info, unfortunately theres no Runid or eh_ar_idnr or similar for me to identify..

    SNIP: 
    20210225/113315.766 - UCUDEACT SEND DEACT to *SERVER JPEXEC_R MQWP MsgID: 0419321578 b-acv: 00000000 c-acv: 00000000 MsgType: W Prio: 255 ClientPrio: 200 Client: 4111 UserID: 0001666010
    20210225/113315.766 - EXIT UCUDEACT RET: 0000000000 TIME: 0000,00000 RETTEXT=' '
    20210225/113315.766 - STRT UCUMELD OPC: 00002 user-idnr=0000000000 client=04111 MSGNR=0000000000 MSG-INSERT=' ' quelle=0000000000 kategorie=0000000000 art=0000000000
    20210225/113315.766 - EXIT UCUMELD RET: 0000000000 TIME: 0000,00006 RETTEXT=' '
    20210225/113315.766 - STRT UCUREP OPC: 00216 client=00000 typ=' ' ah-idnr=0000000000 oh-idnr=0000000000 msgnr=0000000000 msg-insert=' '
    20210225/113315.766 - EXIT UCUREP RET: 0000000000 TIME: 0000,00001 RETTEXT=' ' blkcnt=0000000000'
    20210225/113315.766 - STRT UCUREP OPC: 00217 client=00000 typ=' ' ah-idnr=0000000000 oh-idnr=0000000000 msgnr=0000000000 msg-insert=' '
    20210225/113315.766 - EXIT UCUREP RET: 0000000000 TIME: 0000,00001 RETTEXT=' ' blkcnt=0000000000'
    20210225/113315.769 - STRT UCUMEM OPC: 00018 GARBAGE_COLLECTION
    20210225/113315.769 - EXIT UCUMEM RET: 0000000000 TIME: 0000,00002 RETTEXT=''
    20210225/113315.769 - STRT UCUMEM OPC: 00009 CHECK_FOR_MEMORY_LEAKS
    20210225/113315.769 - EXIT UCUMEM RET: 0000000000 TIME: 0000,00001 RETTEXT=''
    20210225/113315.769 - EXIT JPEXEC_R RET: 0000000000 TIME: 0000,008 DB: 0000,006
    20210225/113315.772 - JPEXEC_R RCV DEACT frm GMUC4U#WP001 MQWP MsgID: 0419321551 c-acv: 00000000
    20210225/113315.774 - STRT UCUMIGR OPC: 00002
    20210225/113315.774 - EXIT UCUMIGR RET: 0000000000 TIME: 0000,00000 RETTEXT=' '
    20210225/113315.777 - STRT UCUDEACT OPC: 0002
    20210225/113315.778 - U00009909 TRACE: (DB-DATEN: MQOWP_PK ) 0x7f0716289464 000004
    00000000 EB56FE18 >ëVþ.<
    20210225/113315.778 - >419321579<
    20210225/113315.778 - U00009909 TRACE: (Message-Key: MQOWP_PK ) 0x7f0716289464 000004
    00000000 EB56FE18 >ëVþ.<
    20210225/113315.778 - >419321579<


    do you have a hint?

    many THX
    Wolfgang​​

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 10.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 06:47 AM
    Hi Wolfgang,
    Is the PWP similar busy or only the OWP ?

    /Keld.


  • 11.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 08:08 AM
    Hi

    no it was only the WP with O - Role.

    We were able to solve it with the hint of Markus.

    Lookend in Activ.... Process Monitoring in Clt 4111 and Deactivated some Workflows and jobs that were in an end status.
    That solved the issue. Unfortunately my snippet of the Traces was 0/3 and not 3/3, sry for that.

    cheers, Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 12.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 10:34 AM
    Edited by Michael A. Lowry Mar 17, 2022 08:40 AM
    Do you use the z/OS Agent? We found a z/OS Agent bug last month that manifested in a similar way.


  • 13.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 11:36 AM
    Hi Michael,

    no z/OS agent... the cause was definitely the DB issue.

    THX anyway

    cheers, Wolfgang

    ------------------------------
    Support Info:
    if you are using one of the latest version of UC4 / AWA / One Automation please get in contact with Support to open a ticket.
    Otherwise update/upgrade your system and check if the problem still exists.
    ------------------------------



  • 14.  RE: WP with role "O" on 100% all day long

    Posted Feb 25, 2021 05:42 PM
    Hi @Michael A. Lowry,

    Which version of the z/OS agent ​is the bug related to?

    /Keld.


  • 15.  RE: WP with role "O" on 100% all day long

    Posted Feb 26, 2021 02:42 AM
    Edited by Michael A. Lowry Mar 17, 2022 11:58 AM
    It was 12.3.x. I'm not sure exactly what version. Now that I think about it, this problem might not have been related to the z/OS agent. We encountered several problems during the same period, and I might have gotten them mixed up. Anyway, I'll describe what I remember.

    The problem was that a workflow was not deactivated after all child tasks completed due to a failure to update the EH_ChildFlags field of the workflow in the EH table. The WP with role O was repeatedly trying to deactivate the workflow, but failing every time.

    You can see if you have the problem using this SQL query.

    SELECT EH_AH_IDNR,EH_CHILDFLAGS
    FROM EH WHERE EH_CHILDFLAGS LIKE '%J%'

    If any activities remain in this state, with a J in position 18, this is an indication of the problem. The fix is to replace the J with a blank space.
    UPDATE EH
    SET EH_CHILDFLAGS = ' 0 0 Y '
    WHERE EH_AH_IDNR = task_id;
    commit
     (where task_id is the ID returned by the first query).

    I recall that an update to the Automation Engine was planned that would resolve the problem in the following way: the Deactivate (forced) function would be updated so that it would be able to deactivate workflows in the aforementioned status.