Clarity PPM

Expand all | Collapse all

CA Clarity Tuesday Tips - Process Maintenance

Christopher Hackett07-24-2012 12:58 PM

  • 1.  CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-23-2012 07:45 PM

    Credit: Jeanne Gaskill of CA Clarity Support

     

     

    DO YOU HAVE A PROCESS MAINTENANCE PLAN?

     

    WHY: If you allow too many completed processes to build up in your system, they can cause performance problems on the Initiated Processes tab and eventually a performance problem with your processes in general. When there are too many rows in your process tables, it eventually slows down performance. In addition, your processes that are in an error state are loaded EACH TIME you restart your process engine. This adds to process load time after the restart and it can cause process performance issues.

     

     

    There are two things you should do to maintain your processes:
    1. Delete old processes in the Done or Aborted state.
    2. Retry or Abort and Delete old processes in the Error state.

     


    DELETING OLD PROCESSES IN THE DONE OR ABORTED STATE

     

    The first thing that you need to do is to decide on and set a policy for how long you want to retain records of old completed and aborted processes. Remember if you are taking regular backups of your Clarity system/database you can go to archived information if you really need to access that data.

     

     

    Once you have chosen how long you are going to retain your completed and aborted processes, you will need to clean up anything older than the amount of time you chose. You should use the “Delete Process Instance” job to do this. It is important to make sure that you select specific dates such that you are deleting 5000 or less processes with each run of the job. Not only will the performance for the job itself go down if you try to delete too many processes at once, it may also affect performance for your users.

     

     

    NOTE: You will need to run the job separately to clean out Done and Aborted processes. This means for every date range, you will need to run the job with Done selected in the Process Status field and a second time with Aborted selected.

     

     

    After all of your old processes are cleaned up, it is time to choose how frequently you are going to run this job. If you are a very heavy process user, you will want to make sure and schedule this job to run frequently enough that you are only deleting a 3000 – 5000 users at a time. If you are deleting a lot of processes like this, it is best to schedule this job to run at off hours.

     

    Per the note above, you will need to schedule a separate job for Done and Aborted processes.[size]

     

     

    CLEANING UP OLD PROCESSES IN THE ERROR STATE
    Processes that have errored out also need maintenance. As I said before - these processes are loaded by the system and if you have a lot of them, they can eventually cause performance issues with your processes.

    Again, the first thing is to set a policy/schedule for how often this should be done so this doesn’t become a huge task.

     

    Now just go to the Initiated Processes tab and search for processes in the error state. Customers who run very few processes may only need to do this once a month.

     

    Customers who run many processes may need to do this once a week or even daily.

     

    When you are ready to clean up your errored processes, look at each process and determine the following:

     

    1. Determine whether or not this process instance is still needed. If it is not needed, abort the process.

     

    2. Check for any processes that you see repetitively in an error state. If all or many instances of the process are failing, you need to look at why this is happening. In many cases this indicates a process that has a problem with its logic. It is a good idea to go back to the process developer and ask them to fix the issue. If it is a problem with the process logic, put the process on hold until it can be fixed. You may need to ask the developer to give you a second version of the fixed process that continues from where the problem occurred so that you can run it on all of the object instances where it failed so that the process can complete

     

    3. If the process is still needed, double-click the error icon under the Messages column to see what the error says. If it gives you a checkbox next to the step, it means the problem is likely to be something you can fix. Read the error message and look at the process details page (go back to the list view and click on the process id) to determine what the problem is. Then go to the object instance indicated in the list view and fix the problem. Once the problem is fixed, you can click back into the error icon, click the checkbox beside the step and click the retry button.

     

    NOTE: sometimes it is a good idea to retry the step even if you don’t understand how to fix the cause. If it was a temporary error, the step will succeed and the process will continue.

     

     

    NOTE: If you cannot determine what is causing the problem with a process or how to fix what looks to be an error you can retry, contact support for assistance.

     

    Once all of your errored processes are cleaned up, the Delete Process Instance job with the Aborted status selected will clean up the aborted processes once they are old enough.



  • 2.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-24-2012 04:30 AM
    Thanks for the goodie.

    How would you deal with a process that fails and tries to restart repeatedly for ever and ever?

    Martti K.


  • 3.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-24-2012 10:26 AM
    Hi Marti,

    Do you mean that the same process is starting over and over again on the same object instance?

    If so, the most common problem that causes this symptom is a poorly constructed start condition for an auto-start process that starts on update.

    In the short term, put the process on hold (so that it quits restarting).

    Let's say you create a start condition that autostarts on update and is built on a field called Status (a static lookup) and one of the possible values is "Work in Progress". Your start condition is:

    Status = "Work in Progress"

    You have validated and activated your process.

    Now someone opens an object instance, changes the status field to your desired value, and the process runs as expected. Next, a little while later either that same person (or someone else) opens the same object instance and makes changes to some other field or maybe they make no changes at all. In either case they click on Save or Submit.

    Because the value of the Status field is still "Work in Progress", the process will run again. If this is a popular object instance, it will run again every time someone clicks save or submit on that process instance until the value for the Status field is changed to some other value.

    The way to fix this is to add an AND condition to the Start Condition using the Previous Value option.

    Your new Start condtion will look something like this:

    Status = "Work in Progress" AND Status != Status [Previous value]

    Another alternative is

    Status = "Work in Progress" AND Status [Previous value] != "Work in Progress"

    Either alternative will force the process to only run when the Status field has changed to the value you want and it was not that value previously.

    If I have misunderstood your problem, please open a case with support so that we can help you explore what is happening with your process.


  • 4.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-24-2012 01:20 PM
    Thanks Jeanne! Great post and follow-up! :grin:


  • 5.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-26-2012 04:11 PM
    Hello Jeanne,

    My question relates to a v12.0 SP 4 instance which was duplicated with a db dumb.
    There were some performance issues and when looking at the sessions we spotted a user who had a session initiated frequently something like 260 a day.
    To our knowledge the user was not logging into the system at all.
    The user was not running any scheduled jobs either so we looked at the processes and found that there was a particular instance which was starting over and over again and never completing.
    On the source system there was no such problem.
    We tried to stop it without success.
    Eventually a colleague deleted the record from the table.
    I don't recall the details and could not find any any more.

    So if you have such a process instance which apparently normally runs OK, but due to something in the system like restarting while it is running, you get into a situation where you can cancel the instance, can't delete it in the GUI what can you do to get it tamed?

    Martti K.


  • 6.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 09-05-2012 12:08 AM
    I have a question regarding the 'Delete Process instance' job. We have a process that call a sub-process. The completion mode is set to Independent. In the production envrionment we also run the Delete Process Instance job and it removes all completed instances. But we have observed that when the parent process is deleted it also deleted the running instance of the sub process.
    Is there any configuration change needs to be done to avoid this?


  • 7.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 09-05-2012 02:27 PM
    Hello SK2
    This KB may be related to your question, available on the Support site with keyword search "Clarity process delete sub"

    TEC573321
    "Clarity: Delete Process Instance Job Deletes Sub-processes That Are Still Running "

    should you need further assistance please log a Support case to allow specific troubleshooting and necessary follow-ups

    Regards
    Connie


  • 8.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 09-05-2012 02:29 PM
    Hello Martti
    If this is still an issue, please log a Support case so we can look into the details and properly address it with you

    Regards
    Connie


  • 9.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 09-05-2012 03:06 PM
    Thanks Connie.
    That was just an one time issue which was solved and did not reoccur.

    Martti K.


  • 10.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 07-24-2012 12:58 PM
    Thanks for all the great info Connie! :grin:


  • 11.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 10-15-2012 07:33 PM
    Hi Connie!!

    Great read! I will definitely look to implement a plan like this for Process Maintenance.

    I had a question regarding server downtime/process engines. What happens to processes that are supposed to be kicked off if the process engine is down? During testing of our application, we have had multiple times where the BG servers went down. Will processes get run as soon as the engine is back up?

    Also, what are recommendations for monitoring the process engine heartbeat for a failure? I know other threads recommended running a query to check for last heartbeats that occurred greater then 20 minutes ago.

    Thanks!

    Feras


  • 12.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 10-16-2012 06:49 PM
    Hi Feras,

    That answer depends on your configuration and how long the process engine(s) are down.

    If you have more than one process engine and one goes down, the following will happen:

    1. All new events will be processed by the remaining process engine(s). i.e. all new processes will be run on the remaining pe's.

    2. After two hours any processes residing on the down engine will be moved over to the remaining engine(s).


    If you only have one engine or if all engines are down for more than two hours, the events older than two hours (by default) will be cleared out of the nms_message tables so that there will no longer be a record of these expired events. Those processes will not start. This is a setting that can be controlled via the Clarity CSA/NSA. Most customers do not need to change this setting.

    Clarity does not provide an out-of-the-box method for monitoring the process engine heart-beat. The method you found that is used by other customers seems like a reasonable alternative. Since the heartbeat is supposed to occur every 5 minutes, and since checking that time should not be resource intensive, I would be tempted to run that query a little more often, say every 10 to 15 minutes checking to see if it has run in the last 5 or 6 minutes.

    Please let me know if you would like clarification on any of the above information or have additional questions.

    Jeanne Gaskill
    Senior Support Engineer - Clarity


  • 13.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 10-17-2012 05:48 PM
    Hi Jeanne,

    One of the sticking points I had when we upgraded to 12.1.1 was the fact that the heartbeat was not refreshing every minute automatically. This is good know! Thanks for posting!

    Stephanie


  • 14.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 12-03-2013 12:52 AM

    Hello All,

    We have round about 6000+ error processes do we have any way to cancel whole Error Processess in Single shot!!

    Thanks

    Awadhesh



  • 15.  RE: CA Clarity Tuesday Tips - Process Maintenance

    Posted 03-18-2014 01:35 PM

    This is great, thank you Connie, I do want to know if there is a way to implement a batch "retry" of numerous processes?? 

    "Once the problem is fixed, you can click back into the error icon, click the checkbox beside the step and click the retry button. "

    Thanks,

    Keri