AutoSys Workload Automation

Expand all | Collapse all

CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

  • 1.  CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Broadcom Employee
    Posted 02-21-2017 11:17 PM

    CA Tuesday Tips: Are you clearing spool files? Jobs will not run if a spool file cannot be created.

     

    You must clear agent spool files periodically to maintain disk space availability

    If the agent cannot create a spool file your jobs will not run on the agent.

     

    You can configure the agent to clear the workload spool files automatically by modifying the agentparm.txt file. 

     

    Example: Delete Spool Files Older Than 10 Days

    Suppose that you want to configure the agent to review the spool files every 36 hours and delete spool files that are older than 10 days.

    Add the indicated values to the following parameters in the agentparm.txt file:

    runnerplugin.spool.clean.enable=true runnerplugin.spool.expire=10D runnerplugin.spool.sleep=36H

     

    For additional options and examples please see the agent documentation:

    https://docops.ca.com/ca-workload-automation-system-agent/11-3/en/configuring/maintain-spool-and-log-files/spool-file-maintenance#SpoolFileMaintenance-ConfiguretheAgenttoClearSpoolFilesAutomatically



  • 2.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 02-22-2017 12:28 AM

    Good one, except this caveat: Even spool files of active jobs get deleted if you set the expire value to too less/frequent a value. For example, you set the value to 5H (5 hours) and you have a job running longer than 5 hours, then the spool of that job would get deleted as well.



  • 3.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 04-21-2017 07:44 AM

    It is always best to only have the completed files removed. on success or delete failed spools after so many days. 

    as the old agent worked. to me this other setting feels like clean_files but not respecting the file is still opened. 



  • 4.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 04-21-2017 07:46 AM

    That sounds like a bug to me Chandru_V.  Or undesirable at any rate!



  • 5.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 04-21-2017 07:55 AM

    agreed with the latter, CA / cybermation will state it works as designed .. *cough*



  • 6.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 04-22-2017 08:39 PM

    Agreed Antony, it does seem like a glitch. We'll take it up with Dev and update you back. Thank you, Chandru



  • 7.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-03-2017 05:29 PM

    Circling back on this.

     

    Agent Dev said the current spool maintenance is time-based file management and does not associate any jobs, active or completed.The Agent documentation has been updated with the following [caveat];

     

    runnerplugin.spool.expire
    Specifies the file expiration time. The agent deletes spool files that are older than this value.

    When spool files are removed automatically, all files and empty directories under the spool directory that are older than the time specified in runnerplugin.spool.expire are deleted, regardless of their source or the state of the job that generated them. To ensure that files for running jobs are not deleted, the value in runnerplugin.spool.expire should always be larger than the time it takes for the longest-running job to complete.



  • 8.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-10-2017 02:14 PM

    It's good that the documentation has been updated and bolds the fact that files will be removed even for running jobs. This will of course result in a chase failure in any environments that regularly schedule chase -E to ensure jobs that are truly not running anymore are updated as failed and can run when their starting conditions have been met. This domino effect can result in parallel processes being triggered when they shouldn't be. It would be best for the log cleanup process to only remove files that are no longer active. Do we need to raise an idea for this? Trying to identify the longest-running job for each agent is futile. Jobs come and go and some will always run longer than others. That's not a feasible suggestion.



  • 9.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-10-2017 02:24 PM

    In the real world, Lisa's suggestion "It would be best for the log cleanup process to only remove files that are no longer active", is a better solution than this "the value in runnerplugin.spool.expire should always be larger than the time it takes for the longest-running job to complete".

    What is the logic behind this: "spool files are deleted, regardless of their source or the state of the job that generated them" ?



  • 10.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-10-2017 02:44 PM

    This is all fallout from the cybermation product and not autosys. with that said however, i am unsure how cleanfiles behaved cleaning up the auto_rem files. oh wait that's right it was there to clean up the failed stuff :-)

    so yeah its cybermation fallout. 



  • 11.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-11-2017 12:15 AM

    Hi Lisa,

    Please raise an Idea for this feature enhancement.

     

    Thank you,

    Chandru



  • 12.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-09-2017 10:58 AM

    Does this apply to long running file watcher jobs?  I'm certain some of our app teams have fw jobs that run for a month or more...  They should probably restart the process daily.



  • 13.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-09-2017 05:13 PM

    Yes it does apply  to FW jobs too.

     

    Thanks & Regards,

    Chandru

    CA Technologies



  • 14.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-10-2017 12:14 PM

    I think the agents should behave like the EPs, if the agent cannot write to the directory SHUTDOWN.



  • 15.  Re: CA Tuesday Tips: Are you clearing agent spool files? Jobs will not run if a spool file cannot be created.

    Posted 05-10-2017 01:52 PM

    +1 for Steve's suggestion.

     

    Perhaps, also have a setting in the agent parm such as:

    runnerplugin.spool.level = n [n= level of debug detail]

    runnerplugin.spool.archive = n [n= 0.1.2.3. as in the "log" settings]

    runnerplugin.spool.maxsize=n  [n= max size of spool file]

    in addition to the already available:

    runnerplugin.spool.clean.enable=true
    runnerplugin.spool.expire=7d

     

    Chris  <CJ>