WAAE - is it possible to setup josb in a box to only run one-at-a-time?

[LeRoy Graham]

​Hi
We are using CA WA AE. I want top know fs there a way to tell WA AE "for Box A only run 1 job at-a-time"? The reason this is required is explained below...

I have a box with 40 jobs, the box is scheduled to run Mon-Fri at 9AM, the jobs are scheduled to run from 10PM that night at 5 min intervals specified in the start_times attribute.

When the box starts each job is automatically placed ON_HOLD (using auto_hold: 1).  At 3PM during the day we run a script to check if we have received data feeds for any of the 40 jobs (one feed per job). For each fed received the script will instruct CA AE to place the corresponding job OFF_HOLD ready to run that night.
Even though we have 5 min intervals between jobs and also use the notrunning starting condition there are occasions when the jobs overlap, when this happens several jobs might fail (due to locks on the database table they are updating.) and this is something we want to avoid.

The notrunning condition does not give us the control we want like so I am looking for better solutions. I wondered if there was a way to ensure only one job at a time will run in this box?, any thoughts / suggestions/ ideas?
   

[John Hiett]

Hi Graham,

Have you considered using renewable resources, documented here. Alternatively, you could try virtual machine queuing, documented here.

Kind regards,
John

[LeRoy Graham]

Thank you John
You know I had a look at virtual machines a while back but could not get my head around what the manual was telling me. Since you email I went and look at the newest version of the user guide and it makes sense now!, it certainly looks like a runner.

​

------------------------------
Your help is very much appreciated
LeRoy
------------------------------

[Steve Carrobis]

OMG why do you do auto hold? 
check if we have received data feeds for any of the 40 jobs (one feed per job). For each fed received the script will instruct CA AE to place the corresponding job OFF_HOLD ready to run that night.
Even though we have 5 min intervals between jobs and also use the notrunning starting condition there are occasions when the 

That's a filewatcher ... 
also you can do one at a time with s(job) or d(job)
as well as using workload balancing. 
you are scheduling the scheduler. use filewatchers and let the scheduler do the heavy lifting. 

[LeRoy Graham]

We do not get all 40 feeds every day, the number could be zero, 6, 10, 2, we don't  know in advance. 
We have already tried various configuration and the best so far is what we have in place.

Yes we could use FileWatchers, but our script does more that FileWatcher can such as sending out an alert to confirm file has been received and the job is scheduled to run that night.

using starting condition won't always work, for example, if ....
Feed A arrives and FileWatcher is successful, Job A is activated to run at 20:00
Feed B arrives and File watcher is successful JoB is activated to run at 20:05, condition is d(Job A)
Feed C does NOT arrive FileWatcher terminates, Job C is NOT activated  (20:10 normally, condition is d(Job B)
Feed D arrives FileWatcher is successful, job D is activated for 20:15  condition is d(Job C) - this job will NOT RUN  because Job C is Inactive 


------------------------------
Your help is very much appreciated
LeRoy
------------------------------

[LeRoy Graham]

   


​

[Steve Carrobis]

forget time. use dependency. have the filewatcher kick off. and you can send alert mail. send_notification on. 
and you are done. LeRoy it appears you are married to your code which schedules the scheduler. I am her to tell you . sit back and say to yourself how do i make this better and more hands off.
auto_hold. and off hold is way too manual! 
But what do i know, i have only been doing this for decades.. ;-)

as for only processing one at a time i said use workload automation. use a an autosys virtial machine with one real machine assigned
max_load: 1
each job you give it a job_load: 1 and priority incremental to the number of jobs
star at 1 and end with 40. This isnt hard you just have to WANT to do it. 

Good luck..
Steve C.

[LeRoy Graham]

Hello Steve
I thought I recalled your name from the Sterling System days​ :-)

------------------------------
Your help is very much appreciated
LeRoy
------------------------------

[Steve Carrobis]

 

You would recall correctly I am that Steve C. (Stirling Systems Group) now FTE at GM.

 

 

Steve C.

 

Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.