CA 7 Workload Automation

 View Only
Expand all | Collapse all

File Trigger Job Type question

  • 1.  File Trigger Job Type question

    Posted Jul 01, 2020 08:26 AM
    We are running CA7 11.3 and have questions re: File Monitoring in Unix. I've been experimenting with the File Trigger Job Type on Distributed Agents, and it works well. However, one of the requirements the users have is that the File Trigger only monitor for a specific period, if the file doesn't arrive by a specific time, they would like for it to abend, which would cut a high Incident ticket and page their oncall to notify them that the file is late. I don't see any options for this when defining the File Trigger job.

    I've used the "Filewatch" process with another Workload Automation product, and you could define these parms to make the Filewatch error out after a specific period of time. I see a "Filewatcher" script in the Agent install directory, but I can't find any documentation on the correct usage/parameters format for it.

    Any suggestions or direction to documentation would be much appreciated. Thanks!

    ------------------------------
    IT Engineer
    Cigna/Express-Scripts
    OH
    ------------------------------


  • 2.  RE: File Trigger Job Type question

    Broadcom Employee
    Posted Jul 01, 2020 11:54 AM
    There is nothing in the file trigger job definition that allow will the job to run for a specified amount of time and then fail. You can use the EXIST option on the FILENAME statement in the job definition. This will cause the job to run and look for the existence of the file and if the file is not there the job will fail. Here are couple scenarios assuming the file need to be created by 10:00 and if not the job should FAIL.
     
    Scenario 1
    Schedule your file trigger job to run at 10:00 and look for the file to EXIST. If the file does not exist the job will fail. Problem here, "what if the file came in at 8:00". When you file trigger job run at 10:00 it will complete, but this is now 2 hours later. See next scenario.

    Scenario 2
    If you know the file should come in some time between 8:00 and 10:00, you can schedule your job to run at 8:00 with the same EXIST keyword and setup an ARFSET to check for the job to FAIL and restart the job after waiting for 10 minutes or whatever wait time you decide. On the ARFSET definition you will use the RSTC (restart count) to determine how many restarts should be attempted. So, if you run the job every 10 minutes,  you will set RSTC LE 12. 
    ARF will detect the job fail and after wait period it will restart the job up to 12 times. This will take you up to 10:00.

    I hope this make sense. But this is all I can come up with. I also understand the ideal way would be to have an option for file trigger jobs to run for a specified time. But that does no exist today. Great enhancement request, that I recommend you post to the IDEA board. 
     
    -Roderick




  • 3.  RE: File Trigger Job Type question

    Posted Jul 01, 2020 01:37 PM
    Hey Rodney,

       Thanks for the quick response! The "abend after a certain period" piece is just one piece, and I think I understand what you're saying about the ARFSET solution. I will need to discuss that with the Scheduling Team, as I really don't understand the ARFSET function. You know I'm not really a Scheduler ;) Since the Filewatch process is really nothing more than the Unix "test" command, I may be able to use that if all else fails.

    But my pressing situation is that the Users want a File Trigger to be running & monitoring 24X7 for ANY files to be dropped into a specific folder on the server. Anytime it sees a file hit that folder, it should complete and trigger in a job that moves those files to another server without any delay, or manual intervention.

    So I have a File Trigger job setup on a Test box, and whenever it sees a file hit that folder, it completes and triggers in the File Move job. Then once the Move job completes, it triggers the File Trigger Job back in and the monitoring cycle starts all over again. This works fine 99.9% of the time. However, if there is any disruption between CA7 and the Agent (Like an IPL or server reboot), it seems to lose track of what it was doing. And even that disruption doesn't seem to be consistent. The File Trigger job survived an IPL 2 weekends ago, but not the IPL this past weekend. So that's weird.

    As I mentioned, I see a Filewatcher.sh in the Agent install Directory, and I know it requires some parameters, but evidently the parms it is looking for are different from the Filewatch.sh that comes with the TWS install, because I tried the same parms we used over there and it doesnt' like them. Any ideas on that Filewatcher script? Documentation on it maybe?

    ------------------------------
    IT Engineer
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 4.  RE: File Trigger Job Type question

    Broadcom Employee
    Posted Jul 01, 2020 02:38 PM
    Lowell,

    Development is looking into an enhancement that will give you what you are really looking for. The option to send a list of files/directories to the agent and it will constantly monitor for file creations / updates and send notification back to CA 7 so an action can be taken, such as trigger a job. Similar to what we had in the UUJMA agent.
     
    As far as the filewatcher.sh, I am not sure what that script does. It does not come into play with a file trigger job defined to CA 7. But I will look into it to find out what it is used for.






  • 5.  RE: File Trigger Job Type question

    Posted Jul 01, 2020 02:55 PM
    Ahh OK. That's cool. It would be a nice enhancement.

    SO maybe that Filewatcher in the install directory IS the same as the Filewatch script that comes with TWS. It just needs the right input parms. I'll keep looking, but it you find anything, let me know. Thanks Rodney!!

    ------------------------------
    IT Engineer
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 6.  RE: File Trigger Job Type question

    Posted May 20, 2024 03:04 PM

    Hey Rodney, I know it has been a minute since we had this conversation, but I'm being pecked to replace the "homegrown" Filewatch scripts that I created, and replace them with actual native CA7 File Triggers that WILL fail after a specified amount of time. Has there been any movement by development on this? 

    For some reason, I thought R12 was supposed to enable this, but we're on R12 of CA7 & the Agents, and I still don't see it.



    ------------------------------
    Sr IT Advisor
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 7.  RE: File Trigger Job Type question

    Posted May 21, 2024 11:30 AM

    Hey Rodney,

       Not being a jerk, but...that is what development said almost 4 years ago LOL

    But I will wait patiently, and continue to use the scripts I wrote to track files. Thank you sir.



    ------------------------------
    Sr IT Advisor
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 8.  RE: File Trigger Job Type question

    Broadcom Employee
    Posted May 21, 2024 04:45 PM

    Hi Lowell (good to hear from you),

    The enhancement request for continuous monitoring in the manner it was done with UJMA, where you can specify a directory and it will continuously monitor for any file that gets created/updated is still in product management review backlog. I will try and get an update for you. 

    As for being able to have a file monitor job start and FAIL if the file is not created within  specific time, that functionality does not exist within the agent. You will have to control this within the manager using some automation to cancel the job.




  • 9.  RE: File Trigger Job Type question

    Posted May 22, 2024 08:32 AM

    Hey Rodney,

       I will just use the scripts I threw together. They work fine, I had just been asked to look into a possible native function in CA7 or the Agent. Thanks again, and good talking to you too!



    ------------------------------
    Sr IT Advisor
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 10.  RE: File Trigger Job Type question

    Posted Jul 01, 2020 02:22 PM
    Hey Rodney,

      I think you can scratch that last pat about the Filewatcher script. I was able to take a look inside of it and it appears that may be the process that the File Trigger job tye executes to do what it does. So it's not the same Filewatch.sh I'm used to. Sorry about that.

    ------------------------------
    IT Engineer
    Cigna/Express-Scripts
    OH
    ------------------------------



  • 11.  RE: File Trigger Job Type question

    Broadcom Employee
    Posted Jul 02, 2020 10:42 AM
    The filewatcher script is something used by Autousys only and should not be removed or altered. CA 7 does not use that script.