CA Workload Automation ESP Edition

Expand all | Collapse all

file watcher - if file exists, fail the job

Jump to Best Answer

Chandru.Venkataraman07-24-2016 07:03 PM

  • 1.  file watcher - if file exists, fail the job

    Posted 07-23-2016 01:16 AM

    Hello,

     

    I have an ssis package that will generate a text file (ex, badData.txt) when certain condition is met, and save that file on a folder (ex, ../watch/) .

    I would like to create a job that watch for file with extension .txt. If file exists, raise alarm.

    The problem i have with file watcher is that if the file exists, it returns success - which is not what i wanted. I thought of using look back but can't figure out how to approach it.

     

    Is it possible to do what i need?

     

    Below is my file watcher job setting. I set alarm_if_fail to 0 because i only need it to alarm if success, and that is where i'm stuck at. Where do i go from here?

     

    /* ---------------------------------- */

    insert_job: nightJob   job_type: FW

    machine: winAgent

    owner: someone@US

    permission:

    date_conditions: 1

    days_of_week: mo,tu,we,th,fr

    start_times: "5:00"

    description: "watch for file with .txt extension"

    alarm_if_fail: 0

    watch_file: ../watch/*.txt

    watch_interval: 60



  • 2.  Re: file watcher - if file exists, fail the job
    Best Answer

    Posted 07-23-2016 04:08 AM

    Try file trigger instead (FT),  It has an option to cause the job to fail if the file exists; watch_file_type: NOTEXISTS (https://docops.ca.com/ca-wla-ae-wcc/11-4-2/en/reference/ae-job-information-language/jil-job-definitions/watch_file_type-attribute-specify-the-type-of-file-activity-to-monitor-for)

     

     

     

     

     

    Thanks and Regards,

    Chandru

    CA Technologies



  • 3.  Re: file watcher - if file exists, fail the job

    Posted 07-23-2016 09:52 PM

    Thanks, Chandru!

     

    Can I set the watch_interval and max_run_alarm just like the file watcher job? I would like the job to only stop when file exist or after 9 hours, which ever occurs first.

    The first point in the notes (see below) concerns me. Does "trigger occurs immediately" mean the job will report success immediately?

     

    NOTEXIST File Trigger Type Notes

    When specifying the NOTEXIST value in the watch_file_type attribute, consider the following points:

    • If the file does not exist when the job starts, the trigger occurs immediately.
    • If the job definition specifies a directory that does not exist or the directory is deleted during monitoring, the job fails.
    • If the file name specified in the watch_file attribute contains wildcards, the file with the most recent modification time that matches the criteria is monitored.


  • 4.  Re: file watcher - if file exists, fail the job

    Posted 07-23-2016 11:34 PM

    Hi,

     

    max_run_alarm is optional for all job types.

     

    watch_interval doesn’t apply to File Trigger (FT) jobs. If the file watched for exists when the job starts, the FT (with watch_file_type: NOTEXIST) fails and raises a JOBFAILUJRE alarm.

     

    Thank you,

    Chandru



  • 5.  Re: file watcher - if file exists, fail the job

    Posted 10-05-2016 02:12 PM

    Hi Chandru_V,

     

    I finally got this job into production couple days ago. But i've been getting false alarm. The job is supposed to fail only if a text file exists, but it failed regardless. I looked at the note on NOTEXIST, it says "If the file does not exist when the job starts, the trigger occurs immediately". This makes me wonder if  NOTEXIST is the right choice. Attached is the job log.

    When specifying the NOTEXIST value in the watch_file_type attribute, consider the following points:

    • If the file does not exist when the job starts, the trigger occurs immediately.
    • If the job definition specifies a directory that does not exist or the directory is deleted during monitoring, the job fails.
    • If the file name specified in the watch_file attribute contains wildcards, the file with the most recent modification time that matches the criteria is monitored.

     



  • 6.  Re: file watcher - if file exists, fail the job

    Posted 07-25-2016 03:14 PM

    I second that response.



  • 7.  Re: file watcher - if file exists, fail the job

    Posted 07-25-2016 03:20 PM
      |   view attached

    Another way is put the FW in a box and have the job after it fail if the filewatcher comes back.

     

     

    Steve C.

    Batch System Engineer

    Batch Innovation Team

    steven.carrobis@gm.com

    T (313)553-1727|  C (770)502-5519

     

     



  • 8.  Re: file watcher - if file exists, fail the job

    Posted 07-23-2016 03:00 PM

    Hello Jonathan,

    One possible solution is to create a simple job which will be started on Success of the File Watcher job and will use “sendevent” to send the alarm you want to send. Example :

     

    insert_job: send_alarm

    job_type: cmd

    command: sendevent -E ALARM -A JOBFAILURE -C "File with extension .txt found"

    machine: winAgent

    condition: success(nightJob)

     

    I hope this can help. I tried it and it worked perfectly.

     

    Thanks,

    Roya



  • 9.  Re: file watcher - if file exists, fail the job

    Posted 07-23-2016 09:54 PM

    Hi Roya,

     

    Thanks for suggesting me a solution (and even tested it too). This will be my back up plan if the FT (suggested by Chandru) job does not work in my environment. The FT job will save me from creating an extra job.

    Cheers!



  • 10.  Re: file watcher - if file exists, fail the job

    Posted 07-24-2016 12:09 PM

    I would like to mark responses from both, Chandru and Roya, as the "answer". But since I can only select one, I'd select Chandru's. His/her solution saves me from having an extra job to accomplish what I wanted.  Thanks both for your help!



  • 11.  Re: file watcher - if file exists, fail the job

    Posted 07-25-2016 05:50 AM

    Hi Jonathan,

     

    Thank you very much for your confirmation :-)

     

    Have a great day,

    Roya



  • 12.  Re: file watcher - if file exists, fail the job

    Posted 07-24-2016 07:03 PM

    Thanks, Jonathan!



  • 13.  Re: file watcher - if file exists, fail the job

    Posted 10-05-2016 07:35 PM

    Hi Jonathan,

    NOTEXIST causes the job to fail if the file does not exist, and that is what "trigger occurs immediately" means. Going by the error "unaccessible or an error has occured", it could be an issue with the network share access. Are you able to test with a local file first, please?

     

    Thank you,

    Chandru



  • 14.  Re: file watcher - if file exists, fail the job

    Posted 10-05-2016 08:07 PM

    Hi Chandru,

     

    My original question was for a job to watch for file with extension .txt. If file exists, raise alarm.

    Regarding the "unaccessible or an error has occured", it seems only file trigger job has a problem with it. 

    I created a file watcher job to test and it was able to access the network share drive and returned expected outcome. This doesn't mean the FW job did what I needed, but it proved the point that the file path was not the problem.

    Attached is the job definition for the FT job.

     



  • 15.  Re: file watcher - if file exists, fail the job

    Posted 10-05-2016 10:37 PM

    Filetrigger works different than filewatcher. We may need to investigate this behaviour with File trigger via a Case with CA Support. However, can you please share the following info to further help?

    1) Does the NFS (network share) required special access?

    2) What version of the System Agent runs on the machine_name

    3) Does the FT work if you didn't use wildcard *.txt? specify an absolute filename instead?

    4) Please consider using a local file for this test, that'll help isolate the issue to the NFS access with FTs.

     

    Thanks,

    Chandru