Automic Workload Automation

Expand all | Collapse all

File event with multiple hits

  • 1.  File event with multiple hits

    Posted 06-12-2017 09:31 AM
    I’m interested in checking for the existence of files whose names match a specific pattern. If multiple such files are found, the event should process only the oldest one (based on a date stamp in the file name). What’s the best way to accomplish something like this?


  • 2.  File event with multiple hits

    Posted 06-12-2017 09:42 AM

    Hi Michael

    As of my understanding, the logic to only process the oldest file would be with the workflow/object the corresponding event activates.

    The question in that case would then be how to identify the oldest file or make this information available to the system?

    If you're on V11.2+ you might do a JOBS object to get the latest file and register the output using the REGISTER_VARIABLE & make it available using PSET / PUBLISH.

    Also an idea would be to use an EXEC VARA to extract this information. If your !Process activates a workflow, the workflow could get the latest file from the EXEC VARA and inherit this information to the jobs below or do an direct access using GET_VAR or - in JOBF objects - use the {} syntax.

    Regards
    Joel



  • 3.  File event with multiple hits

    Posted 06-12-2017 09:47 AM
    Good suggestions,joel_wiesmann_automic. My first idea was to use :PREP_PROCESS_FILENAME to get a list of matching files, and find the oldest by doing filename comparisons in AE scripting.


  • 4.  File event with multiple hits

    Posted 06-12-2017 10:05 AM

    Hi Michael

    To determine the oldest one you would need to have some timestamp in the filename I guess? As far as I remember the PREP_PROCESS_FILENAME returns in alphabetically sorted order and does not provide information on the age of the file if I'm not wrong.

    Regards
    Joel



  • 5.  File event with multiple hits

    Posted 06-12-2017 10:29 AM
     Correct. I would have to have an idea of the maximum number of files that could appear, store the names in an array, and then sort them. 


  • 6.  File event with multiple hits

    Posted 06-12-2017 11:40 AM
    I have used PowerShell in the past to do something similar, and then capture the results;

    $NewestFile = get-item "c:\temp\*.*" | sort creationTime -descending | select -first 1
    write-host "Newest File in the folder: $NewestFile.Fullname"


  • 7.  File event with multiple hits

    Posted 06-14-2017 05:21 AM
    On 2017.06.12, Joel Wiesmann wrote:

    If you're on V11.2+ you might do a JOBS object to get the latest file and register the output using the REGISTER_VARIABLE & make it available using PSET / PUBLISH.

    I set up a file event to periodically check for the presence of any files matching the pattern. This event starts the workflow that identifies the oldest file and then processes it.
    :SET &OBJECT_TO_RUN# = "UC0.MAL.HANDLE_FILE.JOBP"
    :SET &QUEUE# = "UC0"
    :SET &RUNID# = ACTIVATE_UC_OBJECT(&OBJECT_TO_RUN#,WAIT,,,,PASS_VALUES,&QUEUE#)
    :IF &RUNID# = 0 OR 20423
    :  PRINT "ERRROR starting object &OBJECT_TO_RUN#"
    :ELSE
    :  PRINT "Started object &OBJECT_TO_RUN# with run ID &RUNID#"
    :ENDIF
    The event is configured so that one one task can run at once. This, combined with using the WAIT keyword in the ACTIVATE_UC_OBJECT function that executes the workflow, should ensure that only one file is processed at a time.

    In the workflow, a UNIX job identifies the oldest file based on the datestamp in the file name.
    filename=$(find /tmp -name testfile_\* 2>/dev/null | sort -bn | head -1)
    echo "File name : $filename"
    :REGISTER_VARIABLE FILENAME#,$filename
    The post-process uses :PSET to set the variable at the level of the parent workflow.
    :PSET &FILENAME# = &FILENAME#
    A successor job uses the &FILENAME# variable to process the file.

    This seems to work well. If anyone has ideas for simplifying or improving on this approach, please let me know.