AutoSys Workload Automation

 View Only
Expand all | Collapse all

External jil validation (autoextvj) - job_type management in pJobInfo and pJob structures

  • 1.  External jil validation (autoextvj) - job_type management in pJobInfo and pJob structures

    Posted Jan 15, 2021 08:24 AM


    The topic is about managing external jil validation rules using the autoextvj functionnality. 
    After installing the api packages for autosys 11.3.6SP8, we build a first poc release of a configuration and internal validation process of jil, which :
     
    1) should determine the job_type of the object manipulated in jil
    2) loads an external yaml file with "rules configuration". The file is determined by the job_type, since we want to apply different check rules if the job type is a box, a filewatcher, or a command job. 
    3) apply the rules of the pJobInfo structure, the return denied if a rule fail, or validated if all rules are ok. 

    About the two available structures AUTOEXTVJ_JOBINFO and AUTOEXTVJ_JILKEYVAL passed as parameters to the AUTOEXTVJ_ValidateJob function, there is something strange about the job_type and box_name members of the JOBINFO structure. 

    Whenever the job_type attribute is applied in the jil statement, the pJobInfo->job_type member of the JOBINFO structure is defined (CMD, SAP, whatever...). 
    Whenever the jil statement is an "update", the pJobInfo->job_type member contains the symbol "^", and not the kind of job_type. 

    This is weird, since when we update an object via a jil update statement, the current values of the object (retreived from database) and the values defined in the jil statement are merged into the pJob (JILKEYVAL) struct, but the job_type and box_name members are not up to date in the JOBINFO struct. 

    As a matter of fact, we can't determine what kind of object it is when using an update jil statement. 

    This is definitively a caveat, since we can't work on attributes other than the common jil attributes for all kind of jobs (description, owner, application). All rules related on a specific job_type will be crippled. 

    To expose the problem with concrete sample data : 

    [ltual@bdfmgr1 src]$ echo "insert_job: sample_command_job \
    job_type: cmd \
    owner: ltual machine: localhost \
    command: /bin/true \
    description: sample_command_job \
    group: unix \
    application: LIBAUTOEXTJV \
    " |jil
    ______________________________________________________________________________
    
    CAUAJM_I_50323 Inserting/Updating job: sample_command_job
    CAUAJM_I_10719 AUTOEXTVJ Information: BDF control jil conformity
    conf: /home/logiciel/WorkloadAutomationAE/autouser.Q11/autoextjv_CMD.yaml
    job_type: CMD
    DEBUG : job_name: sample_command_job       <- from the JOBINFO structure
    DEBUG : box_name: ^                        <-
    DEBUG : job_type: CMD                      <-
    CAUAJM_I_50205 Database Change WAS Successful!
    ______________________________________________________________________________
    
    CAUAJM_I_52301 Exit Code = 0
    ______________________________________________________________________________


    Since this is obviously a command job, we loaded our "autoextjv_CMD.yaml" configuration file (which includes rules like : no empty description, no empty application, group in unix or windows values, owner=root ? then no "su -" command and so on...).

    But if we do update this job

    [ltual@bdfmgr1 src]$ echo "update_job: sample_command_job  description: " | jil
    ______________________________________________________________________________
    
    CAUAJM_I_50323 Inserting/Updating job: sample_command_job
    CAUAJM_I_10719 AUTOEXTVJ Information: BDF control jil conformity
    conf: /home/logiciel/WorkloadAutomationAE/autouser.Q11/autoextjv_^.yaml
    DEBUG : job_name: sample_command_job    <- ok, the job_name
    DEBUG : box_name: ^                     <- ok, we never defined a box
    DEBUG : job_type: ^                     <- gnii ?? why no job_type in pJobInfo ?
    CAUAJM_I_50205 Database Change WAS Successful!
    ______________________________________________________________________________
    
    CAUAJM_I_52301 Exit Code = 0
    ______________________________________________________________________________


    Okay, we did not specify job_type: cmd in the update statement, but this is still a command object ?
    -> the lib tried to load a autoextjv_^.yaml file, instead of autoextjv_CMD.yaml file. And then, we are stuck !

    The only way when updating is to specify again the job_type like in 

    echo "update_job: sample_command_job job_type: cmd description: a new description" | jil 

    And then the pJobInfo will have its job_type member set to CMD. Which is once again is weird.The database value is never retreive before the as_server passes through the autoext_validate function.
    And troublesome, update statement via tools like iXP will never keep the job_type attribute on an update statement... -> the autoextvj is doomed ;( 

    As anybody experience about how to retrieve the job_type of an object upon update jil statement in autoextvj ?
    Is this a bug ? 

    Lionel



  • 2.  RE: External jil validation (autoextvj) - job_type management in pJobInfo and pJob structures

    Broadcom Employee
    Posted Jan 19, 2021 12:37 AM
    Hi Lionel,
    job_type is not needed in order to update a job and one cannot change the job_type during the update, and moreover  the validation structure is provided with the user given input.

    I agree that the users can not load the proper *.yaml file based on their usecase.
    But currently as a work around the only way is to execute autorep -jq automatically and extract job_type and pass it during update_job.
    Anything other than this requires change in the product the way autoextvj library is designed.

    Thanks
    Venkat D


  • 3.  RE: External jil validation (autoextvj) - job_type management in pJobInfo and pJob structures

    Posted Jan 19, 2021 01:50 PM
    Hi Venkat. 

    We preferred to guess the job type (instead of adding a system call to autorep inside AUTOEXTJV_validated function) according to the present jil keywords in the struct. 

    watch_file ? this is a file trigger or a file watcher.
    sap_job_name ? this is a sap job type
    and so on. 

    Then we load our rules. The build is in testing. Seems fine. This is not as "clean" as it should be, but manage-able (the company uses a very small set of job_types). 

    Anyway, thanks for your post !

    L.

    ------------------------------
    IT Scheduler consultant
    ITS Group
    ------------------------------