Automic Workload Automation

 View Only
Expand all | Collapse all

Dependencies between UC4 systems

Michael A. Lowry

Michael A. LowryNov 18, 2022 06:03 PM

  • 1.  Dependencies between UC4 systems

    Posted Nov 08, 2022 08:20 AM
    We are in the progress of migrating from one cloud platform to another. During the transition phase, we will have two UC4 systems running side-by-side, and workflows will be gradually migrated from the old system to the new one. It would greatly ease the migration if we could set up dependencies between tasks in one UC4 system and tasks the other.

    Is there a way to do this?


  • 2.  RE: Dependencies between UC4 systems

    Posted Nov 08, 2022 09:52 AM

    Hi,

    I do not think that this can be done out of the box. Do you talk about external dependencies set in a workflow?

    Regards,
    Juergen



    ------------------------------
    Juergen Lechner
    Managing Consultant
    setis GmbH
    Germany
    ------------------------------



  • 3.  RE: Dependencies between UC4 systems

    Posted Nov 09, 2022 03:41 AM
    Hi,

    I also don´t know a solution for this out of the box.

    But a workaround could be triggering jobs via REST-API, which are set as external dependencies.
    For example:
    Workflow A in system A triggers at some point job A via REST-API in system B
    The workflow in system B has an external dependency for job A.

    Not an elegant way, but it should work.

    Regards,

    Stephan Schiller


  • 4.  RE: Dependencies between UC4 systems

    Posted Nov 09, 2022 03:50 AM
    Good morning,

    since this is a short term workaround, it will be the only way of doing this. It is a direct approach that does not need third party tools or data / status storage. I was also thinking about this too. But before that I wanted to know that I have understood the question. :-)

    Regards,
    Juergen

    ------------------------------
    Juergen Lechner
    Managing Consultant
    setis GmbH
    Germany
    ------------------------------



  • 5.  RE: Dependencies between UC4 systems

    Broadcom Employee
    Posted Nov 09, 2022 06:10 AM
    Hi Michael,
     
    I recommend creating dummy SCRI objects in the target system and using them as external dependencies to the dependent job. Using the RESTAPI (or any of our APIs) you can then execute the dummy SCRI object to fulfill the external dependency. Make sure you include somehow the name of the predecessor task name in the name of the dummy SCRI so that you can see which predecessor in the calling Automic instance fulfills the dependency.
     
    In theory you could also use SYNC objects (SET_SYNC), but in my opinion the usage of dummy SCRI objects is much more visible for the user. 
     
    Regards, Markus



  • 6.  RE: Dependencies between UC4 systems

    Posted Nov 09, 2022 06:32 AM
    Besides the already mentioned REST-API solution, Broadcom still ships the old "CallAPI" utilities for multiple operating systems (API\CallAPI folder from image file).
    Platforms for CallAPIs (automic.com)
    With this, the step supposed to trigger an object on the target system would be an OS step, that runs the CallAPI executable with connection information for the target system, and the script to trigger the object there using AE script language.

    Similar result using REST-API, also not elegant, but also prevents "too much" custom code.


  • 7.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 01:59 AM
    Edited by Michael A. Lowry Nov 18, 2022 03:06 AM
    My first attempt to post this reply failed, and the content was lost. Trying again.

    As I see it, there are two main approaches to setting up dependencies between jobs in different UC4 systems.
    1. Polling from successor system
    2. Trigger from predecessor system

    Current set-up

    Approach 1: polling from successor system

    Approach  2: trigger from predecessor system



    Approach

    Pros

    Cons

    1. Polling from successor system

     

    • Simpler for batch developers to implement.
    • No changes required to predecessor workflows.
    • Polling means more network traffic and at least some delay between completion of the predecessor job and start of the successor job.
    • Development of status checker job could be somewhat involved.

    2. Trigger from predecessor system

     

    • No polling, just a single trigger upon completion of the predecessor task.
    • Requires changes to both the predecessor and successor workflows.
    • Requires even more work if more than one successor follows the same predecessor.


    Did I overlook anything? Thanks in advance for additional feedback and suggestions.




  • 8.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 02:30 AM
    Edited by Michael A. Lowry Nov 18, 2022 03:05 AM
    I can already say that we are leaning toward the first approach: polling from the successor system. The main reason is simplicity for the people developing the workflows. There are hundreds of "batch developers" in the company, each responsible for his or her own batches of workflows and related objects. We don't want to force them to make big changes to their workflows. The more painless we can make the migration for them, the more likely they will be to embrace the change.

    This brings me to the status checker job. It probably won't be a job, but rather a workflow of its own, with several child objects that do the work of resolving the remote dependency. The status checker will have to take several parameters. These are derived from the parameters one can specify on external dependencies:
    • Object name
    • Within parent
    • With alias
    • Task status
    • Same logical date
    Additionally. the status checker will need to read several parameters to tell it in what UC4 system to find the predecessor task, how often to check, and so on:
    • Remote UC4 system name
    • Polling interval
    • Maximum number of polling attempts
    • Maximum time to check
    Once we have developed the status checker, it'll be relatively straightforward for batch developers to drop in this object in place of external dependencies setting the required parameters when they do so.

    The parameters will likely be set via object variables overridden at the workflow task level. (We might even use a prompt set for this, to aid input of valid values. The sticking point is probably finding a way to ensure that all prompt set values are set ahead of time, so that execution is not interrupted when the status checker starts.)



  • 9.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 02:49 AM
    In your polling approach you could get some trouble if "JOB2" runs more often than one time. It could be possible that the wrong "JOB2" is detected. If this doesn´t occur in your system or the parameters in combination (object name, within parent, with alias) are unique, the polling approach should be fine.

    Regards

    Stephan Schiller


  • 10.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 02:56 AM
    Edited by Michael A. Lowry Nov 18, 2022 03:04 AM
    Yeah, there are options one can specify to fine-tune the behavior of external dependencies. In our case, recreating this logic is probably overkill, because very few people are actually using these options. The MVP will cover the most common use case. We can add features later if necessary. :)



  • 11.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 03:06 AM

    There are several solutions to the parallel jobs run.
    1. Modify the Alias and add the RunID to the Alias in the pre processing 
    2. When triggering the job, add the RunID as Key and the job name as value to a VARA. The status checker checks the VARA and deletes the entry when the job is finished

    We are using both (regarding to the usage) successfully since years.

    Regards

    Peter



    ------------------------------
    Best regards

    Peter Gross
    Senior System Engineer
    Raiffeisen Schweiz
    St. Gallen Switzerland
    ------------------------------



  • 12.  RE: Dependencies between UC4 systems

    Posted Nov 21, 2022 06:17 AM
    Edited by Michael A. Lowry Nov 21, 2022 06:22 AM
    I have developed a proof of concept for a workflow that can be used as a drop-in replacement for external dependencies. The dependency options are specified via a prompt set. A REST job fetches the remote task status, and a second REST job fetches parent task info if necessary. Retries are accomplished via task post-conditions.


    All of the checks up to and including the check for same logical date are working. The remainder are yet to be implemented, and may not be depending on how many people are actually using these options today.

    And speaking of that, here's a little SQL query I wrote that lists all but the most commonly used types of external dependencies in the AE DB.

    SELECT OH_Name, JPP_Object, JPP_ParentObject, JPP_Alias, JPP_ParentAlias, JPP_ExtWhen, JPP_ExtSLTType, JPP_ExtSLTWithin,
    JPP_ExtElse, JPP_ExtElseAlarm, JPP_ExtTimeout, JPP_ExtTimeoutE, JPP_ExtTimeoutA, JPP_ExtExecute, JPP_PreCntExt
    FROM OH LEFT JOIN JPP ON OH_IDnr = JPP_OH_IDNR
    WHERE OH_CLIENT = 100
    AND OH_DeleteFlag = 0
    AND ( JPP_ParentObject IS NOT NULL OR
    JPP_ParentAlias IS NOT NULL OR
    JPP_ExtElse <> 0 OR
    JPP_ExtElseAlarm <> 0 OR
    JPP_TimeoutFlag <> 0 OR
    JPP_ExtExecute IS NOT NULL )
    ORDER BY OH_Name, JPP_ParentObject
    This will help us to get an idea of how prevalent the different kinds of options are, so that we can decide which ones are worth the effort of recreating in the dependency checker tool.


    Here are a couple more SQL queries.

    Populate the Check expected status pick list

    SELECT ZUTYP_STATUS
    FROM UC_ZUTYP
    WHERE ZUTYP_ISALIVE = 0
    ORDER BY ZUTYP_STATUS ASC


    Fetch the list of status codes for a given ZUTYP_STATUS

    SELECT ZUTYP_STATUSRANGE
    FROM UC_ZUTYP
    WHERE ZUTYP_STATUS = ?

    The JOBI that expands this into a full comma-delimited list has already been posted to another thread.

    More to come.




  • 13.  RE: Dependencies between UC4 systems

    Posted Nov 30, 2022 01:01 PM
    Edited by Michael A. Lowry Dec 01, 2022 06:30 AM
    The PoC has now evolved into a nearly feature-complete remake of the External Dependency feature, but for cross-system dependencies. I'm just working on a few remaining details, and I have a few questions:

    • When the mismatch option is set to wait, how often is the dependency checked? (In one test, it looked like it was every 20 seconds.)
      • When the option to execute an object on mismatch is also enabled, is this object executed every time the dependency is checked and fails to match?
    • There is only one field for object to execute upon mismatch and/or timeout. Is there in way to tell in the called task, whether the task was called due to a mismatch or timeout?
    • If the mismatch or timeout option is set to skip, the external dependency task ends with the status ENDED_SKIPPED. Successor tasks might be configured to act upon the basis of the predecessor's status. Is there really no way to force a task (e.g., a workflow) into the ENDED_SKIPPED status, to mimic the behavior of external dependency tasks?



  • 14.  RE: Dependencies between UC4 systems

    Posted Nov 18, 2022 06:03 PM
    Edited by Michael A. Lowry Nov 21, 2022 07:08 AM
    Duplicate post due to Higher Logic bug.


  • 15.  RE: Dependencies between UC4 systems

    Posted Nov 22, 2022 05:59 AM
    Edited by Michael A. Lowry Nov 22, 2022 05:59 AM
    I'm trying to find out how prevalent the same logical date option is in the external dependencies in our UC4 systems.
    I cannot find the field in the JPP table that stores this information. Does anyone know?

    Ping @Joel Wiesmann & @Philipp Elmer.
    ​​


  • 16.  RE: Dependencies between UC4 systems

    Posted Nov 22, 2022 08:07 AM
    Edited by Christian Boeck Nov 22, 2022 08:23 AM
    Hi Michael,
    IMHO - you can find the logical date only in the tables
    EH -> eh_ldate
    respectively
    AH -> ah_ldate

    Missunderstanding, you are looking for the checkbox column. I keep searching

    ------------------------------
    Thx & rgds
    Christian

    ------------------------------
    Managing Consultant
    Setis GmbH
    ------------------------------
    ------------------------------



  • 17.  RE: Dependencies between UC4 systems

    Posted Nov 22, 2022 08:55 AM
    Hi Michael,

    so now, I think you are looking for

    jpp_extslttype = 3

    table jpp

    ------------------------------
    Thx & rgds
    Christian

    ------------------------------
    Managing Consultant
    Setis GmbH
    ------------------------------
    ------------------------------



  • 18.  RE: Dependencies between UC4 systems

    Posted Nov 22, 2022 09:12 AM
    Edited by Michael A. Lowry Nov 26, 2022 09:06 AM
    Yes, that's it! This is not documented in the JPP Table schema documentation.




  • 19.  RE: Dependencies between UC4 systems

    Posted Nov 22, 2022 09:03 AM
    Edited by Michael A. Lowry Nov 23, 2022 04:55 AM
    To clarify, I'm looking for the field in the JPP table (or other table) that stores the status of the checkbox Check if the external task was activated with the same logical date as the workflow in the external dependency.

    Thanks to @Christian Boeck for answering the question. My current understanding is depicted below.




  • 20.  RE: Dependencies between UC4 systems

    Posted Nov 23, 2022 04:40 AM
    Edited by Michael A. Lowry Nov 26, 2022 09:04 AM
    Given how the controls work behind the scenes, I've reworked the PRPT a bit, combining the two timing-related controls into one. I also reworded the options to make them easier to understand.

    I'm not sure why the task timing options are implemented via two separate controls in the AE when a single radio button control is sufficient.

    Broadcom, feel free to use the above idea in a future product update. :)




  • 21.  RE: Dependencies between UC4 systems

    Posted Nov 24, 2022 03:42 AM
    Edited by Michael A. Lowry Nov 24, 2022 03:44 AM
    After looking more carefully at the documentation page External Dependency Tab, it became clear that implementing the remote dependency checker in way that is faithful to the current behavior of the external dependency feature will require more effort than originally anticipated.

    These bits in particular stood out in the documentation:

    Important! The system is designed to avoid overlapping situations. This means that an instance (an execution) of a task is used in only one instance (execution) of the Workflow in which it is referred to as external dependency.
    ...
    13:00 - The next execution of the Workflow starts but it must wait. Although the latest execution of the external task lies within the designated two hours, that execution has already been considered by the previous Workflow execution. In this case, the designated hour starts with the end of the latest instance (execution) of the Workflow, namely at 12:10.

    This means that the dependency checker will have to keep track of the run IDs of tasks that have previously matched, to avoid the situation wherein a single predecessor task fulfills the dependency checks for two or more instances of the same successor.

    We could track previously-fulfilled dependencies in a STATIC VARA.



  • 22.  RE: Dependencies between UC4 systems

    Posted Nov 25, 2022 06:10 AM
    Edited by Michael A. Lowry Nov 25, 2022 01:41 PM

    I'd like to know how the Automation Engine uniquely identifies external dependencies to ensure that subsequent instances of the same external dependency are not resolved by the same external task.

    I'm guessing it's something like the combination of the OH_Name of the workflow plus the EJPP_Lnr of the external dependency task.

    Can someone confirm this, or if it's not correct, let me know what the unique identifier is?




    • 23.  RE: Dependencies between UC4 systems

      Posted Nov 26, 2022 09:00 AM
      Edited by Michael A. Lowry Nov 26, 2022 09:03 AM

      In some quick tests, I found some evidence to support my hunch about the unique ID the AE uses for external dependency matching:

      U00011574 External dependency: 'EBM.DUMMY_TASK.SCRI(EBM.DUMMY_WORKFLOW.JOBP)' / Lnr: '00002' satisfied by Workflow RunID: '0434586903' / task RunID: '0434564643' / Lnr: '00002'.




    • 24.  RE: Dependencies between UC4 systems

      Posted Nov 28, 2022 05:06 PM
      Edited by Michael A. Lowry Nov 28, 2022 05:07 PM

      AFAIK, there is no predefined variable for the Lnr of a task, so I use this SQL query to look up the Lnr of a workflow task based on its run ID.

      SELECT EH_EJPP_Lnr FROM EH WHERE EH_AH_IDNR = ?


      I'm using this in an SQLI VARA in the process tab of the workflow.

      :SET &INPUT_RUN_ID# = &$RUNID#
      :PSET &STATUS_CHECKER_LNR# = GET_VAR(EBM.GET_WF_TASK_LNR.VARA_SEC_SQLI)

      (The SQLI VARA uses the script variable &INPUT_RUN_ID# in the bind parameter.)




    • 25.  RE: Dependencies between UC4 systems

      Posted Nov 29, 2022 01:29 AM
      Hi Michael

      I’m using this in the PreProcess of every JOBS and JOBP:
      :SET &RUNDID# = SYS_ACT_ME_NR()
      :SET &JOB_TYPE# = GET_STATISTIC_DETAIL(&RUNDID#,DST_HOST_TYPE,)
      :SET &TOP_JOBP# = &TOP_JOBP#
      :IF &TOP_JOBP# = ""
      :PSET &TOP_JOBP# = SYS_ACT_TOP_NAME()
      :ENDIF

      This is storing the absolute top level JOBP (regardless how many levels of sub workflows you have).
      It enables you to use the variable &TOP_JOBP# in every subsequent object (in our case for alarming.

      If you only need the runid of the parent workflow, just use:
      https://docs.automic.com/documentation/webhelp/english/AWA/11.2/AE/11.2/All%20Guides/Content/ucaagp.htm


      Viele Grüsse
      Peter




    • 26.  RE: Dependencies between UC4 systems

      Posted Dec 09, 2022 10:49 AM
      I'm continuing my work on developing a drop-in replacement for the External Dependency capability of the Automation Engine, that facilitates creating dependencies between separate AE systems. As a part of this, I'm trying to understand the various External Dependency options. I learned something that I thought others might be interested to know.

      There are effectively four different options for specifying the required timing of the target task:
      1. External task was activated with the same logical date as the workflow containing the external dependency.
      2. External task ended after the end of the last execution of the workflow containing the external dependency.
      3. External task ended within a certain margin (hhh:mm:ss) before the start of the workflow containing the external dependency.
      4. External task ended after the start of the workflow containing the external dependency.

      Option #3 is described in the documentation in a way that suggests a slightly different behavior than what I've observed in my tests.

      The External Dependency documentation includes this description and figure:​

      Select hh:mm:ss before this Workflow starts to define the period within which the external task must have ended.
      The period that you define here starts before the current execution of this Workflow starts.
      Graphic showing a Worlflow with an external dependency, the period defined as 60 seconds and the end time of the external task within this period.
      The wording and figure suggest that the external task must end within the 60 minute period immediately preceding the start of the workflow containing the external dependency.

      My tests revealed a slightly different behavior.
      • The external dependency is not resolved if the external task ends before the 60 minute period immediately preceding the start of the workflow containing the external dependency.
      • The external dependency is resolved if the external task ends within the 60 minute period immediately preceding the start of the workflow containing the external dependency.
      • The external dependency is also resolved if the external task ends after the 60 minute period - that is, after the start of the workflow containing the external dependency.
      I guess this is the intended behavior, because if the dependency could not be satisfied by a task that ended after the start of the workflow , one could end up in the situation wherein because the external task ended too late, no subsequent execution could ever resolve the dependency.

      I believe the documentation could be more clear though that with this option enabled, there is only an earliest accepted end time of the external task, and not a latest accepted end time.

      Ping @Gabi Oberreiter.



    • 27.  RE: Dependencies between UC4 systems

      Posted Dec 09, 2022 10:16 PM
      Edited by P Verghese Dec 09, 2022 10:16 PM
      Hi @Michael A. Lowry,

      I too have also seen the same behavior in my past tests on the 3rd point and was okay with the result. The condition is satisfied even if the job (Ex dep) is started after workflow that contains the ex. dep is started..   

      Guess the wording is bit misleading



      ​Regards
      P Verghese


    • 28.  RE: Dependencies between UC4 systems

      Posted Dec 10, 2022 01:00 AM
      Edited by Michael A. Lowry Feb 15, 2023 12:23 PM

      Exactly. The option should be labeled something like:

      after the point hhh:mm:ss prior to the start of this workflow




    • 29.  RE: Dependencies between UC4 systems

      Posted Dec 12, 2022 07:19 AM
      Isn't it dependent on what you have selected in the Else section?  So if it doesn't fall within the 60 minutes prior to the start of the workflow with the external dependency, then it can potentially Wait, Skip task or Cancel the workflow depending on your selection.


      From the external dependency documentation.

      "Important! If the external dependency end time does not lie within this period, the successor task in the Workflow is not processed and the settings in Else (on condition mismatch) are applied."



    • 30.  RE: Dependencies between UC4 systems

      Posted Dec 12, 2022 08:34 AM
      Edited by Michael A. Lowry Dec 12, 2022 08:40 AM

      @Jared Kessans, @Jared Kessans (Which one are you? Higher Logic lists two of you!)

      I did a quick test and confirmed that if the else (mismatch) action is set to something other than Wait, then upon check of the dependency, it will be counted as a mismatch if the external task ended after the start of the workflow containing the external dependency.

      The dependency check can happen after the start of the workflow, e.g., if the dependent task has predecessor tasks other than the external dependency. (The AE waits for all other other predecessors to complete before it begins checking external dependencies.)

      So, when the task timing margin option is used, the start time of the workflow containing the external dependency plays a role only if the else (mismatch) action is set to Skip task or Cancel the workflow. If the mismatch action is set to Wait, the AE will continue periodically checking the dependency and it can be fulfilled by a task that ended after the start of the workflow containing the external dependency.

      My test also revealed another quirk of the External Dependency feature.

      U00011590 The validity period for the external dependency 'EBM.DUMMY_TASK.SCRI' has been reduced from '000:05:00' to '000:01:15' due to the last workflow execution.

      ​​If the end of the previous run of the workflow containing the external dependency falls inside the margin, then the margin is automatically reduced to the gap between subsequent runs of the workflow. The reduced margin ("External task end") is depicted in this figure in the documentation.
      Graphic showing a Worlflow with an external dependency, the period defined as 60 seconds and the end time of the external task within the previous execution of this Workflow.
      The effective margin will always be the lower of these two durations:
      • the user-specified margin;
      • the gap between subsequent runs of the workflow.
      This means that the margin option implicitly also includes the after the last workflow execution end option.




    • 31.  RE: Dependencies between UC4 systems

      Posted Dec 12, 2022 08:50 AM
      Both were mine.  My company changed email domains about 10 years ago and recently made it so our old email addresses no longer work.  The Broadcom site does not make it possible to update your email address.  You have to create a new account and get everything set back up.  It was a pain.



      Also for the last scenario mentioned with the reduced margin of time, they have a graphic on the External Dependency link for the same.  



    • 32.  RE: Dependencies between UC4 systems

      Posted Feb 16, 2023 08:37 AM
      Edited by Michael A. Lowry Feb 16, 2023 08:53 AM
        |   view attached

      Here's the tool I developed.

      EXTDEP tool for defining dependencies between AE systems

      Attachment(s)

      zip
      EXTDEP_v1.0.zip   18 KB 1 version