Automic Workload Automation

 View Only

Variables in the Automation Engine 

Jun 26, 2018 05:55 AM

This document describes variables in the Automation Engine. In the product and its documentation, the term variable is used somewhat ambiguously to refer to several very different things. So just to be clear, this document describes variables that take the form &VARIABLE# — the sort of variable that is used to store a temporary value in the context of an executing task. This document does not deal directly with VARA objects, which are something else entirely.

Variable types

There are several different variable types:

  • Script variables: Variables not defined in the Variables of an object, but set using the :SET, :RSET, or :PSET command. This includes also arrays defined using the :DEFINE script command. They may contain only text. These variable values cannot contain multiple lines. (Variables declared as arrays can contain multiple values.)
  • Object variables: Variables defined in the Variables of an object. They can contain multi-line text.
  • Prompt set variables: Variables defined in a Prompt Set of an object, or a :READ statement in the process tab. The type of data they can contain depends on the prompt set element type. Once a prompt set variable has been set, its value is read-only and can no longer be changed.

Script variables, object variables, and prompt set variables appear in purple in the JUI & AWI. E.g., &MyVariable#.

Read-only variables

There are two additional types of variable that are always read-only:

  • Custom attribute variables: Variables defined by custom attributes on the object. These values cannot be modified during task execution.
  • Predefined variables: Standard variables set by the Automation Engine and accessible to all tasks. Predefined variables fall into two categories:
    • System-level: These variables have the same value across the whole client or system, and do not depend on in which object they are accessed. Examples include &$SYSTEM#, &$CLIENT#, and &$PHYS_DATE_YYMMDD#.
    • Object-level: The values of these variables depend on the executable object which which they are accessed, and its current state. Examples include &$NAME#, &$RUNID#, and &HOST#.

In the JUI & AWI, custom attributes variables appear in purple, and predefined variables appear in blue.

Stetting variable values

Variables can be defined at different levels. The level at which a variable is defined determines where the values of these variables can be read. (The Inherit from parent option in workflows and schedules also affects variable inheritance.)

The values of object variables are set in the Variables tab of the corresponding object. These values can be overridden later using the :SET, :RSET, or :PSET command. The values of script variables are set using the same commands.

The level at which the variable value is set depends on the command used. In the example below, the command is run in the pre-process or process tab of UC0.JOB1. The diagram shows the level at which the variable value is set by each command.
hdg7lqj7rwo2.png

  • :SET — The variable is accessible only from the pre-process and process tab of UC0.JOB1.
  • :RSET — The variable is accessible from the whole job UC0.JOB1, including the post-process tab.
  • :PSET — The variable is accessible from the whole workflow UC0.WF1, including any other child tasks that run after UC0.JOB1 has completed (contingent upon inheritance settings).

With nested workflows — when a workflow is running as a child task in a parent workflow — the :PUBLISH scripting command can be used to set a variable at the level of the topmost workflow.

Order of precedence in variable setting

Variables are set in this order, first-to last:
1. In the executable object's Variables & Prompts tab
2. In the parent workflow, if the object is set to inherit variable values from its parent.
3. In the Variables & Prompts tab of the task Properties where the object appears in a workflow.
4. In the job’s pre-process or process scripting tabs, via the :SET, :RSET, or :PSET commands.

Overriding variables in workflow task properties

The first time an executable object is added to a workflow, if that executable object has any variables defined in its Variables & Prompts tab, these are copied to the new task in the workflow, along with their values in the object. If these variables have different values set in the Variables & Prompts tab of the workflow (and the child object is set to inherit variable values from its parent), then whether the child task inherits the parent workflow’s values or uses its own values depends on whether the value in the task properties has ever been modified. There is no way to see in the JUI or AWI whether the value has ever been modified. However, if you export the workflow to XML, you can see this information in the Mod attribute.
<VALUE state="2">
<Values>
<row ERTUsage="0" Mod="0" Name="&amp;ENV#" Value="DEV"/>
<row ERTUsage="0" Mod="1" Name="&amp;TYPE#" Value="LOCAL"/>
</Values>
<Mode>0</Mode>
</VALUE>


In this workflow task:

  • The &ENV# variable has never been modified in the task properties. It will inherit the value from the parent workflow.
  • The &TYPE# variable has been modified in the task properties. It will use the value set in the task properties.

Reading variable values

Variable values can be read directly in scripting statements by referring to the variable by name, with its leading ampersand.

:SET &var1# = "Hello, world."
:PRINT &var1#

This statement prints the current value of the variable &var1#.

U00020408 Hello, world.

It is not possible to read the value of a variable that has not been set.

:SET &var1# = "Hello, world."
:PRINT &var2#

Attempting to do this results in a syntax error.

U01001308 Variable '&var2#' has not yet been defined.

If you want to be able to avoid such errors, and read the value of a variable whether or not it has already been set, you can accomplish this by inserting a statement that sets the variable to its own current value.

:SET &var1# = "Hello, world."
:SET &var2# = &var2#
:PRINT &var2#

Because the &var2# variable has not been set to anything, this prints a blank U00020408 message:

U00020408

Blank space characters & variable comparison

The Automation Engine stores trailing (and leading) blanks in variable values. However, when performing string comparisons, the AE implicitly removes trailing (but not leading) blanks. For example, the strings "" and " " are treated as equivalent in string comparisons, as as well as the strings "x" and "x    ". If you need to distinguish between two strings that differ only in their number of trailing blanks, you have two options:

  1. Get the length of the strings using the STR_LENGTH script function.
  2. Convert the strings to their ASCII hexadecimal equivalents using the HEX script function.

Variable name termination

In the Automation Engine documentation, virtually all variable names mentioned in examples are terminated with a hash symbol (#). E.g., &CLIENT# or &HOST#. Pre-defined variables for system and object attributes are also all terminated with a # symbol. E.g., &$SYSTEM# or &$RUNID#.

At Automic, and among customers too, it is standard practice to terminate AE variable names with a single # character. This is not a hard and fast requirement, but it is a recommended convention. Why? A basic limitation of the Automation Engine: variable names may not be sub-strings of other variable names.

This means that if you define the variable called &APP, you cannot also define a variable called &APP_ID. This is because the former is a substring of the latter. If you try to save an object wherein two such variables are defined, message U01001312 will appear:

U01001312 Variable name 'APP_ID' is not allowed. A variable with the name 'APP' already exists.

Terminating the variable names with a symbol such as # gets around this limitation. A termination character at the end of variable names ensures that the names do not overlap¹. E.g., &APP# is not a substring of &APP_ID#. If you do not terminate variable names, it’s just more work to name ensure that names do not overlap. Terminating variable names makes scripting easier, because you do not have to work to prevent variable name overlaps; you must simply ensure name uniqueness.

Any character that is allowed in variable names² may be used as a termination character. It is merely convention to use the # character. For this convention to be most effective, you must follow two rules:

  • Use the same termination character consistently, in all variable names. Again, it is a common convention to terminate variable names with the # character.
  • Do not use the termination character in the middle of variable names. Use it only at the end.

Notes:
1. If you examine the AE scripts used internally in the Automation Engine, you will find many variables names that are not terminated. For example, here is an excerpt from the UNIX header include object from client 0 (HEADER.UNIX):

:set &UC_EXVER = get_var(UC_EX_VERSION)
:set &UC_MANDANT = SYS_ACT_CLIENT
:set &UC_REALNR = SYS_ACT_RESTART_ME_NR
:set &UC_NAME = SYS_ACT_JOBNAME
:set &UC_JOBMD = get_var(UC_EX_JOB_MD)
:set &UC_IP_ADDR = get_var(UC_EX_IP_ADDR)
:set &UC_IP_PORT = get_var(UC_EX_IP_PORT)
:set &UC_UNIX_SHELL = get_att(UNIX_SHELL)

Notice that although these variable names are not terminated, none of them is a substring of any other.
2. The characters allowed in variable names are A-Z, a-z, 0-9, _, $, @, §, and #. Variable names may not begin with a number or a $ symbol.

Indirect reference

It is possible to set or read the value of an AE script variable by indirect reference. That is, the name of the variable can itself be constructed using a variable or variables, and then the value of the referenced variable can be set or read.

Choosing variable names at runtime

Note that even though the name of the variable whose value should be read can be constructed dynamically, the variable must already have been set explicitly. That is,  :SET_SCRIPT_VAR is not able to set the values of previously undefined variables. An enhancement request has been opened for this. (See :SET_SCRIPT_VAR should be able to set the values of previously undefined variables.) A work-around is also available for v12.1 and later. (See GENERATE_SCRIPT_VARS — generate script variables on-the-fly.)

Nested variables

Here’s an example of how to use indirect reference to choose dynamically the name of the variable whose value should be read.

:SET &Sys_Name_EXP#   = "AE_EXP"
:SET &Sys_Name_DEV#   = "AE_DEV"
:SET &Sys_Name_TEST#  = "AE_TEST"
:SET &Sys_Name_PROD#  = "AE_PROD"
:SET &Env# = "EXP"
:SET &Sys_Name_var# = "Sys_Name_&Env#"
:SET &Sys_name# = GET_SCRIPT_VAR("&&&Sys_Name_var##")
:PRINT "System: &Sys_name#"

 

Line 6 sets &Sys_name_var# to the name of the variable containing the name of the system.
Line 7 sets &Sys_name# to the value of the variable whose name is stored in the variable &Sys_name_var#.

The result looks like this:

U00020408 System: AE_EXP

This is how the process works:

  1. First round of variable resolution:
    • This escaped ampersand (&&) becomes a single ampersand.
    • The &Sys_name_var# variable is resolved to the value &Sys_name_EXP#.
  2. Second round of variable resolution:
    • The variable &Sys_name_EXP# is resolved to the value AE_EXP.
    • There are no remaining unescaped ampersands, so the value of the &Sys_name# variable is set to AE_EXP.

You can achieve the same result in another way using the :RESOLVE_VAR function or the :RESOLVE command.

:SET &Sys_Name_EXP#   = "AE_EXP"
:SET &Sys_Name_DEV#   = "AE_DEV"
:SET &Sys_Name_TEST#  = "AE_TEST"
:SET &Sys_Name_PROD#  = "AE_PROD"
:SET &Env# = "EXP"
:RESOLVE &Sys_name# = "&&Sys_Name_&Env##"
:PRINT "System: &Sys_name#"

 

It works in a similar multi-step way:

  • First round of variable resolution:
    • This escaped ampersand (&&) becomes a single ampersand.
    • The &Env# variable is resolved to the value EXP.
  • Second round of variable resolution
    • The variable &Sys_name_EXP# is resolved to the value AE_EXP.
    • There are no remaining unescaped ampersands, so the value of the &Sys_name# variable is set to AE_EXP.

The trick to using these approaches is getting the number of leading ampersands right.

How variable resolution works

As you can see, the :RESOLVE command is capable of performing multiple rounds of variable resolution. It appears to work like this:

  1. Resolve escaped ampersands
    Replace any escaped ampersand (&&) with a single ampersand.
  2. Resolve variables
    For any string beginning with an ampersand (&), and followed by at least one character that can be part of a variable name, and at least one character that cannot be part of a variable name (such as a space, or a single or double quotation mark), attempt to resolve this variable. Undefined variables cause a runtime error. Variables with no value resolve to a single blank space (' ').
    Note: VARA object references of the form {VARA,KEY,COLUMN} are also resolved by the :RESOLVE command.

These two steps are repeated until no escaped ampersands and no unresolved variables remain.

Depth of resolution

Each :RESOLVE command is capable of resolving up to two levels of nesting. If you need to resolve three of more levels of nested variables, you can accomplish this by adding additional :RESOLVE commands that resolve the contents of the variable again. Consider the following example with three levels of nesting:

:SET &Env# = "EXP"
:SET &SysName_EXP# = "AE_EXP"
:SET &CompanyName_AE_EXP# = "MyCompany_AE_EXP"
:RESOLVE &System# = "&&&&CompanyName_&&SysName_&Env###"
:RESOLVE &System# = "&System#"
:PRINT "System: &System#"

The first :RESOLVE command resolves the inner two variables, &Env# and then &SysName_EXP#. The second :RESOLVE command then resolves the outermost variable &CompanyName_AE_EXP#. The result looks like this:

U00020408 System: MyCompany_AE_EXP

Variables in VARA objects

When VARA objects are used in executable objects, the parent executable object may set or inherit variables. Whether these variables are resolved when they appear in the context of the VARA object depends on the type of VARA.
  • STATIC & XML: Variables are not resolved in keys or values.
  • SQL: Variables are resolved in the Connection and Login fields. Variables are not resolved in the SQL statement
  • SEC_SQL: Variables are resolved in the Connection and Login fields, as well as in bind parameters. Variables are not resolved in the SQL statement
  • SQLI: Variables are not resolved in the SQL statement
  • SEC_SQLI: Variables are resolved in the bind parameters. Variables are not resolved in the SQL statement
  • EXEC: Variables are resolved in parameters. However, variables set in the parent object are not implicitly inherited by the child executable object. If you want to pass variables to the child object, you must do so explicitly via parameters, and use the :READ command in the child object to read these values.
  • FILELIST, BACKEND, & MULTI: Variables are resolved.

Tip:
To combine the convenience of a SQL VARA with the power & flexibility of an SQL job,
create an EXEC VARA wrapper for an SQL job.

References

Statistics
1 Favorited
96 Views
1 Files
0 Shares
2 Downloads
Attachment(s)
zip file
AWI_Icons.zip   199 KB   1 version
Uploaded - Nov 19, 2019

Tags and Keywords

Comments

Jul 20, 2020 10:34 AM

Broadcom has updated its documentation to include some of the information contained in this document. E.g., Variable names.

Nov 20, 2019 07:37 AM

I added sections on two sub-topics:
  • Order of precedence in variable setting
  • Variables in VARA objects

Jul 23, 2018 02:53 AM

The most straightforward approach is probably something like this:

  1. write the output to the job log in a way that makes it easy to parse using PREP_PROCESS_REPORT.
  2. in the post-process tab of the job, write AE scripting that parses the job output and stores the important bit in AE variables.

Jul 20, 2018 05:01 PM

Appreciate the response Michael_Lowry. Basically, I am making a series of webservice calls; partial output from one call being sent to the next call and so on. I am using this approach instead of SOAP jobs because I can re-purpose a good library of Python code that we already have. But that discussion aside, I am trying to print SOAP responses in Python, scrape the job report from post-process, and see if I can save necessary data to variables. That seems doable.

Jul 20, 2018 04:10 PM

The AE is limited in the types of data it can store in object/script variables. :REGISTER_VARIABLE imposes further limitations. (No arrays, as far as I know.)

 

If you can convert the data to XML or something similar, that might be a good intermediate step. I’m not sure though what to do next. The AE has an XML VARA object type, but I’m not sure what the best way is to get data into such an object.

 

I guess the important question is: what do you hope to do with the data, once they are somehow accessible to the AE?

Jul 20, 2018 11:28 AM

Michael_Lowry. Wonderful article on AE variables! Armed with this, and the information from :REGISTER_VARIABLE — practical examples with Bash, Perl, Python, Ruby, and PowerShell , I was attempting to return a complex datatype, a list or dict from Python back to Automic. It appears this is not easily done. Any suggestions?

Related Entries and Links

No Related Resource entered.