The traditional way of passing information from OS jobs back to the Automation Engine has been to have the OS job print the values to the standard output, and then parse the job log using PREP_PROCESS_REPORT, either in the job post-process, or in the subsequent job.
Automation Engine version 11 introduces a useful new scripting statement, :REGISTER_VARIABLE, that provides an alternative to this approach. With :REGISTER_VARIABLE, it is no longer necessary to have the AE store and parse the whole job log, just to look for one piece of information. Instead, a :REGISTER_VARIABLE statement in the job's Process tab takes a specific piece of information from the job (usually from script variable), and passes it back to the Automation Engine, where it can be for example published to an object variable on the parent workflow, to be used by subsequent jobs in the workflow.
It works like this: During generation of the job JCL, each :REGISTER_VARIABLE statement in the job's Process tab is replaced by a line that runs the Agent's Job Messenger Daemon with a specific set of arguments. The JMD contacts the Automation Engine directly, and sets the variable.
So for example, this line in a UNIX (Bash) job's Process tab:
:REGISTER_VARIABLE MY_VAR1#,$my_var1
becomes this in the script that is sent to the agent:
/opt/uc4/agent/bin/ucxjlx6m JNR=0055002430 MNR=0100 PNR=22160 IPA=192.168.1.56 TYP=V VTYPE=T NAME=MY_VAR1# VALUE="$my_var1"
The Automic Documentation states that the :REGISTER_VARIABLE scripting statement can be used only inside a :BEGIN_EXT_INTERPRETER … :END_EXT_INTERPRETER block¹, but Philipp Elmer discovered that this is not actually a hard and fast requirement. If you use :REGISTER_VARIABLE inside an ordinary UNIX shell script or Windows BAT script, it will probably work fine even if it is not inside an external interpreter block.
What is clear is that if you wish to use :REGISTER_VARIABLE inside an external interpreter block, then you may have to customize the REGISTER_VARIABLE JOBIs so that they run the job messenger in the right way for each interpreter. In my examples, I set up an assortment of external interpreters (Perl, Python, Ruby, Bash, and Powershell), just to figure out how to run the job messenger correctly in each one.
Setting up external interpreters
External interpreters are external programs used by an agent to interpret (and execute) scripts stored in a JOBS object. You can define an arbitrary number of external interpreters in the UC_EXT_INTERPRETERS_WINDOWS and UC_EXT_INTERPRETERS_UNIX VARA objects in client 0. Below are the ones I defined.
UC_EXT_INTERPRETERS_UNIX
| Key | Value 1 | Value 2 | |
|---|
| PERL | .pl | /usr/bin/perl <FILE> | | | |
| PYTHON | .py | /usr/bin/python <FILE> |
| RUBY | .rb | /usr/bin/ruby <FILE> |
| BASH | .sh | /bin/bash <FILE> |
UC_EXT_INTERPRETERS_WINDOWS
| Key | Value 1 | Value 2 | |
|---|
| POWERSHELL | .ps1 | PowerShell -File <FILE> |
I then copied the REGISTER_VARIABLE.UNIX and REGISTER_VARIABLE.WINDOWS JOBI objects from client 0 to my main non-0 client, and customized them as follows:
REGISTER_VARIABLE.UNIX
:set &uc_register_varname = sys_last_registered_varname()
:set &uc_register_valuename = sys_last_registered_valuename()
:set &uc_register_otype# = sys_act_me_type()
:if &uc_register_otype# = 'JOBS'
: SWITCH &$EXT_INTERPRETER#
: CASE 'PERL'
system("&UC_JOBMD JNR=&UC_REALNR MNR=&UC_MANDANT PNR=&UC_IP_PORT IPA=&UC_IP_ADDR TYP=V VTYPE=T NAME=&uc_register_varname VALUE=\"&uc_register_valuename\"");
: CASE 'PYTHON'
from subprocess import call
call_args=['&UC_JOBMD', 'JNR=&UC_REALNR', 'MNR=&UC_MANDANT', 'PNR=&UC_IP_PORT', 'IPA=&UC_IP_ADDR', 'TYP=V', 'VTYPE=T', 'NAME=&uc_register_varname']
value_arg="VALUE="+&uc_register_valuename
call_args.append(value_arg)
call(call_args)
: CASE 'RUBY'
system_args="&UC_JOBMD JNR=&UC_REALNR MNR=&UC_MANDANT PNR=&UC_IP_PORT IPA=&UC_IP_ADDR TYP=V VTYPE=T NAME=&uc_register_varname VALUE="+&uc_register_valuename
system(system_args)
! No separate case for BASH, because the default approach below works fine.
: OTHER
&UC_JOBMD JNR=&UC_REALNR MNR=&UC_MANDANT PNR=&UC_IP_PORT IPA=&UC_IP_ADDR TYP=V VTYPE=T NAME=&uc_register_varname VALUE="&uc_register_valuename"
: ENDSWITCH
:endif
Note that depending on the interpreter type, the JMD command is run in slightly different ways: via Perl's system function, via Python's subprocess module, and via Ruby's system method.
REGISTER_VARIABLE.WINDOWS
:set &uc_register_varname = sys_last_registered_varname()
:set &uc_register_valuename = sys_last_registered_valuename()
:set &uc_register_otype# = sys_act_me_type()
:if &uc_register_otype# = 'JOBS'
: set &uc_windows_typ = get_att(win_typ)
: if &UC_WINDOWS_TYP = "BAT"
&UC_JOBMD JNR=&UC_REALNR MNR=&UC_MANDANT PNR=&UC_IP_PORT IPA=&UC_IP_ADR TYP=V VTYPE=T NAME=&uc_register_varname VALUE="&uc_register_valuename"
: endif
:endif
(The same JMD command appears to work fine in both Windows BAT and PowerShell scripts.)
Next, I created a workflow with five child jobs - one each for Bash, Perl, Python, Ruby, and PowerCenter. Here is the Perl job:
UC0.MAL.REGISTER_VARIABLE.PERL.JOBS
:BEGIN_EXT_INT PERL
:INCLUDE UC0.MAL.SAVE_INTERPRETER_SCRIPT.JOBI
my $username = $ENV{LOGNAME} || $ENV{USER} || getpwuid(<bq>lt;);
printf("User name : %s\n",$username);
:REGISTER_VARIABLE UNIX_PERL_USERNAME#,$username
:END_EXT_INT PERL
Note the :INCLUDE statement. The object UC0.MAL.SAVE_INTERPRETER_SCRIPT.JOBI contains interpreter-specific steps to optionally save a copy of the temporary interpreter script. (See the comment below more details about this JOBI.)
In the job's post-process tab, I added a single :PUBLISH statement. This statement takes the value set on the job with :REGISTER_VARIABLE, and publishes it to the parent workflow.
:PUBLISH &UNIX_PERL_USERNAME#,,WORKFLOW
And naturally, the variable is also defined (with no value) in the workflow's Variables & Prompts tab.
Each of the five jobs runs a command (in this case, a command to determine the name of the executing user), and assigns the result to a variable. It then registers the value of this variable in an AE variable, using the :REGISTER_VARIABLE statement. After all five jobs (Bash, Perl, Python, Ruby, and PowerShell) have completed, a successor AE script runs and prints the values of the five variables. I have attached an XML export of the workflow and its constituent jobs so you can experiment with them.
Using :REGISTER_VARIABLE has a few advantages over using PREP_PROCESS_REPORT:
- Once you have set it up, you can pass a variable from the OS job to the parent AE workflow with the addition of just a couple of statements:
- a
:REGISTER_VARIABLE statement in the process tab, and - a
:PUBLISH statement in the post-process.
- It is not necessary to store the job log in the AE DB.
- It's probably somewhat faster, because the AE does not need to parse the whole job log.
- Setting the variable in the AE is more reliable (e.g., if the log is too large to be stored in the AE DB).
Notes:
1. I do not know why :REGISTER_VARIABLE can be used only inside an external interpreter script block. Perhaps someone can answer this . (Update 2016.11.22: Philipp Elmer discovered that this is not actually a requirement.)
Updates:
2016.06.06 13:35 CEST: Added example for Python scripts.
2016.06.07 8:00 CEST: Added example for Ruby scripts.
2016.06.07 11:05 CEST: Factored-out file copy step to separate JOBI.
2016.11.22 10:30 CET: Added note based on finding by Philipp Elmer.