Automic Workload Automation

 View Only
Expand all | Collapse all

Transport case file format

  • 1.  Transport case file format

    Posted Jan 23, 2017 07:37 PM

    Update 2018.04.17: For the most up-to-date version of this documentation, see: Transport case file format.


    For the benefit of anyone else who might be interested, here is my documentation of the transport case file format.  I based this documentation on transport case files produced by Automation Engine v11.2. This information is provided as-is, and is based on a lot of educated guesswork. Rely on this information at your own risk! B)

     

    If you can fill in any missing information, please comment below.


     

    Overall structure

    The transport case file format consists of many records. Each record occupies a single line in the transport case file. The record type is indicated by  the first character of the line. There are seven different record types. Lines beginning with a semicolon (;) are comments, and are ignored.

    Record types

    Record type

    Description

    V

    File & version information

    T

    Table name

    C

    Table column definition

    F

    Table field data

    O

    Object home folder path (or link path)

    R

    End of group of data fields

    S

    End of file

    ;

    Comment

    Broadly speaking, a transport case file consists of two main parts: table descriptions at the top, and table content at the bottom.

    Table descriptions

    Thetable descriptions section contains a complete description of the structure ofall of the tables in the AE schema that pertain to AE objects. Each table description consists of the table name (a T record) followed by descriptions of the columns in the table (C records).

    ;Table OCA was exported on: 2017-01-20 15:40:36
    TOCA
    C001OCA_OH_Idnr       300004
    C002OCA_Name          700032
    C003OCA_Value         700200

    Table content

    After the table descriptions comes the actual content of the tables. Thetable contentsection contains the actual data for the AE objects in the transport case. Only tables containing information relevant to the objects in the transport case are included. For each table, the table content consists of the table name (a T record) followed by the contents of the fields in the table (F records). Groups of fields are terminated by a single R record.

    TJPP
    F001+0001463068
    F002+00001
    F004CUC0.DEV.TEST#1_1.JOBP
    F006C001135
    F016001
    F019+0000000001
    F024C000000
    F035+00001
    R

    For each object described in the transport case, the table data section begins with data for the OH table, followed by any O records, and then data for other tables related to the object.

    The V record

    All transport case files begin with a single V record. This record encodes basic information about the file and the system whence it originates.

    V08 11.2      TRANSPORT            OH                 0000000056 018

    Fields of record type V

    Description

    Position

    Length

    Example

    Record type

    1

    1

    V

    File version

    2

    4

    08

    System version

    5

    10

    11.2

    File type

    15

    21

    TRANSPORT

    Main table

    35

    20

    OH

    Number of objects

    55

    10

    56

    Unknown

    66

    ?

    018

    T records

    T records indicate the table to which a subsequent group of records pertains.

    TABLOB

    Fields of record type T

    Description

    Position

    Length

    Example

    Record type

    1

    1

    T

    Table name

    2

    ?

    ABLOB

    C records

    Crecords describe the columns in a particular table.

     

    C001ABLOB_AH_Idnr     300004

     

    Fields of record type C

    Description

    Position

    Length

    Example

    Record type

    1

    1

    C

    Column number

    2

    3

    001

    Column name

    5

    15

    ABLOB_AH_Idnr

    Column data type

    23

    1

    3

    Column data length

    24

    5

    00004

    The column numbers correspond to the order of columns in theDB schema documentation.

    There are several column data types.

    Column data types

    Data type

    Description

    A

    Memo, CLOB

    B

    Memo, ?

    C

    Long Integer

    D

    XML

    0

    Memo, binary

    1

    Byte

    2

    Integer

    3

    Long Integer

    4

    Long Integer, counter

    5

    Date/Time

    6

    Unknown (not yet encountered)

    7

    Text

    8

    Memo, ?

    9

    Byte

    For the Memo data types, there is usually more information about the data type in the Description field of the DB schema documentation for the corresponding table.

    F records

    Frecords contain the data for each field in a particular table.

    F004CUC0.DEV.TEST#1_1.JOBP

    Fields of record type F

    Description

    Position

    Length

    Example

    Record type

    1

    1

    F

    Field number

    2

    3

    004

    Field data type

    5

    1

    C

    Field data

    6

    1

    UC0.DEV.TEST#1_1.JOBP

    The field numbers correspond to the column numbers of the corresponding tables, and the order of columns in the DB schema documentation.

    There are several field data types.

    Field data types

    Data type

    Description

    -

    Integer

    +

    Integer

    0

    Byte

    2

    Date/Time

    C

    String

    M

    String with byte count. Can contain multiple parts.

    There may be additional types too. The date/time values in F***2 fields begin with the numeral 2 in byte 5.

    Some fields of type C can contain multiple labeled parts. E.g., JBA_Restis used to store additionalattributes for z/OS jobs.

    F records (data type M)

    F records with data type M have an additional data length field that indicates the length of the data to follow.

    F005M000000010key1value1

    Fields of record type F, data type M

    Description

    Position

    Length

    Example

    Record type

    1

    1

    F

    Field number

    2

    3

    005

    Field data type

    5

    1

    M

    Field data length

    6

    9

    000000010

    Field data

    15

    variable (10 in this case)

    key1value1

    M fields often contain multiple parts separated by ^K (control-K) characters. These fields sometimes include embedded XML.

    O records

    Orecords encode folder paths of objects.

    \APPS{}\UC0{}\TESTS{}\TEST1{This is a folder title}

    Fields of record type O

    Description

    Position

    Length

    Example

    Record type

    1

    1

    O

    Folder path

    2

    ?

    \APPS{}\UC0{}\TESTS{}\TEST1{This is a folder title}

    O records always appear right after the R record concluding the F records of the OH table. If there is no O record for an object, this means the object is in <No Folder>. If there is one O record, this record contains the home folder path of the object. If there is more than one O record, the second, third, and so on represent links to the object. Folder titles appear within curly braces ({,}) after each level in the folder hierarchy.

    The S record

    The transport case file is terminated by a single S record.

    S END

    Original post

    I am working on a general purpose script for reading arbitrary information from a transport case export file. The idea is to make it easy to list the details of objects that meet certain criteria. The new script is based on the approach first suggested by Automic back in March 2014. There are still a few things I haven't figured out. Does anyone have any documentation of the transport case file format?



  • 2.  Transport case file format

    Posted Jan 24, 2017 01:29 AM

    Hi Michael,

    as this is an internal format there is no documentation for the parameters. Only the XML file format and the parameters/tags are documented. So the content might change without informing someone ...

    regards,
    Peter



  • 3.  Transport case file format

    Posted Jan 24, 2017 07:35 AM
    Peter Grundler wrote:
    As this is an internal format there is no documentation for the parameters. Only the XML file format and the parameters/tags are documented. So the content might change without informing someone.
    Thanks for your reply, Peter. This is the answer Automic has usually provided when we have asked for more information about the transport case file format. Based on this, we decided a couple of years ago to begin moving toward using the transport case less, and XML more. We were just beginning to rely on the XML file format when version 12 was released. To our surprise, the description of the XML file format was omitted from the v12 documentation. The most recent version that is available is v11.2.

    Do you know when the documentation of the XML file format will be updated for version 12? Moreover, can we rely on Automic to continue to provide updated documentation of the XML file format in the future? It is important for us to know whether Automic is committed to providing complete and up-to-date documentation. Thanks in advance if you can help to shed light on this topic.


  • 4.  Transport case file format

    Posted Jan 24, 2017 08:37 AM
    Hi Michael,
    Please let me discuss this with our documentation team.
    I will keep you informed via this post.

    Kind regards,
    Antoine


  • 5.  Transport case file format

    Posted Jan 25, 2017 01:17 AM

    Hi Michael,

    itmight be the case that the XML format will no longer be published in the documentation. I remember a discussion with development last year where I was told that the content of the XML file (tags) has changed and will change in the future. I'm not talking about new features - it's regarding existing information.

     

    As some of the important tags (i.e. SYSTEM and IDNR) have already disappeared from the export file without information - which was an issue for at least one customer - I'm afraid what will happen in upcomung versions.

    But let's wait for Antoine's findings ...

    regards,
    Peter



  • 6.  Transport case file format

    Posted Jan 25, 2017 04:34 AM
    Yes, I noticed that the system and client attributes were removed from the object tags. I presume this is because they were never really used, and only caused problems when importing to a different system/client.


  • 7.  Transport case file format

    Posted Jan 25, 2017 09:10 AM
    I have updated my original post with the current version of my documentation of the transport case file format. I worked out these details during the course of solving problems that required extracting information from transport files prior to loading them. (Automic provides no tool for listing information in a transport case file.) This documentation is certainly incomplete, and I am sure that some of the assumptions I have made are wrong. Feel free to provide corrections if you have better information.


  • 8.  Transport case file format

    Posted Jan 27, 2017 08:14 AM
    Here’s the script I’ve been working on, parse_transport_case.sh.

    The script reads a transport case file and prints information about the records, tables, fields, and objects found in the file. Change the value of the verbosity variable at the top of the script to get different amounts of information.


     
     


     
     


     
     


     
     


     
     


     
     

    verbosityDescription
    0Print only the tab-delimited output defined in the printOutput function.
    1Print basic details of each object.
    2Print details of each record.
    3Print complete records.
    4Print debugging information.

    In this version of the script, I have included sections that capture and process information from the JBA and OCA tables. Specifically, with verbosity=0, the script will output:
    • The type, name, and folder path of each AE object;
    • The job name and JBA_Rest field of all z/OS jobs; and
    • The workflow name of all workflows that have a particular custom attribute set to specific value.
    To customize the script for your use, you will probably need to add sections to capture or process additional fields of the OH table, or data for tables other than just JBA and OCA. I have done my best to thoroughly comment the script so that you can see how it works.

    Our batch promotion/deployment system is quite complicated, and is integrated with several other applications and systems. During deployment of an AE batch, we must extract information about the objects in a transport case file for several reasons:
    1. List all objects, to ensure that object names adhere to the naming convention;
    2. List all home folder paths, to ensure that objects have not been moved to invalid locations;
    3. List options pertinent to standards & conventions (Generate at runtimeoption, event interval,etc.);
    4. List z/OS job options & attributes, to validate adherence to standards & conventions;
    5. List specific JOBP objects, for integration with an application that allows dynamic execution of these workflows; and
    6. List all JSCH objects so that stop/start/restart actions can be performed.
    Currently, we use dedicated scripts for some of these steps. My goal is to replace all of these with a single script, so that we need to read the file just once. So far, the script is configured to perform all steps except #3. The tab-delimited output created by the script (with verbosity=0) will be used by later steps in the deployment.

    Example command:
    ./parse_transport_case.sh ./UC0_AUTOMATED_TESTS-TEST1.txt | sort
    Output:
    AE_OBJ  DOCU    UC0.DEV.TEST#1_1.DOCU   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  EVNT    UC0.DEV.TEST#1_1.EVNT_CONS1     /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  EVNT    UC0.DEV.TEST#1_1.EVNT_DB        /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  EVNT    UC0.DEV.TEST#1_1.EVNT_FILE      /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  EVNT    UC0.DEV.TEST#1_1.PCE_EVNT_FILE  /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  FILTER  UC0.DEV.TEST#1_1.FILTER /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBF    UC0.DEV.TEST#1_1.JOBF   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBI    UC0.DEV.TEST#1_1.JOBI   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBI    UC0.DEV.TEST#1_POSTPROC.JOBI    /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBI    UC0.DEV.TEST#1_PREPROC.JOBI     /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBI    UC0.DEV.TEST#1_PROCESS.JOBI     /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBP    UC0.DEV.TEST#1_1.JOBP   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBP    UC0.DEV.TEST#1_1.JOBP_COPY      /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBP    UC0.DEV.TEST#1_1.JOBP_FOREACH   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBP    UC0.DEV.TEST#1_1.JOBP_IF        /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_BOBJ_DEF  /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_BOBJ_PUB  /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_FTP       /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_JMS_RECV  /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_JMS_SEND  /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_PCE_DOM1  /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_SQL       /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_UNIX      /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WEBSERVICE_REST   /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WEBSERVICE_SOAP   /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WIN       /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WIN_PS    /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WS_REST   /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_WS_SOAP   /APPS/UC0/AUTOMATED_TESTS/TEST1/RA_JOBS
    AE_OBJ  JOBS    UC0.DEV.TEST#1_1.JOBS_ZOS       /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  JSCH    UC0.DEV.TEST#1_1.JSCH   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  PRPT    UC0.DEV.TEST#1_1.PRPT   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  SCRI    UC0.DEV.TEST#1_1.SCRI   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  SYNC    UC0.DEV.TEST#1_1.SYNC   /APPS/UC0/AUTOMATED_TESTS/TEST1
    AE_OBJ  VARA    UC0.DEV.TEST#1_1.VARA_STATIC    /APPS/UC0/AUTOMATED_TESTS/TEST1
    JOBP_XA2        UC0.DEV.TEST#1_1.JOBP
    JOBS_ZOS        UC0.DEV.TEST#1_1.JOBS_ZOS       J=''K='6'2='8'6='UC4DEMO'P='MICHAEL'O='1,1'7='THE ATTRIBUTES ON THIS TAB ARE HANDLED AUTOMATICALLY ON THE PRE-PROCESS TAB!'Q=''a="'UC0/APPL'"U='1'S='*ALL'T='0'V='1'R='H'W='H'
    JOBS_ZOS        UC0.DEV.TEST#1_2.JOBS_ZOS       J=''K='6'2='9'6='&JOBNAM#'P='&USERNAME#'O='1,1'7=''Q='OSRZT.UC4.PROD.JCLLIB(UC0&JOBNAM#&ENV_ABBREV#)'a="'D08/APPL'"U='1'S='*ALL'T='0'V='0'R='H'W='H'
    JOBS_ZOS        UC0.DEV.TEST#1_3.JOBS_ZOS       J=''K='6'2='A'6='&JOBNAM#'P='&USERNAME#'O='1,1'7=''Q='OSRZT.UC4.PROD.JCLLIB(UC4TEST1)'a="'D08/APPL'"U='1'S='*ALL'T='0'V='0'R='H'W='H'
    JOBS_ZOS        UC0.DEV.TEST#1_4.JOBS_ZOS       J=''K='6'2='A'6='&JOBNAM#'P='&USERNAME#'O='1,1'7=''Q='OSRZT.UC4.PROD.JCLLIB(UC4TEST1)'a="'D08/APPL'"U='1'S='*ALL'T='0'V='0'R='H'W='H'
    I hope this script is helpful. I’m sure it can be improved. If you make improvements, please share them here! You can find the script here.


  • 9.  Transport case file format

    Posted Jan 27, 2017 09:21 AM
    Back in September, I figured out the structure of the JBA_Rest field for z/OS jobs. I wrote a script to print out the options in human-readable format. Now that I have a general-purpose script for extracting data from a transport case file, I moved the JBA_Rest-parsing algorithm to a dedicated script, parse_JBA_Rest.sh. This new script takes as input the output generated (with verbosity=0) from the parse_transport_case.sh script above. Right now, it is able to parse the JBA_Rest field only of z/OS jobs. If someone wants to add support for SQL, SAP, or RA jobs, please be my guest.

    Example command:
    ./parse_JBA_Rest.sh ./parse_tc_out1.txt
    Output:
    --------------------------------------------------------------------------------
    UC0.DEV.TEST#1_1.JOBS_ZOS
    1  J : Notify
    2  K : Message class                6
    3  2 : Job type                     8 (Automation Engine (AE))
    4  6 : Job name                     UC4DEMO
    5  P : Programmer name              MICHAEL
    6  O : Message level                1,1
    7  7 : Job parameters               THE ATTRIBUTES ON THIS TAB ARE HANDLED AUTOMATICALLY ON THE PRE-PROCESS TAB!
    8  Q : z/OS file name
    9  a : Account                      'UC0/APPL'
    10 U : With non-JES output          1 (yes)
    11 S : Obtain message class(es)     *ALL
    12 T : Purge                        0 (no)
    13 V : Release                      1 (yes)
    14 R : Route to message class(es)   H
    15 W : Return code                  H (highest)
    --------------------------------------------------------------------------------
    UC0.DEV.TEST#1_2.JOBS_ZOS
    1  J : Notify
    2  K : Message class                6
    3  2 : Job type                     9 (z/OS JCL)
    4  6 : Job name                     &JOBNAM#
    5  P : Programmer name              &USERNAME#
    6  O : Message level                1,1
    7  7 : Job parameters
    8  Q : z/OS file name               OSRZT.UC4.PROD.JCLLIB(UC0&JOBNAM#&ENV_ABBREV#)
    9  a : Account                      'D08/APPL'
    10 U : With non-JES output          1 (yes)
    11 S : Obtain message class(es)     *ALL
    12 T : Purge                        0 (no)
    13 V : Release                      0 (no)
    14 R : Route to message class(es)   H
    15 W : Return code                  H (highest)
    --------------------------------------------------------------------------------
    UC0.DEV.TEST#1_3.JOBS_ZOS
    1  J : Notify
    2  K : Message class                6
    3  2 : Job type                     A (JCL incl. z/OS job card)
    4  6 : Job name                     &JOBNAM#
    5  P : Programmer name              &USERNAME#
    6  O : Message level                1,1
    7  7 : Job parameters
    8  Q : z/OS file name               OSRZT.UC4.PROD.JCLLIB(UC4TEST1)
    9  a : Account                      'D08/APPL'
    10 U : With non-JES output          1 (yes)
    11 S : Obtain message class(es)     *ALL
    12 T : Purge                        0 (no)
    13 V : Release                      0 (no)
    14 R : Route to message class(es)   H
    15 W : Return code                  H (highest)
    --------------------------------------------------------------------------------
    UC0.DEV.TEST#1_4.JOBS_ZOS
    1  J : Notify
    2  K : Message class                6
    3  2 : Job type                     A (JCL incl. z/OS job card)
    4  6 : Job name                     &JOBNAM#
    5  P : Programmer name              &USERNAME#
    6  O : Message level                1,1
    7  7 : Job parameters
    8  Q : z/OS file name               OSRZT.UC4.PROD.JCLLIB(UC4TEST1)
    9  a : Account                      'D08/APPL'
    10 U : With non-JES output          1 (yes)
    11 S : Obtain message class(es)     *ALL
    12 T : Purge                        0 (no)
    13 V : Release                      0 (no)
    14 R : Route to message class(es)   H
    15 W : Return code                  H (highest)
    --------------------------------------------------------------------------------

    You can find the script here.


  • 10.  Transport case file format

    Posted Jan 31, 2017 03:30 AM
    Hi Michael_Lowry, when working with the REST field, you might find the table UC_SVALM interesting.
    It maps attribute names to the Rest-IDs (and DB fields)
    This makes reverse-engineering much easier :-)

    Example: 
    select SVALM_Attname, SVALM_AttDBFname, SVALM_RestID from UC_SVALM where SVALM_Otype = 'JOBS' and SVALM_HostAttrType = 'MVS';


  • 11.  Transport case file format

    Posted Jan 31, 2017 03:32 AM
    Philipp Elmer wrote:
    [W]hen working with the REST field, you might find the table UC_SVALM interesting.
    It maps attribute names to the Rest-IDs (and DB fields)
    This makes reverse-engineering much easier :-)
    Wow, that’s great. Thanks! I wish I had known that a year ago. :)


  • 12.  Transport case file format

    Posted Jan 31, 2017 04:59 AM
    PhilippElmer, do you know how to list the corresponding XML key names too?


  • 13.  Transport case file format

    Posted Jan 31, 2017 05:23 AM
    Hi,
    unfortunately not.
    Last information I have regarding the XMl format is: It's not documented and there is no XSD (or XML-DTD) because it is NOT an official interface.
    That's at least what a Senior Manager of Automic Engineering told me three weeks ago in Vienna.


  • 14.  Transport case file format

    Posted Jan 31, 2017 05:52 AM
    Philipp Elmer wrote:
    [U]nfortunately not. Last information I have regarding the XMl format is: It's not documented and there is no XSD (or XML-DTD) because it is NOT an official interface. That's at least what a Senior Manager of Automic Engineering told me three weeks ago in Vienna.
    I wonder if the XML parsing during import is done by uc4.jar or by the AE.


  • 15.  Transport case file format

    Posted Feb 10, 2017 12:32 PM
    Antoine Sauteron said:
    Hi Michael,
    Please let me discuss this with our documentation team.
    I will keep you informed via this post.

    Kind regards,
    Antoine
    Any news on this question, Antoine_Sauteron?


  • 16.  Transport case file format

    Posted Feb 13, 2017 02:06 AM
    Hi Michael_Lowry

    I did not get any answer so far, but I will ask again.

    Best regards,
    Antoine


  • 17.  Transport case file format

    Posted Feb 15, 2017 04:42 AM
    Hi Michael_Lowry

    Here's the answer I got from the head of development about the XML format documentation:
    This section was removed in V12 by intention due to following reasons: 

    1) The XML format is intended to be used for Export/Import Feature only. 
    2) We reserve the right to change the format and the notation at any time. 

    Should you need to know the format of the XML's in a dedicated version then you can export the template for each object from Client zero. 
    This is already aligned with Product Management. 
    Opening the import Format as an official interface would be a feature request. 

    Best regards,
    Antoine


  • 18.  Transport case file format

    Posted Feb 15, 2017 04:51 AM
    Thanks, Antoine_Sauteron. That is not the answer I was hoping for, but it is the answer I was expecting.

    After years of pointing to the XML file format as an open and fully documented  alternative to the transport case, Automic decided to eliminate this documentation without notice or warning. This is very disappointing.


  • 19.  Transport case file format

    Posted Feb 15, 2017 05:12 AM
    Antoine Sauteron wrote:
    Here's the answer I got from the head of development about the XML format documentation
    ...
    Opening the import Format as an official interface would be a feature request. 
    I submitted an enhancement request for this on 23 May 2015:

    The XSD (XML Schema Definition) for the Automic Automation Engine XML export file format should be created if it does not already exist and made publicly available.

    The use of an XML Schema Definition (XSD) eliminates confusion and ambiguity regarding the details of an XML schema, and provides a standard way to validate adherence of an XML document to the schema specification. Further, it facilitates integration with other applications that must read or write XML files according to the schema.
    I do not know the ticket number of this enhancement request. I have not received any feedback on the request.


  • 20.  Transport case file format

    Posted Feb 15, 2017 05:43 AM
    Hi Michael_Lowry
    I searched in the records and it's most likely PMPER-452.

    Description :

    *Requirements description: *
    The XSD (XML Schema Definition) for the Automic Automation Engine XML export file format should be 1) created if it does not already exist, and 2) made publicly available.

    *Current Workflow: *
    There is no published XSD for the Automation Engine XML file format, although most of the details of the schema are described in the document Inside Automation Engine.

    *Goal: *Simplify XML-based integration between Automation Engine and other programs.
    Scope & context: Automation Engine public documentation.

    Business Impact:The use of an XML Schema Definition (XSD) eliminates confusion and ambiguity regarding the details of an XML schema, and provides a standard way to validate adherence of an XML document to the schema specification. Further, it facilitates integration with other applications that must read or write XML files according to the schema.

    *Use Cases: *
    Title: Creation, modification, and validation of Automation Engine XML export files.
    Description: In our experimental (EXP) and development (DEV) environments, we allow our batch developers to import their own objects from XML. Some of them have developed tools to create Automation Engine XML files based programmatically based on inputs from other sources. In addition, Swiss Re has been using a conversion tool developed by Automic partner System Partners, to convert TWS for z/OS job streams to UC4 workflows. Lastly, we are developing our own toolkit to handle migration of batches of UC4 workload scheduling objects from DEV to TEST and PROD. This toolkit is based around the UC4 XML file format. In all of the above situations, an official XSD would greatly simplify our work, reduce the amount of manual effort required, and cut down on the number of errors.


    Best regards,
    Antoine


  • 21.  Transport case file format

    Posted Feb 15, 2017 05:51 AM
    That’s the one. PMPER-452 originates in INC00058946 & this thread from two years ago:
    XML Schema Definition (XSD) for Automation Engine XML export file format

    Exporting templates from client 0 is unfortunately not particularly helpful. The high level structure is there, but templates are too simple. Templates do not reveal deeper parts of the schema or relationships between different parts of the schema. Moreover, without documentation there is no clear way to determine the functional role of each element & attribute.


  • 22.  Transport case file format

    Posted Feb 15, 2017 05:59 AM
    I suppose we will have to create a set of representative objects, generate an XSD from the corresponding XML, and reverse-engineer a more complete description from that. The v11.2 XML schema documentation will surely help.


  • 23.  Transport case file format

    Posted Mar 03, 2017 08:53 AM

    I submitted an idea entitled “XML schema description for AE export file format” at the new ideas.automic.com web page. The idea has been accepted for voting.



  • 24.  Transport case file format

    Posted Mar 29, 2017 04:15 AM
    I updated the parse_transport_case.sh script attached to the above comment to fix a small bug that could prevent the last object in the transport case file from being processed correctly.


  • 25.  Transport case file format

    Posted May 16, 2017 03:40 AM
    I discovered that in a certain rather unusual set of circumstances, an O record may be followed immediately by an F record. This happens when an object with only OH records (with records for no other table) is followed by another object. In this case, the following sequence of records will appear in the transport case:
    • TOH marking the beginning of the first object
    • F records for the OH table for the first object
    • O record(s) for the first object
    • F records for the OH table for the second object
    I believe the logic is something like this: if an F record is encountered, it applies to the table of the most recently-read T record, even if there have been intervening R or O records. If this happens after an O record, it can be assumed to be the first F record of the OH table for a new object.

    Normally, after the optional O record, one would expect to see records (T, F, R) for other tables. In this case, the object was a DOCU object with no content. For this reason, there were not even records for the ODOC table — only for the OH table. This is, as I indicated, a situation that is very unlikely to come up in practice.

    I have updated my parse_transport_case.sh script (attached here) to handle this situation correctly.


  • 26.  Transport case file format

    Posted May 17, 2017 08:19 AM
    I created a new thread for all of the various transport case parsing scripts. When I revise the scripts, or develop new ones, I will post an update there.


  • 27.  Re: Transport case file format

    Posted Apr 17, 2018 06:14 AM