Autogen questions

[Karen Turner]

Working on a demo for developers who may not be aware of all the features endevor has to offer that could be leveraged to perhaps improve their productivity.  I'm working on autogen.  In the x years its been around I maybe touched it once.

 

So first let me ask if my little demo set up would exist in the real world:

COPYA used by PGM1 and PGM2
COPYB used by PGM2
PGM3 calls PGM1
PGM4 calls PGM2
PGM5 calls PGM4
PGM6 calls PGM1 and PGM2

 

ACMQ verifies said intertwined relationships:

1 DEMOCPYA COPYBOOK TTEST ENDADMIN S1 1
2 DEMOPGM1 COBOL TPROD ENDADMIN S1 2
2* DEMOPGM1 COBOL TTEST ENDADMIN S1 1
2 DEMOPGM2 COBOL TPROD ENDADMIN S1 2
2* DEMOPGM2 COBOL TTEST ENDADMIN S1 1
3* DEMOPGM3 COBOL TPROD ENDADMIN S1 2
3* DEMOPGM6 COBOL TPROD ENDADMIN S1 2
3* DEMOPGM4 COBOL TPROD ENDADMIN S1 2
3* DEMOPGM4 COBOL TTEST ENDADMIN S1 1
4* DEMOPGM5 COBOL TPROD ENDADMIN S1 2
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 DEMOCPYB COPYBOOK TTEST ENDADMIN S1 1
2 DEMOPGM2 COBOL TPROD ENDADMIN S1 2
2* DEMOPGM2 COBOL TTEST ENDADMIN S1 1
3* DEMOPGM4 COBOL TPROD ENDADMIN S1 2
3* DEMOPGM4 COBOL TTEST ENDADMIN S1 1
3* DEMOPGM6 COBOL TPROD ENDADMIN S1 2
4* DEMOPGM5 COBOL TPROD ENDADMIN S1 2

 

Now the curious question comes into play when I generate both copybooks in the same batch job.   I (mistakenly) assumed Endevor would analyze the relationships of both components and build the generate order accordingly, but alas, it does not.  

If COPYB is generated first PGM2,4,6, and 5 are compiled in the order displayed above.  However, since AUTOGEN wont generate the same using element twice, the generate of COPYA does not generate PGM6, which would (could?) result in a component validation error since PGM6's component list now contains PGM1's previous generate timestamp and not its current one.  

 

C1G0340I OPTION AUTOGEN IN USE, ALL ACTIONS WILL BE RESOLVED BASED ON CURRENT INVENTORY CONTENTS

C1G0345I PROCESSING AUTOGEN FOR ELEMENT DEMOCPYB/COPYBOOK. TARGET IS TTEST/ENDADMIN/S1/1
C1G0343I ELEMENT DEMOPGM2 / COBOL / ENDADMIN / S1 WILL BE GENERATED
C1G0343I ELEMENT DEMOPGM4 / COBOL / ENDADMIN / S1 WILL BE GENERATED
C1G0343I ELEMENT DEMOPGM6 / COBOL / ENDADMIN / S1 WILL BE GENERATED
C1G0343I ELEMENT DEMOPGM5 / COBOL / ENDADMIN / S1 WILL BE GENERATED

C1G0345I PROCESSING AUTOGEN FOR ELEMENT DEMOCPYA/COPYBOOK. TARGET IS TTEST/ENDADMIN/S1/1
C1G0343I ELEMENT DEMOPGM1 / COBOL / ENDADMIN / S1 WILL BE GENERATED
C1G0343I ELEMENT DEMOPGM3 / COBOL / ENDADMIN / S1 WILL BE GENERATED

 

ADD DEMOCPYB
ADD DEMOCPYA
GEN-AUTO DEMOPGM2
GEN-AUTO DEMOPGM4
GEN-AUTO DEMOPGM6
GEN-AUTO DEMOPGM5
GEN-AUTO DEMOPGM1

 

Obviously, if I generate COPYA first, all 6 pgms are recompiled, and COPYB's autogen skips all of them.

OPTION AUTOGEN IN USE, ALL ACTIONS WILL BE RESOLVED BASED ON CURRENT INVENTORY CONTENTS

PROCESSING AUTOGEN FOR ELEMENT DEMOCPYA/COPYBOOK. TARGET IS TTEST/ENDADMIN/S1/1
ELEMENT DEMOPGM1 / COBOL / ENDADMIN / S1 WILL BE GENERATED
ELEMENT DEMOPGM2 / COBOL / ENDADMIN / S1 WILL BE GENERATED
ELEMENT DEMOPGM3 / COBOL / ENDADMIN / S1 WILL BE GENERATED
ELEMENT DEMOPGM6 / COBOL / ENDADMIN / S1 WILL BE GENERATED
ELEMENT DEMOPGM4 / COBOL / ENDADMIN / S1 WILL BE GENERATED
ELEMENT DEMOPGM5 / COBOL / ENDADMIN / S1 WILL BE GENERATED

PROCESSING AUTOGEN FOR ELEMENT DEMOCPYB/COPYBOOK. TARGET IS TTEST/ENDADMIN/S1/1
[insert crickets]

Does any of this make sense or is it safe to say, always run it in SIM mode to decide who you want to re-compile manually?  (The majority of programs here use NODYNAM and/or static links).

 

Thanks for your opinions!

KT

 

[Eoin OCleirigh]

Hi Karen,

 

Nice little test case!  But If you want Endevor to automatically generate main programs and subprograms in a appropriate order today, you might have to create multiple types and add those new types into the Global Sequence table, then copybooks would always be processed first, then subroutines, then main line programs...

 

Personally I'd be tempted to see if it wouldn't make more sense to move away from static linking and use dynamic linking instead, then the compile order wouldn't matter.

 

But I'd be interested to hear if other customer shared your expectation that Endevor could just work it out.  It looks like all of the required information is there in the ACMQ records.  Potentially the problem is that it would be too easy to create a circular reference that could never be perfectly resolved... 

[Mathew Goldstein]

That is interesting.  I will warn the developers not to use AUTOGEN more than once simultaneously in batch SCL.  If they utilize AUTOGEN only with Quick-Edit then that should avoid this problem. 

 

It may be possible for ENDEVOR to detect such circular dependencies in advance but that may require substantial additional computation.  So I guess it makes some sense that ENDEVOR won't allow multiple AUTOGEN to generate the same program element a second time.  If you have a potential circularity problem then you arguably should re-configure your types and processors anyway to avoid type sequence circular dependency problems. In particular, the main routine load modules that are created from more than one subroutine object module should be created using a type LINK.  Or, to put this another way, any type with a generate processor that creates an object module should not also create a load module which inputs from an object module library to avoid circularity.  Dynamic calls avoids the circularity risk, but it introduces an omitted testing risk.  So I recommend static calls with a type LINK for creating the main routine load modules from multiple subroutine object modules.  But that won't solve the problem of the limitations of AUTOGEN.

[Mathew Goldstein]

The more I think about this the more convinced I am that this problem is a result of circular dependencies coded into your generate processors.  For autogen to produce this bad result you must be executing generale processors that have a step which creates an object module followed by a step which link-edits the current object module together with at least one other previously saved object module to create a load module.  If you relied exclusively on a different type to create load modules from previously saved object modules, I will name this type LINK, and type LINK was sequenced after the types that create the object modules, then all the saved object modules would be generated second, after the copybooks (and macros, etc,), all the load modules would be generated third, after the object modules, and everything would be OK.  Note that it is still possible for the types that create object modules to also create load modules but only when the load module is created from the current object module and not from any previously created, saved, object modules (the link-edit step must  not input from ENDEVOR SCM populated object module libraries, only type LINK inputs from ENDEVOR SCM populated object  module libraries when creating a load module). 

Requiring all of the calling main program load modules to be created using type LINK also eliminates type sequencing problems more generally which is particularly important when you activate concurrent processing.

[Karen Turner]

Thanks Mathew,

As I said this was not a real world example. Wanted to brush up on my (lack of) Autogen knowledge and to know if it was a valid example, 2 copybooks used by 2 programs which are called by other programs which are called by other programs .. 3 levels deep.  It looks like the answer is just use Autogen single threaded as opposed to in batches.

 

Karen

Karen Turner

Technical Support Sr. Specialist

Endevor Support

Software Code Management Team

860-902-3064 (IP phone)

508-399-5366 (home)

508-837-0606 (cell)

Upcoming PTO:  Jun 30th .  Jul 15th.  Aug 18th .

[Mathew Goldstein]

The depth is irrelevant, a subroutine could be called in a sequence of subroutines 100 levels down and that makes no difference because every subroutine is its own object module with no generate dependencies on any other object module.  AUTOGEN on the modified assemble and compile time input components will add generate actions for object module types and AUTOGEN on the modified object module types will add generate actions for type LINK.  It is only when the load module is generated with type LINK that which object modules calls what object modules becomes relevant and the binder has the built in intelligence to deal with that.  If the object module libraries are in the binder step SYSLIB DD concatenation then the binder step only needs the name of the main object module and the binder will automatically identify all of the called subroutine object modules and create a single load module linking all of the called object modules.  You can execute AUTOGEN for every modified input component and called program element without any problem because the generate sequence will always start with the compile time input component types followed by the object module types.  AUTOGEN will identify the type LINK program elements that need to be generated and they will be generated last to create the load modules.  This approach not only eliminates the AUTOGEN problem, it also eliminates type sequencing conflicts that are particularly risky when concurrent processing is activated.