Hello Endevor Community,
some days ago - during a research case - I became aware, that Endevor's SOURCE LEVEL INFORMATION history is related to the element as a whole. In case you are searching the source levels related to a particular stage, this information could be misleading.
See my example below, where you can NOT definitely say, which level existed on 18MAR15 07:00 ... in that stage (you also have to know that STAGE ID:P is not a first stage, where the element-source is edited/added).
By using a SMF Report-42, we could verify, that it was Level 0100, because the MOVE of level 0101 of this element occurred on 18MAR15 around 10:00....
Is there an easy possibility to get a stage-related SOURCE-LEVEL INFORMATION of an element for the user?
Are there already other Displays, Prints, Panels etc. where the source-related SOURCE-LEVEL INFORMATION could be viewed by the user?
How do you interpret the SOURCE LEVEL INFORMATION and teach it to your Endevor-users?
Would it make sense to have a second DATE/TIME column with the stage-related level history, and wouldn't this become confusing ?
Afterwards added - for clearification purpose - the underlying scenario:
An Endevor-user investigates and researches a production-issue. This issue occurred at a certain point-of-time. The Endevor-user wants to know, which level of source was active in the prod-stage at this point-of-time. He queries MASTER INFO, but the requested point-of-time is before "LAST ELEMENT ACTION", "CURRENT SOURCE" or "GENERATE" (the other timestamps RETRIEVE / BASE are not usable for this purpose). *) So he remembers to use BROWSE HISTORY. Here het gets a list of source levels combined with timestamps. Then he matches the requested point-of-time of his issue into these levels and might get a wrong information for his issue. He queried the element in production, but the presented element-levels concern the point-of-time, when the source changed in the entry-stage and probably were not yet in production. This Information is correct, but not useful in this scenario - actually misleading.
*) Which different option has the user at this point to get the desired information?
Is there a customization option to get useful/correct information in use-cases like that?
It's hard to advise you with the limited information available. It almost looks like you're doing MOVE WITH HISTORY as you progress up through the stages.... Are you? If so, that would definitely account for and contribute to confusion as to what was where when and by whom.
Regardless, take a look at the CSV utility for doing extracts on the element level information; it contains the user id of the person associated with that level.
thanks for your feedback,
in fact it was an 'O' (Move) Action in Quickedit. In AO (Action Options) "With History" is set 'N'.
From your reply, I assume, that you have a different experience?
In fact, the SOURCE LEVEL INFORMATION is correct, referring to my example above, Level 0101 was created on 17MAR15 18:17. But this Editing took place in the first stage of the Environment.
If you see the element-history of the element in the second stage (STAGE ID P, like above), it seems, that level 0101 was there already on 17MAR15 18:17. But this is not true. In my opinion this is misleading. In other words SOURCE LEVEL INFORMATION does not reflect source changes by actions like MOVE.
Do you know a possibility to make also these source-changes visible in the element-display? Maybe we have to adapt our Endevor-implementation here ...
You might be confusing yourself but again, it's hard to say. So bear with me for a second...
Let's suppose I have an element in production that is currently 1.05. It holds the history of 1.00, 1.01, 1.02, 1.03, 1.04 and 1.05.
I retrieve the element from production and insert it into a lower stage in my life-cycle. When that happens, Endevor will automatically look up the map from that stage and copy back the first element it finds to the stage I am adding my element to. In this example scenario, I am assuming nothing exists at any stage between me and production, so Endevor finds 1.05. Endevor will then copy that element to the entry stage as 1.05, do the ADD of my element, and create 1.06 also at the stage.
In the end I actually have "2" versions of my element at that entry stage; the original unaltered "last version" (1.05) and the new version I just added (1.06).
In the meantime, Endevor is time-stamping different information in different places for different events. The ADD/UPDATE action gets one time stamp, the "last GENERATE" gets another time-stamp, and the last "action" gets another time-stamp. Then you throw in the different association of different CCID's to the different actions and one can get confused pretty quickly if one isn't clear on what one wants to know!
In the end, I would argue you did exactly the right thing; SMF reporting is a gold-mine of information for the administrator and is, in my humble opinion, the go-to place for finding out a complete record of events that took place in Endevor for any given time period.
I may be confusing more than I'm helping, but in the end I think the definitive answer you are looking for changes a bit with the manner in which the question is asked and the person who is doing the asking (and what they're really looking for as an answer!).
thanks a lot for your effort and helpfullness. but now I'm in doubt, whether I described my question clearly enough. Therefore let me take you to the underlying scenario:
Is there a customization option to get useful/correct information in use-cases like that?
What summary information do you get if you just use the Display panel, and then the Element Display panel with "S" (Summary of Changes) to the element in the PROD stage? Does it show the same time stamps beside the different version/levels?
Yes, there are the same timestamps, in "SUMMARY OF LEVELS" the same as in "Summary of Element Levels" in Quickedit.
Again: the information is correct, but: the timestamps refer to changes made in the entry-stage and not in the displayed stage. Someone has to take this into account and has to decide, whether this Information is useful / makes sens for him.
OK. Yes. I understand now.
I'm still thinking about,
how to provide the needed element-related information to the Endevor-User in a simple and clear(!) - unmisleading - way
or how to shape an idea for an Endevor enhancement.
any assists are highly appreciated
There's a lot going on here but I picked up on a couple pieces of information that appear relevant:
a) When a production issue occurs, the troubleshooter needs to know at what point in time the problem began occurring. For this investigation, it is very helpful to be able to quickly identify exactly when the code changed in production.
b) The move to production occurred in foreground or batch mode, via a Quick Edit move action.
My two cents: secure production moves with at least package-processing-required. That way, when there is a production issue, you can:
a) backout out the problem immediately
b) know exactly what date & time that package ran via the Master record info
many thanks for your feedback and your valuable 2ct.
I totally agree to use package-approval-processing for changeovers into production to make these changes secure and audit-proof!
That does not take away, that in the source-level-information (history or summary) of that prod-element are timestamps about levels, which do not mark the source-level-change in this stage, but origin from the source-level creation in a different stage - but this fact is not visible. This is independent, whether the MOVE was executed in foreground, batch or batch-package etc.
Well, you and I and all Endevor-enthusiasts here know that fact and can deal with it.
Imagine an average Endevor-user, who wants to determine the source-level of his prod-element for a particular point-of-time and inquires the element history (please see my example in the original post) or element-summary. I'm pretty sure, that he comes to an incorrect conclusion, then wondering and squirrelling around because this does not match to his other observations and finally asking Endevor-admin to clearify the mystery.
I'm still considering, what the best presentation would be to provide a correct, complete and unambigous picture of the source-level-information to the (average) Endevor-user. Any additional consideration, suggestion etc. is welcome and appreciated.
Kind regards, Josef
This issue is still nagging me so I reread you original post and looked at your provided example.
To clarify my thinking in the example (0100 - 0106), are you saying that between 18MAR15 700 (when the job the developer wanted to investigate ran) and 20MAR15 1800 (when the example provided was screen-captured), the element was promoted/MOVED into that PROD stage 6 times? Or am I really looking at ONE promotion that occurred with 6 history/deltas (i.e. someone moved the element to production WITH HISTORY).... or some combination of the above (sometimes with history, sometimes without)?
Something just doesn't strike me right with what I'm looking at.....
the element was really moved 6 times into prod, but this was only because of a more and more confused Endevor-user.... :-).
You are right, the Job/JCL to be investigated ran on 18MAR ~07:00. The searchung user had assumed that level 0101 was active at that time, but in reality the move took place at 18MAR ~10:00. Afterwards he changed the jcl-source up and down, searching an explanation for the supposed "mystery", called me later and finally we got behind the truth using smf-report. But this is only the story around a principle fault, which I recognized:
The source-change in prod (by the MOVE action) is not recognizable in the source-level-history. There is no indication in the display/ summary etc. that the source-change for level 0101 took place in another stage and was not yet active in this stage.
One may say: "Endevor is working as designed" - accepted!
But this does not take away, that the timestamps in source-level-information of non-entry-stages are wrong (concerning the stage of the element and except the latest level, which can be queried in the MASTER INFO) in either case they are misleading ...
I opened CA case 00057472 to ask for CA's (product Management) official position for this question.
I'm going to add my 3 cents to Dana's two cents to make a nickel. The first thing I find curious is doesnt the footprint of the failing load module give you the element level? The other thing I find confusing, is wouldn't the failing production load bleong to the current prodcution source level? Is 18MAR15 07:00 you list above the load footprint? To Debug a failing load I usually work backwards from the load footprint and look at component history to see whcih source level is associated with that generate or move timestamp's component level.
STEP: COPYLOAD DD=BCHLOADO VOL=P4EN01 DSN=TTAP.END.TPROD.BCHLOAD
MEMBER VVLL DATE TIME SYSTEM SUBSYS ELEMENT
%+0115 ALANTDB2 0103 24FEB15 09:39 ENDADMIN S1 ALANTDB2
%+0114-0115 ALANTDB2 0103 12JAN15 15:23 ENDADMIN S1 ALANTDB2
%+0113-0114 ALANTDB2 0103 06JAN15 15:44 ENDADMIN S1 ALANTDB2
%+0112-0113 ALANTDB2 0102 06JAN15 14:23 ENDADMIN S1 ALANTDB2
these are all generate times, but the VVLL is the element source level.
When I go to the element section of HX, I get source date and time, so vvll 0103 is from 06jan15 at 15:44
-------------------------- ELEMENT INFORMATION -------------------
VVLL DATE TIME SYSTEM SUBSYS ELEMENT TYPE
%+0113 0103 06JAN15 15:44 ENDADMIN S1 ALANTDB2 COBOL
%+0112-0113 0102 06JAN15 14:22 ENDADMIN S1 ALANTDB2 COBOL
%+0110-0112 0100 12OCT12 12:55 ENDADMIN S1 ALANTDB2 COBOL
%+0109-0110 0100 12OCT12 12:55 ENDADMIN S1 ALANTDB2 COBOL
PF3 from HX to B and get 0103 06jan15 15:44 and a generate time of 24feb15 09:39 which lines up with HX:
-------------------------- SOURCE LEVEL INFORMATION ----
VVLL SYNC USER DATE TIME STMTS CCID
---- ---- -------- ------- ----- -------- ------------
0100 TAXM 12OCT12 12:55 199
0101 T157 06JAN15 14:18 200 COMMENTS
0102 T157 06JAN15 14:22 200 COMMENTS
0103 T157 06JAN15 15:44 201 COMMENTS
GENERATED T157 24FEB15 09:39 PROCGRPCHG
Let me know if I'm following everything correctly.
Jumping in for Josef... in his scenario, it's not a "generated" element per se. It's a JCL member..... So there is reliance on the part of a user (not administrator) to have to find "audit" information as to when an element was where while in Endevor itself (given that most users wouldn't have access to the SMF reports).
He has an interesting conundrum; how can a lowly "user", using just available Endevor commands or displays, determine what was where as opposed to what was when independent of where.
Missed that part. I’m still not grasping where the 18mar0700 timestamp comes from, to.
Technical Support Sr. Specialist
Software Code Mangement Team
Wait a minute!!! A thought just occurred to me!!
Josef, can you run a more extensive SMF report against that element showing ALL actions that were done to it during the time frames in question? It suddenly occurred to me.... the user didn't happen to DELETE that element (during their confusion) and re-ADD it in at some point, did they? That would certainly mess up the timings.....
Hello Endevor afficionados,
for this discussion, it’s important to bear in mind – as John underlined earlier – that the focus is on a different kind of elementtype:
For program-elements the element could change, even there is no source-change, (by re-gereate, new copybook etc.) For these types I consider it good practice, that the program outputs its version on execution. For that purpose we induce the Endevor-Source-level and /-dates as literal-variables into the program, when the element is generated. When the program writes this literal-variable to SYSOUT, the executed version of the program is documented in the joboutput.
But here we focus on element-types like JCL and all kind of (SYS1.)PARMLIBs, where the source itself is the „active code". For these types you are more dependent from – resp. oriented on - the source-level-information.
In place of the suggested SMF-reports, I’d like to bring in a simplified, synthetic scenario, to illustrate, where I see the fault. You could easily play this through in reality in your shop:
Let’s say, we have an entry-stage „E" and a production-stage „P" in our environment.
Step a. On 17MAR15 I create a new element in E-stage and move it to P-Stage the same day.
The source-level-information of this element in the P-Stage would look like:
Step b. On 18MAR15 I have to edit the element and create a new source-level in the E-Stage. Due to necessary checks I don’t move it yet to production:
There are now 2 incarnations of the element, one at the E-stage having source-level-information like that:
The other one at p-Stage still has source-level-information:
Step c. On 19MAR15 the checks are ok and I move the element from E- to P-Stage:
The source-level-information at the p-stage then looks like:
Step d. On 20MAR15 I have to edit the element again, but may not yet move it, which results again in two incarnations:
Step e. On 21MAR15 I got the ok to move and then I have a final source-level-information of the element in the p-stage:
When you are so far, then please go to an average Endevor-user, let him inquire the element and ask him „When did source-level 0101 of this element become active in the P-stage?"
I think, wrong or incomplete=misleading information is worse than no information. Or do you think, that your Endevor-user would immediately ask for a smf-element-activity-report to answer the question?
You might say that inquiries for earlier source levels are not requested frequently! Right! But just for these rare occations it is crucial, to get correct, unambigous information.
As a summary and in other terms:
From a programmers perspective the source-level-information of an element in prodution could be seen as correct, from a production perspective as wrong/incomplete=misleading.
Thank you for going through this discussion. Case 00057472 to get CA’s positioning and/or a solution for better information quality is still open (at this moment).
Now, there are really enough of my 2 ct’s, Have a nice day!
With all these 2 or 3 cents, have we got a dollar yet
My basic Endevor users would do a C for changes against any of the levels and the entire SOURCE LEVEL INFORMATION is displayed. This data contains the CCID that the change was made under.
They would then look in our change system and see two dates. The date it moved to stage P, the date it was copied in to the runtime datasets (we don't execute on the same plex as Endevor runs)
This does rely on the CCIDs being used properly but the ALTER action is going to allow us to ensure that the current source ccid will always match the production migration CCID.
So what additonal information are you going to ask for in the idea that must come out of this lengthy discussion?
BTW I do believe that SMF is my 'Go To' source for element activity. There are a few ideas out there that need incorporating in to the SMF records but for me it is a good way to create a view of the activity on a certain date.
just in short:
My best "idea's" so far to improve the information quality of element-source-Level-Information(-Displays) are (1) either to add a second timestamp parallel to the source-level-timestamp. While the source-Level-timestamp remains unchanged, the second timestamp should be updated, when the element is brought to another stage. or (2) the stage-id, where the source-level was created, should be shown together with the source-level-timestamp.
When you see this adapted source-level-information you will get all Information you need, or you would be reminded to search another log-data or not to take false conclusions.
You did an excellent job of summarizing(?)... or.... maybe better documenting(!) the issue. And in my opinion, you have a very interesting point and I think a very valid case for a "user story" that I would definitely vote "up" for since the business need is pretty apparent.
How about a story such as....
"As a developer, I need Endevor to provide me with an online summary or query for any stage that tells me when an element and/or change level arrived there (i.e. ADD, UPDATE, MOVE) uniquely from when the change was created so that I can perform effective post-implementation analysis and post-implementation problem identification/tracking."
As a result of the discussion I created idea Enhance/complete ELEMENT SOURCE LEVEL INFORMATION if you share the opinion, you might vote for this enhancement request.
PS.: Of course, 19MAR15 was the correct answer in my synthetic example above. After step e. this date can nowhere be found, when you inquire the element, although both the source level and the element in this stage still exists.
An addition of an last action record per summary level could provide the needed information. Example.
------------------------- Summary of Element Levels ---------- Row 1 to 2 of 2
Command ===> ________________________________________________ Scroll ===> PAGE
E Edit Element B Browse H History C Changes M Master S Summary Master
Environment: PROD Stage ID: P
System: CMN Subsystem: CMN0 Type: JCL
VVLL User Date Time Statements Inserts Deletes Sync
- ---- ---------- -------- ------- ----- ---------- ------- ------- ----
S 0100 HXBOWI 20JUL10 18:21 51 0 0
S 0101 HXKADO 13JUN13 14:38 44 10 17
----------------------------- Bottom of the List -----------------------------
------------------- Master Summary Display for Element JIJC4 ------ Panel 1 of 1
Environment.. PROD Processor Group. DATA Signout ID.
System....... CMN VVLL............ 0100 N/S........ SOURCED
Subsystem.... CMN0 Last Action..... MOVE Lock Pkg...
Type......... JCL Source Package.. C1987458999
Stage........ P Output Package.. C1987458999
Base Comment. INITIAL SCM LOAD
---------------------- Last Element Action - MOVE -------------------------
User ID...... HXNDVR Date/Time... 25JUL10 09:00 CCID.......... 50545214
Action RC.... 0004 Processor... MOVE (MOVE)
Processor RC. 0000
Comment...... NEW PRODUCTION JOB
System....... CMN VVLL............ 0101 N/S........ SOURCED
Type......... JCL Source Package.. C2055389100
Stage........ P Output Package.. C2055389100
User ID...... HXNDVR Date/Time... 19JUN13 09:00 CCID.......... 20553891
Comment...... SHUT OFF PRINT TO PIJC4; CODE TO ONBASE
Hi, to add $1.50 to all this.
I stumbled across chain of discussion. I understand what it is you are talking about after reading and seeing your example. However, moving elements and its changes are not normally limited to 2 stages. How would the information on when changes are introduced from one stage to another along the SDLC? In your example you focused on your stage E to stage P. The types like JCLPROC the members on the output libraries should have footprints.
But as you asked, in your question, when did the prior level 0101 get moved/introduced into production(stage P). The same question to be asked if QA stage, UAT etc stages. Once the element moved on up the SDLC, the information is gone from one of those stages. That is why one needs to use Packages (in some manner). The 'record of history' of course is the SMF records. Which will have all the activity on an element anywhere at any time. If a user needed to know when a change (prior level(s)) were introduced into production without using SMF, would have to be deduced. In your example, change for 0101 was introduced to Endevor Mar 18 then another change 0102 introduced Mar 20 and after moving Mar 21(the current level) where the master will have when moved to that stage P. You can deduce between Mar 18 to Mar 20 is when 0101 was moved .
With all that, I however agree that there needs to be more historical information needed on an Element so its more complete, clear.
Since I am dealing with V17, I am not yet familiar with what V18 and its iterations of changes have introduced. I presume nothing related to this topic.