Endevor

Expand all | Collapse all

Fixing SYNC issues

  • 1.  Fixing SYNC issues

    Posted Jun 30, 2011 02:59 PM
    SYNC is an ongoing struggle for our users. They like to keep up to date with changes and prevent potential migration issues so they will frequently run a VALIDATE of their elements to proactively find out-of-SYNC elements. They will then fix the code, but Endevor is none the wiser. She still reports the element as being out of sync because the base still doesn't match what's up the map. Using the SYNC option to fix it requires that you go somewhere via MOVE or TRANSFER, but in most cases our users are in the middle of long running projects and are camped out in stages for months.

    Currently our users DELETE and reADD the element to fix the SYNC. Besides annoying them, this maneuver also means they loose their history. We've explored TRANSFERing out and then back into the same location, but that involves a lot of who-hah and requires an Endevor admin. I wish there was a new action called SYNC or that the UPDATE action had a SYNC option.

    Does anyone else have a need to SYNC in place? How do you resolve it?


  • 2.  RE: Fixing SYNC issues

    Broadcom Employee
    Posted Jul 01, 2011 08:13 AM
    This question is pretty usual in Development environment, within single mapping or mapping made of parallel/Concurrent environment.
    Unfortunatly, there is not one answer only but it may be an answer per case and these answers will depends on the organization for a great part.
    Long time development is often impacted when emergency fix are necessary somewhere at the target environment and when moving element and the Sync question, Endevor research a level in the source element that matches the last level of the target element, the time stamp is the base of comparison when you move without history...
    If you want investigate the synchronization question, Add these two DD statements to your SCL execution JCL: //EN$TRSYN DD DUMMY //BSTERR DD SYSOUT=*
    To conclude this post, I would said the first is to identify the main and most common reason of sync situation and see if this can be avoided, what is a big question in general.
    Most of the time, PDM(Parallel Development Manager) can help to consolidate the changes.
    Anyway, if you think something needs to be investigated, never hesitate to open an issues at CA Support.
    Ollivier,
    CA Support


  • 3.  RE: Fixing SYNC issues

    Posted Jul 05, 2011 07:05 AM
    Unfortunately, the more concurrent development you have the more SYNC issues you will encounter. If the last level in PROD (for example) is not the first level you started your development with then...well you know.
    Deleting an element in the middle of the lifecycle while another level exists below will also cause SYNC issues. It is a blessing and a curse, it lets you know that changes were made higher in the map so you dont regress
    code but can only rely on the levels and not the source itself.


  • 4.  RE: Fixing SYNC issues
    Best Answer

    Posted Jul 06, 2011 02:54 AM
    Marc,

    I think you need to warn your users against the two methods that you quoted to "fix "the sync issue:
    - the use of the SYNC method
    - the retrieve delete add back method.
    Both methods, if used without any other form of intelligence will cause a change to be backed out.

    Let's talk about the causes of SYNC (repeat what Joe and Ollivier explained) and explain why both methods need to be handled with extreme caution.
    A SYNC condition occurs when an element's delta versions do not have a common timestamp with the last (current) time stamp with the same element up the map. There are a few simple scenarios which can create a sync condition (consider the map STG1 - STG2 - STG3 - STG4):

    Scenario #1
    - A new element gets added to STG1, then an element with the same name type sys/sbs get added to STG3 and moved to STG4. In this case the element in STG1 has no common delta level with the element in STG4 so when moving the element in STG1, or when validating the element a SYNC condition gets detected (SMGR116).

    Scenario #2
    - An element is retrieved by two users from STG4 and added back with changes into STG1 in different subsystems, both mapping to the element in STG4.
    Both elements are in sync because during the add back (or the QE EDIT operation) the last delta level at STG4 got fetched back. You'll see that the elements have 2 delta levels, the oldest being the one from STG4 (the fetched level) the newest being the change level. When you run a VALIDATE on either of the elements it will result in RC0, both are synchronized with the next element up the map.
    Now one of the two updated elements gets promoted to STG2, then STG3 then STG4.
    If you run VALIDATE on the remaining element in STG1, you will get a SYNC error SMGR116 because the element in STG1 does not have a matching delta level with the most current level in STG4

    Scenario #3 (Joe Bigini's):
    An element exists in STG4, it gets retrieved, changed and added back, moved up to STG3.
    Then the element in STG3 gets retrieved and changed and added back to STG1.
    all the elements are aligned. STG4 has the latest level, STG3 has a common delta level with the one in STG4 and a new level, while the element in STG1 has a common level with the element in STG3 and its own new level.
    VALIDATE actions against each of the elements would not find any SYNC errors.
    Not the element in STG3 is found to be in error (change not needed after all- Scenario 3a, or change needs to be improved Scenario 3b)
    =3a=
    The element in STG3 gets deleted as the change is not needed.
    Now the element in STG1 is no longer connected to the last delta level in STG4 so VALIDATE will correctly report SMGR116
    =3b=
    The element in STG3 gets updated once more, creating another delta level.
    VALIDATE of the element in STG3 will tell you it is synchronized, but a VALIDATE of the element in STG1 will fail, as there is no connection between any level in STG1 and the latest (current) delta level in STG3. (this is very similar to scenario-2).

    ---------------------------

    These are the most common scenarios for sync errors. You can mix the TRANSFER action in or do the same with Quick-Edit, it does not really matter. The current level (last level) at the next higher location in the map must match with a source level.

    ---------------------------

    So what does the SYNC option do and why does it need to be used with extreme care:
    The SYNC option tells Endevor that the user knows what he is doing (...) and that the current source version is the correct one

    In scenarios #1, #2 and #3b option SYNC will replace what's in STG4 and delete all ithe changes that were made previously. The simplest illustration can be made using scenario#2. using SYNC here would simply backout the parallel change that got to STG4 earlier.
    Only in scenario #3a would the SYNC option be harmless, where the user would not corrupt any existing changes but acknowledge the fact that his element is good but not synchornized, although in this case using SYNC would cause the retention of a delta level (the one that existed in STG3) that may have no use and might in the future confuse your users.

    Now let's talk about RETRIEVE, DELETE, ADD BACK.
    Here too, only in scenario #3a this would be the correct solution. For this scenario it would accomplish exactly what was needed. When the add back gets done, the last level of the element in STG4 gets fetched back and the elements are now in sync, leaving no trace of the deleted element in STG3.

    For the other scenarios on the other hand, RETRIEVE, DELETE and ADD BACK would have the same effect as using SYNC with one minor difference.
    When adding back, Endevor fetches the last level of the element in STG4 and from then on the elements are synchronized, but as with the SYNC option, the element that was added back does not contain the changes that were made previously in STG4 and thus it will undo them.

    The only correct solutionis to retrieve the element from STG4, merge the changes of STG1with the ones from STG4 and then RETRIEVE, DELETE, ADD BACK, or simply add back and use the SYNC option to tell Endevor that you know what you're doing.
    The merging process is greatly simplified when using Endevor's PDM function but with ot without PDM, the programmer and in the end the approvers need to validate that in case of SYNC problems nobody accidentally removed any changes that were previously made.

    -------------------------------

    So tell your users to remember well that SYNC is a sledgehammer approach: using SYNC the programmer will basically say that his source is the best and he does not care what changes were made before him. SYNC will push this version of the element through to STG4 and nothing will stop it.

    In most shops that I know of, the use of SYNC is very much restricted and under for good reasons under constant scrutiny.

    The SYNC trace that Ollivier mentioned is a tool for administrators to understand why some more complex sync errors are indeed sync errors.
    If you think that a reported sync error is not an error, run the trace and if you still think Endevor is not detecting the sync error correctly then we invite you to contact Technical Support.

    Avoiding SYNC problems is a very good way to avoid these complexities, but as parallel development has become the norm, we have to live with its constraints and educate our users and avoid them shooting themselves in the foot.



    Regards,
    Jan van Kan


  • 5.  RE: Fixing SYNC issues

    Posted Jul 15, 2011 10:14 AM
    Jan mentions the first of the correct solutions is to "... retrieve the element from STG4, merge the changes of STG1with the ones from STG4 and then RETRIEVE, DELETE, ADD BACK,..". It might be worthwhile to point out an additional action you can take if you prefer not to lose the change deltas following the delete. You can do multiple Retrieves and multiple ADD/Updates- one for each change level of the element before it's deletion. Endevor provides the CCID and comment for each of the change levels, allowing you to capture the CCID and Comment for each level and use them again with the the ADD/Updates actions.

    As Jan implies, the best solution involves a PDM merge and SYNC error resolution working together. In the CA Services area we have responded to a customers' query about how to "marry" projects together. We have a tool that does exactly that, resolving the concurrent development issues for all elements within a Sandbox area brought on by another project. First it executes the PDM and directs the Merge action back into the Sandbox. Then it executes a VALIDATE action on the elements in the Sandbox to identify which elements have a SYNC error. For elements out of sync, it Retrieves (all deltas), Deletes , and ADD/Updates (all deltas) back into the Sandbox area, and SIGNs in the elements back to the persons who had the Signout prior to the Delete. Only elements in the Sandbox area which are also in one other designated Endevor area are impacted. When done, the tool has resolved the merge and Sync challenges and "marries" two projects together - even if one of the projects is up the map from the other.


  • 6.  RE: Fixing SYNC issues

    Posted Jul 18, 2011 05:01 AM
    What we need is a SYNC in place action. I.e. the developer knows that there is a sync issue, he/she has investigated and concluded that no changes are required because the changes have already been applied or aren't needed. The developer then SYNCs the element where it is to remove the out of sync condition.


  • 7.  RE: Fixing SYNC issues

    Posted Jul 26, 2011 07:17 AM
    Hello Emlyn,

    i know some shops, they've written a little user-funktion for that. That's nothing you've perform magic for ;-)

    But, as Jan said, hanle with care!

    regards
    Willi


  • 8.  RE: Fixing SYNC issues

    Posted Jul 26, 2011 11:13 AM
    Hey Jan,

    very nice post!
    I translated it briefly into german an put it into our user handbook!
    I hopefully suppose, i got your permission for that!

    cheers
    Willi


  • 9.  RE: Fixing SYNC issues

    Posted Jul 26, 2011 07:17 AM
    Hi WIlly,

    Sure, Feel free to use the post as you seem fit.

    Jan


  • 10.  RE: Fixing SYNC issues

    Broadcom Employee
    Posted Apr 17, 2014 07:14 PM

    I have essentially the same problem, but perhaps it is different enough there is a solution for it?

    I'm not concerned about changes to program modules.  My group only codes in C, so if there are changes in multiple sandboxes to the same CPGM element, we have no problem getting the code merged and not using sync.  Sure, it would be easier if we could make the 2nd sandbox have the 1st sandbox's changes in D1 and use sync, but this doesn't happen enough to be an issue for our group.

    What is a big issue is what I think is called "footprint" checking.  Every one of our 140 CPGM elements include every one of our 35 CHDR elements. It was just easier to do it that way.  So changing any CHDR affects every CPGM from Endevor's viewpoint.

    But for over 30 years (only last 5 on Endevor) our group has had a policy to never change a CHDR to change existing offsets, or rename or delete an existing field.  So all changes reuse 'filler' area or add new fields at the end of the structure. No CHDR includes any code, only structures, #defines, etc.  Therefore, there is no need for checking that a CPGM module was compiled from the latest CHDR version in P2. Either the CPGM element was not affected, or it would not have compiled.

    So is there a way to just turn off this footprint checking - without using SYNC which would lose change history?

    We would also like to just use the "O" move option to update CHDR elements, and not include them in packages.  We do this in our "base" trunk, but not in GA released trunks.

    Thanks in advance for any help with this.  We often get delayed sending solutions to QA, and it's a lot of work to make the changes twice in D1, retest, regen/insert/upload in MSPS.



  • 11.  RE: Fixing SYNC issues

    Broadcom Employee
    Posted Apr 22, 2014 04:28 PM
    ...But for over 30 years (only last 5 on Endevor) our group has had a policy to never change a CHDR to change existing offsets, or rename or delete an existing field.  So all changes reuse 'filler' area or add new fields at the end of the structure. No CHDR includes any code, only structures, #defines, etc.  Therefore, there is no need for checking that a CPGM module was compiled from the latest CHDR version in P2. Either the CPGM element was not affected, or it would not have compiled.

    So is there a way to just turn off this footprint checking - without using SYNC which would lose change history?...


    While I wouldn't ordinarily recommend it, if you are ABSOLUTELY SURE about your policy for CHDRs you could disable monitoring just for these components by moving them to a separate library, concatenated at the top of your SYSLIB and using the option MONITOR=NONE for that DD.  This should effectively disable ACM for just these 35 common CHDRs - but use with caution.