Endevor

 View Only
Back to discussions
Expand all | Collapse all

Large VSAM ELIB Reorg: REPRO VSAM > BDAM

  • 1.  Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Posted Sep 18, 2023 12:06 PM

    Long ago and far away JvK added this paragraph to the Elib maintenance KD:

    Note:  When dealing with large VSAM ELIBs that are in multiple physical extents, BC1PNCPY can take a long time to run. Although IDCAMS repro use is discouraged as the permanent resolution to your fragmented file it can be used as an intermediate step to speed up the BC1PNCPY process, by reducing the number of physical extents. So following the advice above:  Allocate 2 new ELIBs: 1 VSAM and 1 BDAM, ensuring both are large enough to fit the existing file into a single extent. Use BC1PNLIB to INIT the VSAM file.  Use IDCAMS to REPRO the fragmented ELIB to the un-initialized BDAM file, then use BC1PNCPY to copy the BDAM ELIB file to the newly initialized VSAM ELIB.

    Has anyone done this successfully? 

    I have a 2000 cyl production delta in 2 vsam extents and 5 elib extents.  I do not have a 20 hour window to BC1PNCPY it from old VSAM to new VSAM.

    Testing with a smaller file I get an error on the VSAM to BDAM repro:

     REPRO INFILE(VSAM) OUTFILE(BDAM)                                        
    IDC3300I  ERROR OPENING TTAP.END.TPROD.OTHDELTA                          
    IDC3351I ** VSAM OPEN RETURN CODE IS 118                                 
    IDC3302I  ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM                   
    IDC3309I ** RECORD X'0000000100' NOT WRITTEN.  LENGTH INVALID            
    IDC3302I  ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM                   
    IDC3309I ** RECORD X'0000000200' NOT WRITTEN.  LENGTH INVALID            
    IDC3302I  ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM                   
    IDC3309I ** RECORD X'0000000300' NOT WRITTEN.  LENGTH INVALID            
    IDC3302I  ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM                   
    IDC3309I ** RECORD X'0000000400' NOT WRITTEN.  LENGTH INVALID            
    IDC31467I MAXIMUM ERROR LIMIT REACHED.                                   
    IDC0005I NUMBER OF RECORDS PROCESSED WAS 0                               
    IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12                       

    BDAM JCL:

    //BDAMBR14 EXEC PGM=IEFBR14                                         
    //BDAM DD DSN=TTAP.END.TPROD.TSTDELTA.BDAM,DISP=(NEW,CATLG,DELETE), 
    //       SPACE=(CYL,(10,10)),MGMTCLAS=MCPERMMB,                     
    //       DCB=(RECFM=FBS,LRECL=4096,BLKSIZE=4096,DSORG=DA)           

    REPRO JCL:

    //REPRO EXEC PGM=IDCAMS                                 
    //SYSPRINT DD SYSOUT=*                                  
    //SYSOUT   DD SYSOUT=*                                  
    //VSAM DD DSN=TTAP.END.TPROD.OTHDELTA,DISP=OLD          
    //BDAM DD DSN=TTAP.END.TPROD.TSTDELTA.BDAM,DISP=OLD     
    //SYSIN DD *                                            
     REPRO INFILE(VSAM) OUTFILE(BDAM)                       

    Any insight is greatly appreciated!

    Thanks!

    kt



    ------------------------------
    Karen Turner
    ------------------------------



  • 2.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Broadcom Employee
    Posted Sep 18, 2023 12:25 PM
    Hi Karen,

    No, I do not have any insight into that KB; I avoid ELIBs whenever
    possible.

    My immediate reaction is to copy it to a PDSE instead. PDSE is now the
    recommended format for Delta libs. Do a dry run copying from ELIB to a
    'throw-away' PDSE to see how long it'll take.

    *Dave Harding *
    Client Services Consultant • Mainframe Software Division
    Broadcom Software
    Mobile 317-403-1740 | dave.harding@broadcom.com
    Upcoming vacation:
    September 2 - September 10
    October 4 - October 8



    On Mon, Sep 18, 2023 at 12:06 PM Karen Turner via Broadcom <
    Mail@broadcom.com> wrote:

    > Long ago and far away JvK added this paragraph to the Elib maintenance KD:
    > Note: When dealing with large VSAM ELIBs that are in multiple... -posted
    > to the "Endevor" community
    > [image: Broadcom] <https: community.broadcom.com="">
    > Endevor
    > <https: community.broadcom.com/mainframesoftware/communities/community-home/digestviewer?communitykey="592eb6c9-73f7-460f-9aa9-e5194cdafcd2">
    > Post New Message <broadcom-caendevor@connectedcommunity.org>
    > Large VSAM ELIB Reorg: REPRO VSAM > BDAM
    > <https: community.broadcom.com/mainframesoftware/discussion/large-vsam-elib-reorg-repro-vsam-bdam#bmdae16646-bea3-4d24-81dd-97b8cc1776fd="">
    > Reply to Group
    > <broadcom_caendevor_dae16646-bea3-4d24-81dd-97b8cc1776fd@connectedcommunity.org?subject=re:+large+vsam+elib+reorg:+repro+vsam+%3e+bdam> Reply
    > to Sender
    > <https: community.broadcom.com/mainframesoftware/communities/all-discussions/postreply?messagekey="dae16646-bea3-4d24-81dd-97b8cc1776fd&ListKey=6c02d13b-c28e-4794-b86a-c8b767e00498&SenderKey=434cbcf1-f6ec-4fa2-b1d5-23006686c9e2">
    > [image: Karen Turner]
    > <https: community.broadcom.com/network/members/profile?userkey="434cbcf1-f6ec-4fa2-b1d5-23006686c9e2">
    > Sep 18, 2023 12:06 PM
    > Karen Turner
    > <https: community.broadcom.com/network/members/profile?userkey="434cbcf1-f6ec-4fa2-b1d5-23006686c9e2">
    >
    > Long ago and far away JvK added this paragraph to the Elib maintenance KD:
    >
    > Note: When dealing with large VSAM ELIBs that are in multiple physical
    > extents, BC1PNCPY can take a long time to run. Although IDCAMS repro use is
    > discouraged as the permanent resolution to your fragmented file it can be
    > used as an intermediate step to speed up the BC1PNCPY process, by reducing
    > the number of physical extents. So following the advice above: Allocate
    > 2 new ELIBs: 1 VSAM and 1 BDAM, ensuring both are large enough to fit the
    > existing file into a single extent. Use BC1PNLIB to INIT the VSAM file. Use
    > IDCAMS to REPRO the fragmented ELIB to the un-initialized BDAM file, then
    > use BC1PNCPY to copy the BDAM ELIB file to the newly initialized VSAM ELIB.
    >
    > Has anyone done this successfully?
    >
    > I have a 2000 cyl production delta in 2 vsam extents and 5 elib extents.
    > I do not have a 20 hour window to BC1PNCPY it from old VSAM to new VSAM.
    >
    > Testing with a smaller file I get an error on the VSAM to BDAM repro:
    >
    > REPRO INFILE(VSAM) OUTFILE(BDAM)
    > IDC3300I ERROR OPENING TTAP.END.TPROD.OTHDELTA
    > IDC3351I ** VSAM OPEN RETURN CODE IS 118
    > IDC3302I ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM
    > IDC3309I ** RECORD X'0000000100' NOT WRITTEN. LENGTH INVALID
    > IDC3302I ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM
    > IDC3309I ** RECORD X'0000000200' NOT WRITTEN. LENGTH INVALID
    > IDC3302I ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM
    > IDC3309I ** RECORD X'0000000300' NOT WRITTEN. LENGTH INVALID
    > IDC3302I ACTION ERROR ON TTAP.END.TPROD.TSTDELTA.BDAM
    > IDC3309I ** RECORD X'0000000400' NOT WRITTEN. LENGTH INVALID
    > IDC31467I MAXIMUM ERROR LIMIT REACHED.
    > IDC0005I NUMBER OF RECORDS PROCESSED WAS 0
    > IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12
    >
    > BDAM JCL:
    >
    > //BDAMBR14 EXEC PGM=IEFBR14
    > //BDAM DD DSN=TTAP.END.TPROD.TSTDELTA.BDAM,DISP=(NEW,CATLG,DELETE),
    > // SPACE=(CYL,(10,10)),MGMTCLAS=MCPERMMB,
    > // DCB=(RECFM=FBS,LRECL=4096,BLKSIZE=4096,DSORG=DA)
    >
    > REPRO JCL:
    >
    > //REPRO EXEC PGM=IDCAMS
    > //SYSPRINT DD SYSOUT=*
    > //SYSOUT DD SYSOUT=*
    > //VSAM DD DSN=TTAP.END.TPROD.OTHDELTA,DISP=OLD
    > //BDAM DD DSN=TTAP.END.TPROD.TSTDELTA.BDAM,DISP=OLD
    > //SYSIN DD *
    > REPRO INFILE(VSAM) OUTFILE(BDAM)
    >
    > Any insight is greatly appreciated!
    >
    > Thanks!
    >
    > kt
    >
    >
    >
    > ------------------------------
    > Karen Turner
    > ------------------------------
    > *Reply to Group Online
    > <https: community.broadcom.com/mainframesoftware/communities/all-discussions/postreply?messagekey="dae16646-bea3-4d24-81dd-97b8cc1776fd&ListKey=6c02d13b-c28e-4794-b86a-c8b767e00498">*
    > *Reply to Group via Email
    > <broadcom_caendevor_dae16646-bea3-4d24-81dd-97b8cc1776fd@connectedcommunity.org?subject=re:+large+vsam+elib+reorg:+repro+vsam+%3e+bdam>*
    > *View Thread
    > <https: community.broadcom.com/mainframesoftware/discussion/large-vsam-elib-reorg-repro-vsam-bdam#bmdae16646-bea3-4d24-81dd-97b8cc1776fd="">*
    > *Recommend
    > <https: community.broadcom.com:443/mainframesoftware/discussion/large-vsam-elib-reorg-repro-vsam-bdam?messagekey="dae16646-bea3-4d24-81dd-97b8cc1776fd&cmd=rate&cmdarg=add#bmdae16646-bea3-4d24-81dd-97b8cc1776fd">*
    > *Forward
    > <https: community.broadcom.com/mainframesoftware/communities/all-discussions/forwardmessages?messagekey="dae16646-bea3-4d24-81dd-97b8cc1776fd&ListKey=6c02d13b-c28e-4794-b86a-c8b767e00498">*
    > *Flag as Inappropriate
    > <https: community.broadcom.com/mainframesoftware/discussion/large-vsam-elib-reorg-repro-vsam-bdam?markappropriate="dae16646-bea3-4d24-81dd-97b8cc1776fd#bmdae16646-bea3-4d24-81dd-97b8cc1776fd">*
    >
    >
    >
    >
    > You are subscribed to "Endevor" as dave.harding@broadcom.com. To change
    > your subscriptions, go to My Subscriptions
    > <http: community.broadcom.com/preferences?section="Subscriptions">. To
    > unsubscribe from this community discussion, go to Unsubscribe
    > <http: community.broadcom.com/higherlogic/egroups/unsubscribe.aspx?userkey="f0102c8e-2c1c-47ba-9f18-e9e6816f2ae3&sKey=KeyRemoved&GroupKey=6c02d13b-c28e-4794-b86a-c8b767e00498">.
    >
    >
    > Copyright © 2005-2023 Broadcom. All Rights Reserved. The term "Broadcom"
    > refers to Broadcom Inc. and/or its subsidiaries.
    >
    > Hosted by Higher Logic, LLC on the behalf of Broadcom - Privacy Policy
    > <https: www.broadcom.com/company/legal/privacy-policy=""> | Cookie Policy
    > <https: www.higherlogic.com/legal/privacy=""> | Supply Chain Transparency
    > <https: www.broadcom.com/company/citizenship/governance-and-ethics#supply="">
    > | Terms of Use <http: termsandconditions="">
    >

    --
    This electronic communication and the information and any files transmitted
    with it, or attached to it, are confidential and are intended solely for
    the use of the individual or entity to whom it is addressed and may contain
    information that is confidential, legally privileged, protected by privacy
    laws, or otherwise restricted from disclosure to anyone else. If you are
    not the intended recipient or the person responsible for delivering the
    e-mail to the intended recipient, you are hereby notified that any use,
    copying, distributing, dissemination, forwarding, printing, or copying of
    this e-mail is strictly prohibited. If you received this e-mail in error,
    please return the e-mail to the sender, delete it from your computer, and
    destroy any printed copy of it.


    Original Message


  • 3.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Posted Sep 18, 2023 02:24 PM

    Excellent suggestion Dave!  Didn't think of the obvious ... test run took 45 minutes:

    MEMBERS INPUT          344205  
    MEMBERS ADDED          344205  
    MEMBERS REPLACED            0  
    MEMBERS FLUSHED             0  
    MEMBERS EXCLUDED            0  
    MEMBER RECORDS OUT   11885654  

    Now I just have to see when the next 2 hour maintenance window is.

    Thanks Again!

    K.



    ------------------------------
    Karen
    [JobTitle]
    [CompanyName]
    [State]
    ------------------------------

    Original Message


  • 4.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Posted Sep 19, 2023 02:47 AM

    I HAVE used the JvK trick and it saved hours!
    But you didn't provide a copy of your ELIB inquire output, that should always be the starting point, in any of these discussions
    Yes, it looks like you have a valid BDAM defn for a PageSize 4096, but IS that the pageSize, or the VSAM record length/size.
    I'd like to see Both the INQuire report and a LISTCAT ENTRY(your.current.elib.cluster) ALL before I comment further.

    But I'd add to JvKs Scenarios to start with a rename of the existing library to .OLD and create the copy in .NEW
    so that while you're reorg is running there is no chance of anybody else using an out of date image.
    With modern zXX machines I'd be surprised if your actual copy took 20 hours anyway - but do check that LSERV and/or RLS aren't getting in your way.



    ------------------------------
    Eoin O'Cleirigh
    Lead Systems Engineer @ ANZ +64273888404
    ------------------------------

    Original Message


  • 5.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Posted Sep 19, 2023 01:06 PM

    Hi Karen,

    We use the BDAM method as our ELIBs are massive. Your record length on the BDAM file was too big I think, it should be 4088. Also we create 2 BDAM files, we REPRO the VSAM file to the 1st BDAM file and then use BC1PNCPY to copy it to the 2nd BDAM file - this does the reorg. Then we repro the 2nd BDAM file to a new VSAM file.

    Here is a sample of the JCL that we use, although I am pretty sure it was copied from Broadcom in the first place :) :-

    //*------------------------------------------------------------------- 
    //*  STEP JS000-- OWN THE ELIB FOR REORG - ACTUALLY A WASTE OF TIME    
    //*  AS THE LAST REFERENCE IS ON THE COPY TO BDAM, AVAILABLE THEREAFTER
    //*  RENAME THE ELIB TO .OLD BEFORE STARTING. CHECK THE NEW ONE        
    //*  THEN RENAME THE .NEW TO THE ORIGINAL                              
    //*------------------------------------------------------------------- 
    //*                                                                    
    //*                                                                    
    //*S000     EXEC PGM=IEFBR14                                           
    //*DB      DD  DISP=OLD,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA               
    //*                                                                    
    //*                                                                    
    //*------------------------------------------------------------------- 
    //*   STEP JS005 -- TIDY UP AFTER RERUN                                
    //*------------------------------------------------------------------- 
    //JS005    EXEC PGM=IEFBR14                                            
    //DD1      DD  DISP=(MOD,DELETE,DELETE),SPACE=(CYL,(1,1)),             
    //             DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM1                  
    //DD2      DD  DISP=(MOD,DELETE,DELETE),SPACE=(CYL,(1,1)),             
    //             DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM2                  
    //*                                                                    
    //*                                                                    
    //*------------------------------------------------------------------- 
    //*   STEP JS010 -- ALLOCATE A NEW VSAM ELIB                           
    //*------------------------------------------------------------------- 
    //JS010    EXEC PGM=IDCAMS,COND=(0,LT)                                 
    //SYSPRINT DD  SYSOUT=*                                                
    //SYSIN    DD  *                                                       
     DEFINE CLUSTER (NAME('SYS7.ENDEVOR.PROD.CCM0.DELTA.NEW') -            
            RECORDSIZE(4088 4088) -                                        
            CONTROLINTERVALSIZE(4096) -                                    
            SHAREOPTIONS(3,3) -                                            
            DATACLASS(DCXVSAM) -                                           
            CYLINDERS(3000,100) -                                          
            NONINDEXED           -                                         
            )      -                                                       
     DATA   (NAME('SYS7.ENDEVOR.PROD.CCM0.DELTA.NEW.DATA'))                
    /*                                                                     
    //*                                                                    
    //*------------------------------------------------------------------- 
    //*   STEP JS030 -- ALLOCATE 2 BDAM ELIBS (FOR OLD AND NEW)            
    //*------------------------------------------------------------------- 
    //JS030     EXEC PGM=IEFBR14,COND=(0,LT)                               
    //DD1      DD  DISP=(NEW,CATLG),                                       
    //             UNIT=SYSDA,SPACE=(CYL,(3000,),,CONTIG),                 
    //             DCB=(DSORG=DA,BLKSIZE=4088,LRECL=4088,RECFM=FBS),       
    //             DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM1                  
    //DD2      DD  DISP=(NEW,CATLG),                                       
    //             UNIT=SYSDA,SPACE=(CYL,(3000,),,CONTIG),                 
    //             DCB=(DSORG=DA,BLKSIZE=4088,LRECL=4088,RECFM=FBS),       
    //             DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM2                 
    //*                                                                   
    //*-------------------------------------------------------------------
    //*   STEP JS030 -- INITIALIZE NEW BDAM ELIB USING BC1PNLIB UTILITY   
    //*-------------------------------------------------------------------
    //JS020    EXEC PGM=BC1PNLIB,COND=(0,LT)                              
    //ELIB1    DD  DISP=OLD,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM2        
    //NDVTAG   DD  DUMMY                                                  
    //SYSIN    DD  *                                                      
    INIT       DDNAME = ELIB1                                             
               PAGE SIZE = 4088                                           
               ALLOCATE = (539999,18000)                                  
               RESERVE PAGES = 500                                        
               DIRECTORY PAGES = 16000                                    
    .                                                                     
    //BSTERR   DD  SYSOUT=*                                               
    //SYMDUMP  DD  DUMMY                                                  
    //SYSPRINT DD  SYSOUT=*                                               
    //*                                                                   
    //*-------------------------------------------------------------------
    //*   STEP JS040 -- REPRO TO BDAM ELIB                                
    //* - ELIB ELEMENTS AND FRAGMENTS ARE COPY TO BDAM ELIB.              
    //*-------------------------------------------------------------------
    //JS040    EXEC PGM=IDCAMS,COND=(0,LT)                                
    //INFILE   DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.OLD          
    //OUTFILE  DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM1        
    //SYSPRINT DD  SYSOUT=*                                               
    //SYSIN    DD  *                                                      
     REPRO  IFILE(INFILE) OFILE(OUTFILE)                                  
    //*                                                                   
    //*                                                                   
    //*-------------------------------------------------------------------
    //*   STEP JS050 -- COPY TO THE NEW BDAM ELIB                         
    //* - ONLY ELIB ELEMENTS ARE COPY TO THE NEW BDAM ELIB.               
    //* - ELIB COPY BETWEEN BDAM ELIB IS FASTER THAN THAT WITH VSAM ELIB. 
    //*-------------------------------------------------------------------
    //JS050    EXEC PGM=NDVRC1,PARM='BC1PNCPY',COND=(0,LT)                
    //NDVTAG   DD  DUMMY                                                  
    //IN       DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM1        
    //OUT      DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM2        
    //SYSPRINT DD  SYSOUT=*                                               
    //BSTERR   DD  SYSOUT=*                                               
    //SYSUDUMP DD  SYSOUT=*                                               
    //ABNLIGNR DD  DUMMY                                                  
    //SYSIN    DD  *                                                      
     COPY   INPUT  DDNAME IN                                              
            OUTPUT DDNAME OUT                                             
           .                                                              
    /*                                                                    
    //*-------------------------------------------------------------------
    //*   STEP JS060 -- REPRO BDAM TO VSAM                                
    //* - ELIB ELEMENTS AND FRAGMENTS ARE COPY TO BDAM ELIB.              
    //*-------------------------------------------------------------------
    //JS060    EXEC PGM=IDCAMS,COND=(0,LT)                                
    //INFILE   DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.BDAM2        
    //OUTFILE  DD  DISP=SHR,DSN=SYS7.ENDEVOR.PROD.CCM0.DELTA.NEW          
    //SYSPRINT DD  SYSOUT=*                                               
    //SYSIN    DD  *                      
     REPRO  IFILE(INFILE) OFILE(OUTFILE)  
    //*  

    Regards                     



    ------------------------------
    Jim
    ------------------------------

    Original Message


  • 6.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Posted Sep 19, 2023 01:21 PM
    Thanks Jim and Peter for pointing out my lrecl discrepancy, I’ll give that a try and add it to my copy of the KD. (which, with the exception of Jan’s BDAM/VSAM paragraph, Roberta and I wrote back in 19/20??) 😊
    KT


    Original Message


  • 7.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM

    Broadcom Employee
    Posted Sep 19, 2023 03:52 AM

    Hi Karen,

    I used to test this and it should work.
    Can you confirm you initialized the target BDAM with the same attributes than the source ELIB ?

    Regards,
    Ollivier


    Original Message


  • 8.  RE: Large VSAM ELIB Reorg: REPRO VSAM > BDAM