SYSVIEW Performance Management

 View Only
Expand all | Collapse all

CA Sysview R15.0 Feature 5 PTF Logstream questions

  • 1.  CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Jan 03, 2018 11:38 AM

    We are working on installing the Feature 5 PTF for Sysview.
    About the logstream changes, in case of backout, does the logstream need to be deleted and redefined, or will the prior release, R14.1, just read past or over the Logstream data created while it was under Release 15.0?

    Also, about multi-block Logstream processing, can you briefly explain exactly what that means and how it is different from the prior logstream setups in say, R14.1?

    What types of performance improvements should we expect to see, or are these simply functionality improvements?

    Will this provide any drastic performance improvements for extremely large volumes of data, specifically the CICS Transaction Detail logstream?

    Please contact me directly if you need specific details about our site.



  • 2.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Broadcom Employee
    Posted Jan 03, 2018 02:44 PM

    Hi Peter,

     

    The log stream does not need to be deleted and redefined. There are, however a few points that should be noted:

     

    • For on-line SYSVIEW commands such as CTRANLOG, IMSTLOG, etc:
      • SYSVIEW 15.0 + PTF5 can read log stream data created from SYSVIEW 15.0 + PTF4 and prior releases (including SYSVIEW 14.1).
      • SYSVIEW 15.0 + PTF4  and prior releases (including SYSVIEW 14.1) can NOT read SYSVIEW 15.0 + PTF5 log stream data.
      • If SYSVIEW 15.0 + PTF4  and prior releases (including SYSVIEW 14.1) finds a SYSVIEW 15.0 + PTF5 record  it does not recognize, it will simply skip it, and it will not be displayed.
    • For the SYSVIEW logger exit GSVXLGEX:
      • If SYSVIEW 15.0 + PTF4  and prior releases (including SYSVIEW 14.1) finds a SYSVIEW 15.0 + PTF5 record  it does not recognize, the logger exit will abend.
      • There are no plans to provide toleration for SYSVIEW 15.0 + PTF5 records in prior versions of SYSVIEW.
      • If using the logger exit, then yes, a delete and redefine will be required to be able to report and avoid abends in the logger exit.

     

    Also, about multi-block Logstream processing, can you briefly explain exactly what that means and how it is different from the prior logstream setups in say, R14.1?

     

    • Prior to SYSVIEW 15.0 + PTF5, SYSVIEW would make 1 write request to the system logger for every record it was writing. Also, it would make 1 read request to the system logger for every read request. We observed that, in large part, the latency in reading from a log stream was the time it took for the system logger to return the record. We also noticed that reading larger chunks of data from the system logger at one time did not significantly cause any more delay. Because of this, we are now blocking multiple SYSVIEW records into a single system logger block. This has a few benefits:
      • Less contention for system logger resources as less call are being made to it.
      • Less CPU used by SYSVIEW.
      • Faster throughput to system logger and less buffering of records in SYSVIEW.
      • Faster reading of records for redisplay in SYSVIEW on commands such as CTRANLOG.
    • This set up required us to change the default block size from 32K to 64K to allow SYSVIEW to write as large a block as possible.
    • This set up also required us to recommend a SMS data class for the off load VSAM files to be 24K (up from default 4K) to be able to read data sequentially much more quickly given the larger block size.

     

    What types of performance improvements should we expect to see, or are these simply functionality improvements?

     

    • Performance can vary vastly depending on how resource constrained the system logger is. I will not quote any numbers because it can vary greatly, but in general, we found that with PTF5, the system logger is much less busy and more able to serve read and write requests in a timely fashion. We observed a sizeable decreases in both elapsed time and CPU time per record read.

     

    Will this provide any drastic performance improvements for extremely large volumes of data, specifically the CICS Transaction Detail logstream?

     

    • Again, results will vary, but in general, there is a large improvement, yes. Will you be able to read 5 days of records from CTRANLOG on a production system in a few seconds? No. But I would expect requests that were regularly taking several minutes should take much less time.


  • 3.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Jan 03, 2018 03:25 PM

    Thanks for the explanation.

    Not the best scenario in case we need to backoff, to have to delete monitoring data, but at least it does appear to be limited in scope.

    As you know, our CICS transaction detail is enormous, and only houses two to three days of production data for that reason, where a best case scenario would be three to 5 days, and preferably offloadable as you can do with a REPRO in a KSDS file.

    Still, we'll see where this leaves us.

    In your write up, it seems that the larger MAXBUFSIZE is more of a requirement than an option.  Is that correct.  We are currently defined at the smaller size from our original installs in 2013, so we would need to change that.



  • 4.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Broadcom Employee
    Posted Jan 03, 2018 03:57 PM

    Hi Peter,

     

    I used the "require" word too strongly there. Neither the CI Size change nor the MAXBUFSIZE changes are strictly required, but we strongly recommend them as it further helps SYSVIEW read and write much more quickly. SYSVIEW will still write multi-record blocks with PTF5 to a CI Size 4K/MAXBUFSIZE 32K log stream at no performance detriment compared to your 14.1 install, but a CI Size 24K/MAXBUFSIZE 64K is more efficient/faster in comparison.



  • 5.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 05, 2018 12:28 PM

    We were looking at some of this and we noticed the dataclas settings that are being used are blank in ISMF.  A LISTCAT on the logstream shows the CI-SIZE as 4K .

    Do you know if or where we can check to determine if this is some type of default? 

    Can CI-SIZE be specified via the logstream definition, or is this purely a function of DataClass?

    Some of our LS_SIZE and STG_SIZE we well over 4K, actually approaching 4GB .  I am taking it that the CI-SIZE is different?




  • 6.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Broadcom Employee
    Posted Feb 05, 2018 12:51 PM

    Hi Peter,

     

    If a default is not specified for a CI-SIZE in ISMF, then you will see a blank, however, the default is always 4K as you observed with LISTCAT.

     

    The default CI-SIZE of 4K is documented in a few different places in IBM documentation, but it is also in the help for the "CISIZE DATA" column in ISMF as well.

     

    CI-SIZE can not be specified via log stream definition. It must be specified via an SMS data class definition whose CI-SIZE is set to the desired size.

     

    LS_SIZE is the size of an offload data set. For most log streams, there are a maximum of 168 of these. 

     

    STG_SIZE is the size of the in-memory (and on-disk in many cases) staging area logger uses to buffer data before it offloads the data to an aforementioned offload data set.

     

    CI-SIZE, which has much less to do with the log stream and more to do with how the operating system asks for data from disk, is basically the amount of data the OS will ask disk for at a time. In other words, we are recommending that the CI-SIZE be changed from 4K to 24K to reduced the number of times the system logger needs to go to disk and ask for data. This is important to improve the rate of read from the log stream. This has no affect on how much data the log stream can store. It only affects how the log stream moves data around, in this case more efficiently.

     

    Jason Brosius

    CA SYSVIEW



  • 7.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 05, 2018 01:27 PM

    We probably need to make the dataclas change separately.  Can this be done via an override, or does the logstream need to be deleted and rebuilt?  If a rebuild is not required, will the data with the old 4K CI-Size be accessible since the newer data will be at the new number?

    While our largest logstream regularly rebuild after about 3 days due to the amount of activity, smaller ones, such as the Auditlog might site for over a year or two before old data is purged.




  • 8.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Broadcom Employee
    Posted Feb 05, 2018 01:42 PM

    The log stream can updated dynamically with an UPDATE command in the IXCMIAPU utility, detailed here:

     

    https://docops.ca.com/ca-sysview/15-0/en/administering/use-the-logger/log-stream-optimization

     

    There is no DELETE/DEFINE required.

     

    After you update the log stream's LS_DATACLAS() definition, it will take affect the next time your log stream needs to allocate another offload data set. For your CICS transaction log stream, this might occur with hours/minutes. For your other low volume log streams, it could take some time as you suggested. (Also note, all log stream connections need to be broken with the log stream in order for the UPDATE to take affect. You can verify this from the LGCONN command. You will need to stop SYSVIEW for this to occur, and ensure no TSO users are on a log stream command as well.)

     

    The old 4K CI-SIZE data is compatible with the 24K CI-SIZE data. The CI-SIZE is simply how much data is taken from disk at one time, it does not affect how the end-point application sees the data, nor does it affect the system logger.

     

    Jason Brosius

    CA SYSVIEW



  • 9.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 20, 2018 09:51 AM

    We are running some tests now in a sandbox before setting this up in production.
    Another question came up while looking at this.

     

     

    Most of what I read refers to the setting for "LS_DATACLAS".

     

     

    Should we also apply the dataclas change to "STG_DATACLAS", or is that not needed?



  • 10.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 23, 2018 02:48 PM

    Jason,

     

    Can you please answer my one outstanding question about STG_DATACLAS, above #SYSVIEW #logstream



  • 11.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions
    Best Answer

    Broadcom Employee
    Posted Feb 26, 2018 08:25 AM

    Apologies Peter. There was no notification for your response. 

     

    LS_DATACLAS should be changed to the CISIZE 24K as talked about previously.

     

    STG_DATACLAS should NOT be changed to CISIZE 24K. In fact, you will get errors if you do this. The stage must have a CISIZE of 4K (the default), else the log stream will fail to allocate when SYSVIEW starts. This is a IBM system logger requirement.

     

    Regards,

    Jason Brosius

    CA SYSVIEW



  • 12.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 26, 2018 08:28 AM

    Thank you!

     

    If there is something else I need to do to trigger a notification, please let me know.

     

    It’s hard enough to find it every time, but once I got in there, I looked and could not find a setting to do that.

     

    Peter T. Brown

    Phone: 412-236-0429



  • 13.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Broadcom Employee
    Posted Feb 26, 2018 09:07 AM

    Hi Peter,

     

    I don't believe there is anything for you to do on your end to trigger a notification. On our end, we need to "follow" the associated Communities area or the specific thread.

     

    Frank.



  • 14.  Re: CA Sysview R15.0 Feature 5 PTF Logstream questions

    Posted Feb 26, 2018 09:17 AM

    Thanks.

     

    I believe Jason DOES.  That’s why this was odd.

     

    Peter T. Brown

    Phone: 412-236-0429