Idea Details

JCLCheck integrated cross-reference repository

Last activity 09-27-2018 07:02 PM
michael.schmitt's profile image
09-27-2018 07:02 PM

One thing developers and production support people want to know is where is a data set used in a JCL jobstream? Where is a (high level) program used?  Where is a proc used?  This is part of impact analysis, so the developer can know what needs to change for a program change. And finding where a data set is used is vital for tracing the flow of data through a system.


JCLCheck can answer these questions when you scan one or more jobs: it creates Data Set and Program Cross-Reference reports.  But what it doesn't do is create a searchable repository of the cross-reference results. This means that to do this analysis for an entire system, you'd have to first scan the entire system. And then either save the result (and re-scan when any job change), or do the scan all over again each time you need to query.


This is why JCL checking products sometimes have a repository product.  ASG's companion to JOB/SCAN is DOCU/TEXT.  And CA has a companion to JCLCheck, called APCDOC.


The problem with APCDOC is that a) it is too much product, and b) it is no longer actively supported by CA.  While no "stabilization date" date has been announced, the last release of APCDOC was in 1999(!), and since then it is difficult to get fixes applied.  (And since APCDOC uses JCLCheck internally for its JCL scanning, APCDOC is brittle: changes to JCLCheck tend to break it in obscure ways.)


As for "too much product", what this means is that APCDOC is big. It has a bunch of VSAM databases in its repository, and complex ISPF interface, the ability to run queries in both batch and interactively ISPF, the ability to store the reports for later use and so on. And, it was created back when the operators needed to create runbooks and tape pull lists. And APCDOC can be used a documentation repository. And much more.


But, we still have the need identified above: the ability to query for data set, program and proc cross-references.


This Idea is that JCLCheck should integrate a limited set of this type of functionality directly into JCLCheck.


I'm not saying that JCLCheck should clone all or part of APCDOC. There is no requirement to maintain the same APCDOC interfaces, report formats, database structures, complex ISPF interface, or anything else. Just that the feature of a cross-reference repository should be in JCLCheck.


The requirements are:

  1. A data store or repository to remember the results of JCLCheck scans.  That can be whatever CA wants to use: VSAM, PDS, ISPF tables, integrated database, whatever.  "Remember results" doesn't mean all the results; just what JCLCheck needs to remember in order to query later (see below).
  2. When jobs are scanned, the user has an option to have JCLCheck save the cross-reference data into the store.
  3. The store doesn't need to have a documented structure; it is internal to JCLCheck.
  4. For security and other reasons, it should be able to use different stores, not just one store per site.  For example, there may be multiple customers running on the same system, and we don't want the store to mix the data.  (In APCDOC you pass it the high level node of the VSAM files; that designates a particular store.)
  5. When we scan the jobs, we should be able to designate a "system" (or some similar name) that the JCL belongs to.  This may correspond to a job stream. If we're querying the cross-reference for which jobs use a program, we'd want to be able to distinguish between the test system and production, for example.
  6. It should be able to update the store via delta scans.  So if the JCL for job A changes, I should be able to re-scan job A by itself, and have it update the store. We don't want to have to re-scan the entire system just because one job changes.
  7. There should be a way to delete obsolete jobs out of the store.
  8. There needs to be a way to query the store. I don't mean doing SQL, just a defined syntax to generate a report.  Such as:  give me the data set cross-reference for data sets matching pattern X for jobs matching name pattern Y in system Z.
  9. The data set cross reference report should include: data set name, generation (e.g. was it +1, 0, etc.), job name, job step, step name, program name of that step, DD name, disposition.
  10. We don't require it to try and track the order jobs run, as APCDOC does.
  11. We'd like to have a data set and program cross reference.  (Program means the same as what JCLCheck identifies in the program cross reference).  A proc cross-reference would also be useful.  JCLCheck also does a report cross-reference but for various reason, I've never found that useful.
  12. JCLCheck could have the ability to use the proc cross-reference to drive scans.  By this I mean, if it has a store, then you could say, "scan all the jobs that use this proc", such as when the proc is changed.  Currently there's no way to do this; you have to figure out what jobs are affected yourself.
  13. We'd like to be able to query for where a library MEMBER is used.  (APCDOC can't do that.)
  14. You should be able to run the query from batch.


And that's it.  If JCLCheck integrates a limited set of cross-reference functionality directly into the product, it makes JCLCheck a better and more competitive product.


And, it would decrease the need for keeping APCDOC alive. If nothing else, having this Idea would mean that it would be easier to maintain the cross-reference repository and query, since it would all be inside JCLCheck.