CA OPS/MVS

I was hoping someone could run me through the new logic for Hooked tasks in SSMv3. I know there is the checkbox in policy editor to tell SSM it is a hooked task and I know the process is internal but there doesn't seem to be much documentation on how exactly things worked. At least in SSMv2 there was a distinct and traceable path via the rules and tables to see what was going. However it was this complexity that led me away from using hooked tasks. Now that SSMv3 has made it simpler, I was hoping to make use of them (say for CAS9/CA-RIM). I think I am halfway from point A to point B so if someone could kind of flow chart the process for me here so I can confirm what I know and learn what I don't, I would appreciate it.

Hello Travis,

This is Cesar Molina from Customer Support Level 1 team.

At this moment I do not have the information you are requesting at the detail you needed.

All I can confirm is what you probably already know when you tried yourself the CAS9 JOB defined via the policy manager as a hook type resources. I can see dynamic rules being created/firing and states being updated without the need to use the AOF REQ rule SSMHOOK we used to run under SSMv2.

With above being said we would like to have a ticket open so we can discuss this with you.

I already have a Sustaining Engineer committed to work on your case as soon as it arrives.

Since this Community has global visibility your case will be handled with the priority you wish us to respond.

Let me know if this all sounds like a plan Travis.

Regards, Cesar

Note: To the rest of the audience please feel free to update this thread if you are interested to have a summary on this Community post once the case with Travis is addressed.

Documentation of the function certainly would be helpful.  It does not appear to be specified in any detail in any CA documentation and I'm really disappointed that this thread wasn't completed with the pertinent information.  Hook tasks are being reported as failed in SSM table and they functioned perfectly.  Precisely why they are seeing this on an SSMv3 system is not clear: 

HSS0001I HSS SUBSYSTEM 'HSS#' HAS BEEN INITIALISED
OPS7914T SSMv2 AUDIT: STCTBL.EACRLOD UPDATED by EACRLOAD SSM.KMSGHSS CURRENT_STATE=UP

-EACRLOAD ENDED. NAME- TOTAL CPU TIME= .00 TOTAL ELAPSED TIME= .07
EACRLOAD S0660133 ENDED MAXCC=00000 SUBSYS=STC
$HASP395 EACRLOAD ENDED - RC=0000

OPS7914T SSMv2 AUDIT: STCTBL.EACRLOD UPDATED by *MASTER* SSM.SSMEOM CURRENT_STATE=TERM
OPS7914T SSMv2 AUDIT: STCTBL.EACRLOD UPDATED by *MASTER* *DYNAMIC.ESSMEZ CURRENT_STATE=FAILED

I apologize for not updating this thread but it kind of dropped off my radar. I did find something interesting as I spoke with various OPS gurus. The SSMEOM rule provided by CA uses the variable GLOBAL1.SSMHOOKS which is set using the HOOKED rule mentioned earlier. I modified the SSMEOM rule to now check the SSM#HOOK column instead and that helped a lot.

Edited for clarification.

I'll be verifying this in about an hour or so, but the reply I got from the CA tech assigned to the conversion at our shop is that in the Inactivation Event field you enter 'NOACTION' to get past the Policy Manager requirement that something is there... so after you load it to the STCTBL you go into the Resource Edit, display the fields not shown you'll see SSM#INACTMP = NULL.  If you go back into Policy Manager and edit the resource, all the Inactivation Event fields will be blank.  I'm going to be recycling the test system in a few so we'll know then. 

Interseting, I was not aware of the NOACTION option. I was looking for some sort of functionality like that when setting up SUBTASKS (Tasks controlled by a parent, PDSMANDB for example). I couldn't find a way to put something in for the start and stop commands that didn't "do something". I have been using MVSCMD(DELETE ME) to get around it but then each time a SUBTASK would be loaded you would have to remember to modify the action table and delete the MVSCMD(DELETE ME)'s. The NOACTION option seems to be what I was looking for. I'll have to double check the manual but I don't recall seeing that as an option but maybe I overlooked it specifically because it said NOACTION.

As I was typing this I went and tried putting it in to a user created type. I initially tried using it for the stop and start command fields but it came back with an "INVALID ACTION CLAUSE" error. I reread what you wrote and realized that you said to put it in the INACTIVATION COMPLETE EVENT field. That should work for HOOKED tasks assuming it leaves the fields blank as noted above. You also need to make sure the STOP action is SETCOL("CURRENT_STATE,DOWN") so SSM doesn't try to shut it down.

Hello Travis and Ambros,

You should not use the rule SSMEOM with SSMV3. For tasks that in previous releases used the SSMEOM rule you should update the "Inactivation Event" in the Policy Manager with "Event type" EOM and "Event id" equal to the resource name. SSMV3 will create dynamic rules that will detect de inactivation of each resource. After you update and load all resources in the Policy Manager you must disable the rule SSMEOM.

Regards,

Mario

Carlos Mario Filho
Principal Support Engineer
CA Technologies

I initially tried to take advantage of the dynamic EOM rule that is created but that rule does not handle hooked tasks correctly. All it does is set the CURRENT_STATE to DOWN which if the task is HOOKED and other resources depend on it being in an UP state doesn't work. Here is the code that was generated by a resource I called DUMYTASK. I entered the EOM type and ID just as you described. This code is the same for every task, including hooked tasks.

)EOM * (4)
)PROC
job = eom.jobname
If job = 'INIT' then Return
statetbl=OPSINFO('STATEMANTABLE')
statemode=OPSINFO('STATEMANSTATUS')
ADDRESS SQL "SELECT MANAGED_TABLE TABLE_MODE INTO mt tmo FROM" statetbl
Do i = 1 to mt.0
   "SELECT DESIRED_STATE MODE SSM#INACTCMP" ,
   "INTO dst mode icmp FROM" mt.i "WHERE JOBNAME = :job"
   If sqlcode <> 0 Then iterate
   (1) If POS('TYPE(EOM)',icmp.1) = 0 Then iterate
   (2) If dst.1 = UP & statemode <> PASSIVE ,
   & tmo.i <> PASSIVE & mode.1 <> PASSIVE Then
      c_state = FAILED
   Else c_state = DOWN
   (3) "UPDATE "mt.i" SET CURRENT_STATE = :c_state" ,
   "WHERE JOBNAME = :job"
End
Return

1. This IF statement checks to see if TYPE(EOM) is in the SSM#INACTCMP field. If not, it iterates. If it is then we fall through to the next statement.

2. This statement is checking the mode of stateman, the mdoe of the table and the mode of the task. It also checks to see if the desired state is UP. This is the most important of the checks for purposes of this conversation. So assuming all the mode checks are TRUE and the task's DS=UP it will set the c_state to FAILED.

3. Finally we set the CS of the task to DOWN.

4. Even though I set the ID to DUMYTASK, the EOM event is still generic.

So let's look at the sequence of a HOOKED task:

START HOOKTASK, CS=STARTING, DS=UP

HOOKTASK initializes and does whatever it needs to do.

STC HOOKTASK (EOM EVENT)

The EOM rule above fires and following the logic I detailed above it sets the CS=DOWN and with the DS still UP. This means that the HOOKTASK will now attempt to restart and eventually move to a FAILED state based on the restart information. At no point does the HOOKTASK ever enter an UP_UP state which is what we want. The dynamic EOM rule above doesn't check if the task is hooked.

A hooked tasks should set the CURRENT_STATE to UP on EOM or at the very least UNKNOWN so the SSMSTATE rule can process it and perform a check via SSMSTAUX. This allows a task like CAS9 be a prerequisite for CA resources. The SSMv2 SSMEOM rule does take HOOKED tasks into account but where it gets the information about whether a task is HOOKED or not is not valid for SSMv3. It uses a global variable which relies on SSMv2 Hooked Task Processing. Thus I modified this rule to find the information by looking at the SSM#HOOK column. Once I did that, my hooked tasks' current state was set properly.

If we are to use the dynamic EOM rule I think it needs to take into account hooked tasks and the ID needs to be included. Maybe this is a bug, maybe an oversight but either way, it does not work for a hooked task.

Hi Travis,

A hook task won't have the string "TYPE(EOM)" in the SSM#INACTCMP field so the dynamic EOM rule won't take any actions upon this type of tasks. 

Regards,

Mario

Carlos Mario Filho
Principal Support Engineer
CA Technologies

Not true. I just created a HOOKTASK entry from scratch and this was the result:

KEYS                  <SSM#ACTCMP        SSM#INACTCMP
HOOKTASK                                           TYPE(EOM) ID(HOOKTASK) TEXT()

However it did create a new EOM rule albeit still with the generic ID:

)EOM *
)PROC
job = eom.jobname
If job = 'INIT' then Return
statetbl=OPSINFO('StatemanTable')
ADDRESS SQL
"SELECT managed_table INTO mt FROM" statetbl
Do i = 1 to mt.0
"SELECT DESIRED_STATE SSM#ACTCMP INTO dst acmp" ,
"FROM" mt.i "WHERE JOBNAME = :job"
If sqlcode <> 0 Then iterate
If POS('TYPE(EOM)',acmp.1) = 0 Then iterate
c_state = UP
"UPDATE "mt.i" SET CURRENT_STATE = :c_state" ,
"WHERE JOBNAME = :job"
End
Return

Notice the SQL statement. That is not the SSM#INACTCMP column. Interesting. I will have to do more research after lunch.

Edit After Lunch:

I had a chance to analyze the code a bit more closely now and 2 things have changed. First, it no longer checks the status of Stateman or the mode of the resource. While these are not necessary for our discussion on this thread, it might not be a bad thing to include them unless of course it can be logically shown that because the dynamic rule exists, SSM must be active, etc. Second, the SSM#ACTCMP column has replaced the SSM#INACTCMP column in the SQL statement. Theoretically, SSM#ACTCMP should never have TYPE(EOM) in it and so the POS function will always return 0 and then iterate. Thus, even in this rule, the CURRENT_STATE is never set to up because it ends up never being set at all. Now if we put EOM in for the ACTCMP Event it does show up in the table and thus the POS function would return something > 0 and then set the c_state variable to up which then works to set the hooked task CURRENT_STATE correctly. The question then becomes, does this work for most, if not all, hooked tasks? 

The fact that the ID isn't being placed in the EOM rule spec also bothers me. I specifically entered it and it is still generic. This means that it will fire for every EOM event and assuming there are multiple EOM rules for multiple tasks the rules may step on each others' toes. That seems like a bug to me. The fact that I have two different source code listings for two tasks that I set to HOOKED seems to be an issue as well. It may be something to look at.

Hi Travis,

How exactly are you defining the hook tasks in the Policy Manager? Perhaps you should open a case at this time as I suspect the problems you are having are due to the way you are defining those tasks.

Regards,

Mario

Carlos Mario Filho
Principal Support Engineer
CA Technologies

I will use CAS9 as an example but I used this for setting up other hooked tasks like TMSINIT as well.

JOBNAME: CAS9

SSMHOOK = Y

START CMD: MVSCMD("START CAS9")

SSM#ACTCMP = *.SSMCAAPI

STOP CMD: SETCOL("CURRENT_STATE,DOWN")

SSM#INACTCMP = *.SSMEOM (Modified)

Hi Travis,

SSM#INACTCMP should be NULL for hook type tasks.

Regards,

Mario

Carlos Mario Filho
Principal Support Engineer
CA Technologies

And how do you set it to NULL in Policy Manager? As far as I can tell Policy Manager enforces the fields such that they are not blank and must be a valid value so a value of NULL would fail. Maybe an example definition would help.

Additionally what you mentioned earlier:

"You should not use the rule SSMEOM with SSMV3. For tasks that in previous releases used the SSMEOM rule you should update the "Inactivation Event" in the Policy Manager with "Event type" EOM and "Event id" equal to the resource name. SSMV3 will create dynamic rules that will detect de inactivation of each resource. After you update and load all resources in the Policy Manager you must disable the rule SSMEOM."

Hello Travis,

You can just left "Inactivation Completion Rule" and "Inactivation Completion Event" as blanks. They  are not required fields.

You are right regarding my previous statement. SSMV3 will generate only one generic EOM dynamic rule but it won't affect hook type tasks because they won't have the string "TYPE(EOM)" in the SSM#INACTCMP field.

Regards,

Mario

Carlos Mario Filho
Principal Support Engineer
CA Technologies

Actually they ARE required fields. I am looking at the screen(s) right now and this is what I am seeing:

* Required field

* Action to START the resource:

* Activation Complete Event:

* Action to STOP the resource:

* Inactivation Complete Event:

If I try to leave ANY of those fields blank or place something in them that is not valid I get an error and SSM PM will not let me proceed. In some cases, I haven't even been able to F3 to exit out because it is performing validation before exiting.

Lot of confusion here and fully agree because of lack of quality documentation/guidance.  Primary thing to remember here is that IF you were a pre-existing SSM user before the Policy Manager utility was developed AND you CHOSE to move to it to add/delete/change attributes of your SSM resources, then its one way or the other specifically with the managing of HOOKS. If you chose to go to PM, then you will need to disable the SSMEOM rule as it has logic to manage hook jobs set via user customizable SSMHOOK setup. If you implement other  user logic into SSMEOM, then you would have to create that logic in your own type of EOM rule (exclude logic to manage hooks) and probably incorporate CS=DOWN/FAIL logic to handle all resources and then specify this rule in the INACTIVATION rule fields. Ideally, if you were an established SSM user prior to PM , specifically a user of SSM and you did not do the original setup, then you are better off staying at V2 and not using PM. If you still pushed forward and CORRECTLY disabled old SSM functionality  prior to going to SSM v3 and PM setup , meaning disabled SSMEOM, now using v3 SSMBEGIN, removed SSMHOOK, etc, then follow these steps to set a 'HOOK' job within PM:

Setting up Hook jobs in PM V3 (assuming user has disabled all pre PM Logic if old user and went to PM)

Activation setup steps for all hooks:

1.SSMHOOK=Y

2.Activation command = START &JOBNAME or whatever to start

3.Activation  Complete Event = MSG msgid any text data (if CS=UP generated) or EOM jobname (if no CS=UP msg AND you simply assume EOM=UP)

Inactivation setup steps:

For a Hook that does not issue a STOP command/process (IPL’d over)

1.Inactivation command = SETCOL CURRENT_STATE = DOWN

!!!!! Do not specify anything in Inactivation Complete Event!!

For a hook that does have a STOP process (some Modify command, or START of some stopping STC)   

1.Inactivation command = MVSCMD or PGM invocation

2.Manual CS=DOWN rule needs created (fire on msg of all done, or eom that fires on same hook jobname with logic to set CS=down in shutdown mode such as check on step of job?,IPL variable set?,etc) and specify this rule in Inactivation complete rule field  

If you are still having issues with setting up a HOOK job after following these steps, then open up an OPS/MVS support case, forwarding hook name, start command, and CS=UP msg event, and stop command/process, as well as archived OPSLOG that has a manual start/stop of the task.  

Thank you for the detailed explanation. This is what needs to be in the manuals.