Clarity PPM1

Expand all | Collapse all

How to invoke custom Java class from GEL script

Jump to Best Answer
  • 1.  How to invoke custom Java class from GEL script

    Posted 08-16-2019 12:49 AM
    Edited by Pavan Bugata 08-16-2019 10:51 AM
    Hi,

    I have a use case - a sequence of three shell scripts to be executed on Linux box. I got the java jar deployed under clarity/lib which accepts the parameters. So I need to pass DB password from properties.xml and shell script name/s (as an ArrayList). Is there a way to invoke the Java class with parameters from GEL script? 

    [Edited]
    My goal is to run the shell scripts in sequence on Linux box. If we cannot invoke Java class from GEL, please let me know if we can run shell scripts from GEL directly.

    Thanks,
    PB


  • 2.  RE: How to invoke custom Java class from GEL script
    Best Answer

    Posted 08-17-2019 01:51 AM
    Yes you can call java from GEL, and you can either choose to do that directly with your jar - since it's placed in either clarity/lib or clarity/customlib it is normally automatically added to the classpath when the app/bg services start.

    What you do with it would not be supported; it's your own code after all and nobody elses, and generally speaking whilst things like this might work in the SaaS environments, calling scripts would be deeply frowned upon and discouraged.

    Since you put the jar files in clarity/lib though, I will assume you're on-premises and not SaaS.

    I don't have Clarity installed on Linux currently to demo right now, but I can do it from Windows and it's pretty much the same deal in other OS's so long as you use appropriate script names and contents.  It should go without saying that these ought to be non-interactive scripts that cannot stop and prompt for any inputs, or else they may get stuck in limbo and cause the process engine to time out over the long-running (stuck) script, which will then bring down the process engine for everything.

    With the disclaimers out of the way, if I have a script like this called external_script.bat (and we assume for now it's placed in the clarity/bin folder although I'd suggest putting it some place else):

    @echo off
    echo Called by a gel script > logfile.txt
    echo Shhh, don't tell anyone the %1 >> logfile.txt
    echo Do I talk too much?  Well you said it was ok right? %2 >> logfile.txt​

    If I create a GEL script like this and run it:

    <gel:script 
      xmlns:core="jelly:core"
      xmlns:gel="jelly:com.niku.union.gel.GELTagLibrary"
    >
    
    <!-- example script name and parameters -->
    <core:set var="dbPwd" value="secret" />
    <core:set var="scriptName" value="external_script.bat" />
    <core:set var="verbosity" value="--verbose" />
    
    <!-- make a 1D array of objects needed for invoke(obj, string...) call -->
    <core:invokeStatic var="stringClass" className="java.lang.Class" method="forName">
      <core:arg type="java.lang.String" value="java.lang.String"/>
    </core:invokeStatic>
    
    <core:invokeStatic var="stringArray" className="java.lang.reflect.Array" method="newInstance">
      <core:arg type="java.lang.Class" value="${stringClass}"/>
      <core:arg type="int" value="${3}"/>
    </core:invokeStatic>
    
    <!-- assign the array items -->
    <core:invokeStatic className="java.lang.reflect.Array" method="set">
      <core:arg type="java.lang.Object" value="${stringArray}" />
      <core:arg type="int" value="${0}" />
      <core:arg type="java.lang.Object" value="${scriptName}" />
    </core:invokeStatic>
    
    <core:invokeStatic className="java.lang.reflect.Array" method="set">
      <core:arg type="java.lang.Object" value="${stringArray}" />
      <core:arg type="int" value="${1}" />
      <core:arg type="java.lang.Object" value="${dbPwd}" />
    </core:invokeStatic>
    
    <core:invokeStatic className="java.lang.reflect.Array" method="set">
      <core:arg type="java.lang.Object" value="${stringArray}" />
      <core:arg type="int" value="${2}" />
      <core:arg type="java.lang.Object" value="${verbosity}" />
    </core:invokeStatic>
    
    <!-- create builder -->
    <core:new className="java.lang.ProcessBuilder" var="builder">
      <core:arg value="${stringArray}" />
    </core:new>
    
    <!-- start the script -->
    <core:invoke on="${builder}" method="start" var="proc" />
    
    </gel:script>

    I get a logfile.txt created with the following contents:

    Called by a gel script
    Shhh, don't tell anyone the secret
    Do I talk too much?  Well you said it was ok right? --verbose​

    The tricky part of this, if there is to be one, is that ProcessBuilder's constructor takes either a List<String> or String... args list; and constructing those in GEL as you can see takes a disproportionate amount of effort for the results (compared to plain java equivalents).
    Reference: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/ProcessBuilder.html

    Other packages and classes can be more simply called, where the <core:arg> value can either be accepted directly from a GEL variable or maybe just coerced by setting the type="..." attribute.

    A lot (but not all) of the jelly commons core library tags work as documented, so for more variations such as needing to call static methods or similar, check it out.
    Reference: http://commons.apache.org/proper/commons-jelly/tags.html

    Make sure that all the files (scripts, jars, etc.) have the appropriate permissions set, so that the OS user running the bg services where the process engine is running will be able to find and execute what it needs.



  • 3.  RE: How to invoke custom Java class from GEL script

    Posted 08-17-2019 01:54 AM
    Edited by Nick Darlington 08-17-2019 01:55 AM
    Of course, you may be able to 'cut out the middle man' here if your scripts are just going to call your .jar file, as I said by using core:new / core:invoke / core:invokeStatic directly against the .jar.

    However depending on what it does, you may not want to, as it will be loaded within the same thread (by default) and process that the bg service is using to execute the gel script, and that might mean using memory or other resources in a way that aren't advisable within a GEL script.  In that case, having it carried out as an external call via the shell script (or even calling java -jar blahblah directly without a script) so that it is in its own process may be the better way to go.

    Just something to think about and consider.


  • 4.  RE: How to invoke custom Java class from GEL script

    Posted 08-17-2019 10:48 AM

    Take note what Nick is saying.  In my early experiences with pushing GEL to do more complex things, I ran smack into what he is pointing out.

    Since then, I have setup an external web helper application on an external server.  On the web helper, I deploy soap (some REST for use in html portlets) web services to perform any heavy lift process (creating complex spreadsheets/access databases, update internal systems (we are OnDemand), populating external data stores…).

    With the combination of using this web helper and being able to invoke process on CA PPM, I can kick off process from either an external process or via an event in CA PPM.  Using this approach, I can support request via an html portlet or a process.

    For example, A user request for some heavy lift processing say from a html portlet. The request is received via the web helper and the user in informed the process is queued and will be notified when complete. The web helper processes the request let's say for a finance spreadsheet (we call them checkbooks).  The service builds the checkbook and uploads it to our CA PPM (on demand) server.  The web helper request service then kicks off a CA PPM process to pull that spreadsheet from the FTP file location and attach it to the user's project.  The process closes by letting the user know that their request is available.

    On a side note, you will run into some issues, seems to be mostly around Factory style methods using the core tags.

    V/r,

    Gene




  • 5.  RE: How to invoke custom Java class from GEL script

    Posted 22 days ago
    Thanks @Eugene Greiff for the response. Yes, I don't want to handle complex things using GEL. So password encryption/decryption is being done in the shell script itself and using Java class to invoke the shell script​. However, as I said in my earlier response there is some issue with the BG which I could not figure out why the process was successfully executing the shell script after restarting the BG. Do you have any thoughts on this?

    ~PB




  • 6.  RE: How to invoke custom Java class from GEL script

    Posted 22 days ago
    So what is triggering the GEL script to run the three shell scripts?  What method are you using for the encryption/decryption?  What environment variables are you looking for to export to the bash shell?



    V/r,
    Gene


  • 7.  RE: How to invoke custom Java class from GEL script

    Posted 22 days ago
    Thanks for the response @Nick Darlington. Sorry for my late reply.
    Yes, you were right ​our application is on-prem and not SaaS. Instead of calling shell scripts in sequence - I have consolidated into one single shell script and a process has been created that contains Java Job as Action Item that accepts input string parameter (here shell script name). Everything was working fine if I run shell script manually from Linux box but if the shell script is executed by the process sometimes it couldn't get the environment variables and throw following errors on some nodes
    line 30: sqlplus: command not found 
    some other nodes throw as CLARITY_HOME path as NULL

    In the above cases, environment variables are properly set and exported in .bash_profile. These errors are caused by BG and when I restart the BG on that particular node - the process works fine. Couldn't identify the issue what actually BG does and why it was not able to pick up the environment variables sometimes. Is it because of cache or something? It would be great if this issue is resolved as I'm stuck here

    Thanks,
    PB





  • 8.  RE: How to invoke custom Java class from GEL script

    Posted 22 days ago
    I wouldn't expect intermittent behavior in the availability of an environment variable within the background service.  Are you sure the variable was set and exported in the profile (of the OS user that is specifically running the bg service) BEFORE the bg services were started?  If the service was running under a different user account and/or it was set after the service was started, that might explain it.

    However, you haven't shared how you are going about getting the environment variable to use it.  Was it exactly like this (aside from the variable name to hold the result), or something else?

     <core:invokeStatic className="java.lang.System" method="getenv" var="clarityHome">
      <core:arg value="CLARITY_HOME"/>
     </core:invokeStatic>




  • 9.  RE: How to invoke custom Java class from GEL script

    Posted 21 days ago
    Edited by Pavan Bugata 21 days ago
    Yes @Nick Darlington the variable was set and exported in the profile BEFORE the bg services started (profile last update date was 08/11 and bg.status file (apps/clarity/bin) was on 08/26) and service is running on the same user "clarity" where my shell scripts exist (apps/clarity/lib)

    I am not using GEL script to execute the shell scripts. Created a process that kickoffs the Job of Java Type that accepts parameters. Parameter holds the shell script name and passed to the java job.

    I'm pretty sure the issue is environment variables are not being picked up in one of the nodes in UAT. Below is the example where I could see the sqlplus path printed in the logs on node 1(success case)
    ENV variable values are retrieved here  and we can see the path printed


    The below is the Null Env variables logs from the other node
    Here env variables are NULL


    Here are the bash_profile env variables

    environment variables set
    Here is the shell script which is being used:
    Sqlplus lines are failing as it couldn't get the values from profile

    Is there any way I can set the env variables in the shell if the values are NULL?

    Thanks,
    PB


















  • 10.  RE: How to invoke custom Java class from GEL script

    Posted 21 days ago
    I get that GEL may not be directly invoking things, but GEL has - in my experience - not been inconsistent about the presence of those environment variable values; I can't say the same for the contexts under which other processes it launches may be invoked with.

    So if you are finding inconsistencies within the other processes and shell scripts, perhaps GET the values in the GEL script and pass them TO your other processes, if that would give you the consistency you seek.

    Otherwise for env variables set and exported in a user profile that is running the bg services, I can't see how that would ever have those kinds of inconsistencies.

    Unfortunately once you get outside of Clarity and into the host environment to run other things, I'm not able to offer any suggestion really.  As you're witnessing, that seems to be fairly environment and code specific.  But if you can rework it to read the env vars more reliably in the GEL and store/pass them over to the other external code to use when it executes, I hope that will get you to workaround those environmental issues.


  • 11.  RE: How to invoke custom Java class from GEL script

    Posted 21 days ago
    No sure this would help but since you are on premise, I have build a set of jobs options to support running cmd, gel, java type jobs from PPM.  You could extend it to support any language or modify the cmdprocess class to put the needed environment information and passing it out.  The other nice thing is it supports fire and forget in order to release the BG thread.  Let me know if you would like the code. You would need to recompile it as it was developed for V13.

    V/r,
    Gene