Service Virtualization

 View Only
  • 1.  Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted May 20, 2016 04:50 PM
      |   view attached

    All,

     

    While looking at the current ways we can integrate Jenkins with CA Application Test, I came up with the following methods:

    1. If DevTest is installed on the same server as Jenkins, we can use the Ant/junitlisa integration to locally run our tests, generate a JUnit report, and integrate it into Jenkins reporting.

    2. If DevTest is not installed on the same server as Jenkins, we can use a plugin like HTTP Request Plugin to leverage the DevTest Invoke API to start tests remotely and then follow up with the test results out in the DevTest Portal to find out which ones passed/failed.

     

    What I was looking for was a third option where I could leverage the DevTest Invoke API to kick off a set of tests, but then also receive some feedback within Jenkins about how many of my tests passed/failed. This sent me on a quick path of Googling how to create Jenkins plugins. What I came up with was a relatively small Java project which builds a Jenkins plugin that does the following:

    Allows you to choose your remote DevTest host, port, test suite path, and config path, and then will kick off your test suite remotely using the Invoke API. The plugin will then scrape the output of the Invoke call and provide a JUnit style xml output file that can be integrated into Jenkins reporting. I'm publishing the plugin here to solicit feedback and help others who may have a similar need, but it must be understood that:

     

    THIS PLUGIN IS NOT PART OF THE OFFICIAL DEVTEST PRODUCT

    THIS PLUGIN IS NOT SUPPORTED BY CA

    IF THE OFFICIAL DEVTEST INVOKE API CHANGES, IT MAY CAUSE THIS PLUGIN TO STOP WORKING

    USE AT YOUR OWN RISK

     

    If you'd like to see the source code, or you'd like to extend this plugin to do something special, please reach out to me and I'd be happy to work with you.

    I've tested this plugin against DevTest version 9.1, but I suspect it will work fine with older/newer versions that also contain the Invoke API.

     

    If you would like to try out this prototype plugin, here are the instructions:

    1. Download the attached hpi plugin file and extract it from the zip file

    2. In your Jenkins instance, go to Manage Jenkins -> Manage Plugins, and on the Advanced tab, scroll down to Upload Plugin, browse to the downloaded plugin and click Upload

    3. In your Jenkins projects, you now have a new option under "Build Step" which is CA DevTest Invoke Test Suite.

    4. Add the requested information. For the server, be sure to include http:// or https:// in front of your hostname. For Test Suite Path and Config File Path, be sure to check the hint for those fields which says to be sure you update property lisa.portal.invoke.test.root found in lisa.properties on your registry server.

    5. In order to see your test results, under Post Build Actions, add a step for "Publish Junit test results report" and include the path devtest_results/**xml

    6. Now kick off a build. You'll need to actually run your build twice the first time in order to see the test report show up (this is a Jenkins thing.)

     

    When you see your test results report, you can drill down into it. There are a couple of things to note about this prototype:

    The plugin only has access to the information that comes back from the invoke API. That means it only has the overall duration of the entire suite, not individual tests. Also, it only tells us how many tests passed/failed, not which individual tests passed/failed. What this means is that in the test results report, the overall time is accurate, however the individual test times are just an average time of all the tests. Also, the drilldown which shows which tests passed/failed will be accurate as far as the numbers of each, but not which ones passed or failed. To see this information, continue drilling down into any of the tests and you will see the REST Invoke output which includes a link you can follow that will take you directly to the DevTest report for your test suite which contains all the accurate info.

     

    If you download and use this, please let me know, and let me know what you think. I hope this helps someone since it was a fun side project for me to work on.

     

    -Chris

    Attachment(s)

    zip
    devtest_invoke.hpi.zip   1.14 MB 1 version


  • 2.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jun 06, 2016 03:28 PM

    This is cool Chris !



  • 3.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jun 07, 2016 10:20 AM

    This is great! Melanie_Giuliani - can we convert this into a blog or doc by Chris so it won't get lost in the discussion feed?



  • 4.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jun 07, 2016 12:25 PM

    Hi Stefana,

     

    If I convert it, it will show me as the author.

     

    reich05 - if you'd like to convert this to a doc, where it may get more attention, click on "Actions" and choose "convert to document" - either way i think this content is great and i will be sure to promote it for more attention!

     

    Thanks,

    Melanie



  • 5.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jul 15, 2016 11:55 AM

    Hi Chris,

     

    can we get the source code posted to github or any other place. This is pretty cool .

     

    Thanks

    Jagadish



  • 6.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jul 15, 2016 12:07 PM

    Hi Jagadish,

     

    I haven't yet posted the project to any repositories, but I'd be happy to walk you through the source via WebEx and/or share the project with you directly (It's a Maven build, FYI.) You can email me directly at Christopher.Reites@ca.com and we can set something up.


    Thanks,

    Chris



  • 7.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jul 15, 2016 01:23 PM

    Thanks Chris,

     

    I have sent you a mail for webex.



  • 8.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Oct 24, 2016 03:10 AM

    Thanks Chris. We  are looking the same solution. 

     

    How do you transfer LISA testcase/suite related files from Jenkins Slaves to Remote LISA Servers? Is there any LISA Invoke AI to transfer files to LISA Servers?

     

    Thanks,

    Chocks



  • 9.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Oct 26, 2016 12:20 PM

    Hi Chocks, the LISA test cases and suite will already live on the LISA server, and you are just remotely kicking them off via the API from Jenkins. If you're looking for a way to get your test cases and suites onto the LISA server, I would recommend having all of these checked into source control and pulling the LISA project down to the LISA server from source control. If needed you could use some of the built in Jenkins capabilities to automate some of that (apart from the DevTest plugin.) Hope this Helps,

    Chris



  • 10.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Jan 31, 2017 03:06 PM

    Hi Chris,

     

    Thanks for the solution its works perfect for us. One thing I could not get is devtest_results/**xml in step5.

    where is the devtest_results path. Do I need to give any devtest server path here? Little confused here.

     

    Thanks,

    Vibhash



  • 11.  Re: Unofficial Prototype of Jenkins Plugin for CA Application Test Integration

    Posted Feb 15, 2018 05:21 AM

    HI Chris,

     

    Could you please let me know if this plugin has the capability to take env variables for the parameters that needs to be given in the plugin details section.

     

    Ex: If i have to change the IP address of the devtest registry url, i have to go manually and change it as it does not take the env variable that i would like to pass.

     

    Regards,

    Amrutha