Idea Details

Jenkins-Plugin that uses Lisa Invoke API

Last activity 06-13-2019 09:24 AM
stephan.burkard's profile image
04-11-2016 05:33 AM

Hi

 

Update 2016-04-15: I updated the description below with first findings on Invoke-API.

 

The approach of integrating Lisa tests into CI through the Lisa Ant task has the problem of remoting. The Ant task must run on a Lisa server while the trigger to run the tests and the report files must come and go to the CI server. I guess nobody wants to install Lisa on the CI server(s).

 

There are a lot of questions around how to integrate Lisa into CI. I think it would be great to have a Jenkins plugin for Lisa that uses the Lisa Invoke API and can do the following:

  1. Execute tests or testsuites on a remote machine via REST call
    1. This is possible with Invoke API (for example "runSuite" command)
  2. Executions can be made synchronous
    1. Test results are immediately available in the response
    2. BUT I think the result is unfortunately NOT JUnit compatible
    3. E.g. if a testsuite is executed and 1 test succeeds and 1 fails, I don't see which one failed
  3. Executions can also made asynchronous
    1. Tests or even suites can take a long time to run, so the REST call cannot wait for every result
    2. There is a parameter to run an execution async
    3. The response has no results, but a "callbackKey" that can be used to poll the results
  4. Then the plugin should poll the test results with the given callbackKey in a configurable interval and number of times
    1. Tests or suites can massively differ how long they run, so it would be nice to configure the interval and the number of times to poll per test/suite execution
    2. The number of times to poll is important to avoid infinite polling
  5. The test results could easily be transformed to a JUnit compatible format, so Jenkins can generate graphics etc as for other tests
    1. see http://stackoverflow.com/questions/4922867/junit-xml-format-specification-that-hudson-supports for the JUnit XML report format that is needed for Jenkins

 

Stephan


Comments

10-18-2018 06:05 PM

The official Jenkins Plugin from SV was released earlier in the year and is available on Jenkins.io marketplace.Changing the status of this idea to Delivered.

02-14-2017 11:39 PM

02-14-2017 03:03 PM

CA competitor SOASTA has a lovely Jenkins plugin with an AJAX window where you can view test results.

CA should be willing to invest the resources to create/support something similar!!

 

Jenkins is the most popular CI/CD tool in the world.

We need an official plugin!!

02-06-2017 10:38 AM

Is there any more conversations about coming up with an official released version from CA?

05-26-2016 04:39 PM

Check out this plugin - Unofficial Prototype of Jenkins Plugin for CA Application Test Integration 

 

Internally, we use TeamCity and run via lisajunit stuff with our build.

05-03-2016 01:32 PM

We do something very similar to this where I work. we have a Jenkins Build server set up on a machine where DevTest resides and run the commandline API in the background to create then deploy the virtual services.

 

I would love to see something either designed for Jenkins make ease of use higher (reduce explanation to users on what the commands are doing, and hide our admin account passwords) or some functionality added to the API that would lend itself better to work with Jenkins from DevTest's side of the fence.

05-03-2016 01:23 PM

Good question - let me work on a blog on the subject. I have to pull some data from the dev team on the "how". would be a good case study.

04-14-2016 11:56 AM

Stefana_Muller: If I remember correctly, you told us at the User Group Meeting in London this year, that you are using the DevTest API in connection with Jenkins to drink your own champagne. Is that correct?

 

I think it could be very interesting if you could give us more insight into how you are doing this!