Service Virtualization

Expand all | Collapse all

Generic Tests to Prove VS is Up to Date

  • 1.  Generic Tests to Prove VS is Up to Date

    Posted 01-10-2019 06:37 AM

    Hi Team,

    Is there a way to achieve the below using CA Application Test or any other Devtest solution. 

     

    • Test To prove virtual service is still up to date against back-end
    • Will alert us when a VS is out of date, meaning either it needs rerecording or a test needs changing.

     

    Thanks,

    Manik



  • 2.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-10-2019 08:27 AM

    Hi Manik, 

     

    Moving this thread into the CA Devtest community for visibility with our Devtest experts.  

     

    Regards, 

    Heena



  • 3.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-10-2019 08:42 AM

    Thanks Heena.



  • 4.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-15-2019 10:40 AM

    What are your requirements for determining if a VS is up to date to the back-end application?



  • 5.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-15-2019 01:22 PM

    Hi,

     

    We recently implemented a solution for a customer which deals with a similar problem as your question. On a high-level there is a similarity. Not sure if what we did will work for your use case. Marcy is right to ask for more details about your requirements.

     

    It is a recent implementation, I do not yet have good visual material to present the solution, so I'll try to describe it as simple as possible.

     

    This particular "Virtual Service Validation" solution is a specific implementation of a Low-code API Test Generation framework that we have built.

     

    The objective of the framework is to easily build a whole suite of test cases (.tst files) without having to manually adapt test steps, and filters, and assertions. The functionality of a test case (i.e. the steps, filters, assertions ...) is created based on a template test case. So, this means that the generated test case can be quite complex - meaning rich in functionality -  if needed, but every test case has a similar structure, i.e. functionality, as it created from the same template. So, this is ok for API Tests where every testcase is similar, i.e. sending a request payload to the API, and then running some assertions on the response to validate it is as expected. So, this type of test we implemented via the "Template_APITest" template.

     

    Something similar to your requirement was implemented via our "Template_VSValidation" template. The functionality of a generated VS Validation testcase is as follows:

    1. Pick up a request payload
    2. Send the request to the Live API
    3. Send the request to the Virtual Service API
    4. Filter sets of relevant information out of the Live Response
    5. Filter the same sets of relevant information out of the Virtual Response
    6. Assert that the filtered Live Response information is equal to the filtered Virtual Response information, if so test ends, otherwise test fails

     

    The definition of such a test suite would be done completely by editing some files in a directory, it would look something like this:

     

     

    The -req.xml files would contain a request to be sent to the API, it would look something like this:

     

     

    The -val.xml files would contain the definition of the filters and assertions as they would be defined/required by the "Template_APITest" template, it would look something like this:

     

     

    After creating a test definition like above for every test you want to generate, we would run a batch file which eventually would use testrunner to execute another test - i.e. CreateTestV3 - which would write out a .tst file for every pair of -req.xml + -val.xml file.

     

    The generated testcase would look like:

     

     

    The actual API calls are done inside a subprocess (doesn't have to be but here that is how the template is built). I selected the "Filter Live Response Payload" step, you can see it has 2 filters created (those are the ones defined in the "LiveCallFilters" Filters element of the -val.xml file. You can see the 2 assertions created in the final "Validate Filtered Data Equal".

     

    So, the above is the long answer, the short answer to your question is 'yes, this is achievable'.

     

    Cheers,

    Danny

     



  • 6.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-16-2019 12:03 AM

    Hi Danny,

    Thanks for providing such an in depth explanation. It would be really helpful if you can share me the project in zip so that I can play it around. I would also require the Template_VSValidation to look into it.

     

    I am looking for something similar bit in a more simpler and quickest way. Something like the steps mentioned below :-

     

    1) Create a virtual service from Live.

    2) Based on the transactions recorded during VS recording, on click of a button it will generate tests with Request / Response artifacts at one shot for all transactions with assertion on Response. The same tests were collected and a Test Suite will be created for Test Run on the VS as part of Regression testing the same against the New Live Environment.

     

    Thanks,

    Manik



  • 7.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-16-2019 03:39 AM

    Hi Manik,

     

    Some remarks:

     

    • What you describe in 2) as requirements is very similar to another framework that we have built once upon a time. You find some info about it here Executable and Testable Contract-Based Design 

      More particularly this part is an answer to your requirement "... It is basically a testcase that opens a VSI file - which in itself is XML - and processed all the transactions defined in that VSI file. Each transaction becomes a testcase for the consumer of the contract fired against the System Under Test ..."

     

    • Tricky part for me is the requirement "... with assertion on Response ...". The response of the Live system is going to have dynamic data e.g. timestamps, dates, newly created business object identifiers like ordernumbers, ticketnumbers, ... which are different from the data in the responses at the time of recording. The assertion will have to be more intelligent than a simple text compare.

     

    I will make both frameworks available (for the whole community), I will put them on github where they can not only be downloaded/consumed but I would hope that any corrections and/or additions made would be uploaded again such that the frameworks grow in features and functionalities. I will let you know the link asap.

     

    Cheers,

    Danny



  • 8.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-16-2019 05:48 AM

    Thanks Danny.



  • 9.  Re: Generic Tests to Prove VS is Up to Date

    Posted 01-17-2019 04:43 AM

    Hi Danny,

    Could you please share the frameworks so that I can have a look into it?

     

    Regards,

    Manik