Service Virtualization

 View Only
Expand all | Collapse all

XML Payload To Fixed Width Flat File

  • 1.  XML Payload To Fixed Width Flat File

    Posted Jan 26, 2019 03:46 AM

    How to convert copybook payload xml to flat file after manipulation based on the copybook definition file ?

     

    Regards,

    Manik



  • 2.  Re: XML Payload To Fixed Width Flat File

    Broadcom Employee
    Posted Jan 28, 2019 04:37 AM

    Hi Manik,

     

    We would need some more context here of what you are trying to achieve?

     

    Are you working in a virtual service model? In a virtual service model the "copybook payload xml" is converted back to a flat cobol structure by adding the Copybook DPH as a response DPH in the Respond step.

     

    If you want to reuse this mechanism in a testcase in order to really write the flat cobol structure to a file, then you must first understand that the response side DPH works on a pre-existing object, ie. the lisa.vse.response object. That object exists in a .vsm because it is created by the "VS Service Image Selection" step. In a testcase you will have to use some scripting to create and populate that object before you add the Copybook DPH to some random step.

     

    On a high-level the procedure would be to:

    • add an "Execute Script" step
    • inside the script, create an ArrayList object eg. "responseArray"
    • Create a transient response object, ie "response"
    • Set the body text of the response object to your copybook payload xml, eg. response.setBodyText("<yourCopybook PayloadXml");
    • Add the response object to the responseArray, eg. responseArray.add(response);
    • Store the arraylist as the lisa.vse.response property, eg. testExec.setStateObject("lisa.vse.response",responseArray);
    • Now you can add a VirtualServiceEnvironment filter to this script step and configure it for responses and for your copybook definition
    • After the above step has executed, in a next step, you can retrieve your copybook string in a script using eg. lisa_vse_response.get(0).getBodyBytes()

     

    Cheers,

    Danny



  • 3.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 28, 2019 09:40 AM

    Thanks Danny. I will give it a try. Few queries .

    1) I hope you are telling about Data Protocol filter-->Copybook Data Protocol in VirtualServiceEnvironment filter.

    For the statement "

    • After the above step has executed, in a next step, you can retrieve your copybook string in a script using eg. lisa_vse_response.get(0).getBodyBytes()" 

    Just confirm are you referring to Responder step.

     

    Regards,

    Manik



  • 4.  Re: XML Payload To Fixed Width Flat File

    Broadcom Employee
    Posted Jan 29, 2019 03:17 AM

    Hi Manik,

     

    We are talking about a testcase, right? A .tst file? In that case no, the step I am referring to does NOT have to be a Responder step (a Respond step would throw an error outside a .vsm if there is no Listen step)

     

    Let us know if not clear.

     

    Cheers,

    Danny



  • 5.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 29, 2019 07:38 AM

    Hi Danny,

    Let me help you to understand the stuff in a more detailed way.

    I have created a VSM without VSI wherein I am converting the String request (stream of characters just like flat file data)  into xml (copybook xml payload with data in tags), then manipulate the xml by replacing one of the tag values and then convert the same into flat file i.e String and send it back as response.

     

    The part of converting the string to xml is happening perfectly but I am unable to convert the xml payload back to String.

    Please note its the VS generated copybook payload xml with values of tag inside it.

     

    Hope this helps to make my requirement clear to you , else happy to have webex.

     

    Thanks,

    Manik



  • 6.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 29, 2019 12:19 PM

    The problem is still not totally clear. 

    Are you using both request and response side Copybook DPHs? 

    Have you configured both request and response side copybook mappings in DevTests copybook mappings XML structure?  They are different.  It could be that your request side mapping is working, but your response side mapping is not working.

     

    I believe DannySaro approach may still apply. 

     

    A normal VSM flow looks like this:

    - Listen Step (with Copybook DPH filter)takes stream and converts to XML based on Copybook mappings defined to DevTest.

    - VSI Selection creates a response list containing Copybook XML response having {{magic strings}} noted in the VSI response.

    - Responder step (with Copybook DPH) converts formatted XML back to stream and applies magic string substitution to {{ }} tagged data.  Response is sent on the wire as a stream of data.

     

    In a normal VSM, the VSI selection step above creates the lisa.vse.response property which holds a List of TransientResponse objects. This List of TransientResponses which contain Copybook XML is used internally by DevTest when the Responder step executes. Then the Response Side Copybook DPH is used to convert the response XML back to a stream. <-- This requires a copybook mapping xml definition in DevTest.

     

    But, it seems the VSM is performing its own VSI Selection logic against a Copybook XML request payload. In other words,

    - Listen Step (with Copybook DPH filter)takes stream and converts to XML based on Copybook mappings defined to DevTest.  Or, you have added custom script logic which takes the lisa_vse_request body String and converts it to Copybook XML using some custom script.

    - Additional script logic manipulates the Copybook XML and applies some replacements. This logic DOES NOT appear to be creating a lisa.vse.response containing a List of TransientResponse objects.

    - Responder step executes and fails

     

    If the description of the custom VSM above is true, try this. 

    - Add a VSI selection step to your model after the Listen step.

    - In the VSI, leave the operations and signatures blank. In other words create an empty VSI that has no matches. 

    - In the VSI's Stateless image response, change the MetaData's HTTP-Response-Code to 200, Text to OK, and content-type to an appropriate value.

    - By default, when the VSI Selection step executes, DevTest will send back a lisa.vse.response object containing a TransientResponse having the Stateless Image Response Not Found as its body.

     

    After the VSI selection step, execute your custom logic to create a response payload.  Then, add some script logic to replace the VSI response body created in the VSI Selection Step with your converted Cobol Copybook content. 

    [e.g., 

    // set your manually manipulated payload into the response body

    lisa_vse_response.get(0).setBody( testExec.getStateValue(<yourCopybook>) ); 

    ]

     

    This line of script will replace the Stateless Image Response Not Found with the content you are manually creating and manipulating. And, it will do so in a proper lisa.vse.response list with a proper TransientResponse object.

     

    In the Responder Step,

    - configure the responder to use lisa.vse.response

    - If your custom response contains copybook XML

       - Add the proper mappings in the copybook mapping definition file

       - Add the Copybook DPH and configuration as a response side filter to allow DevTest to convert the Copybook back to a string response

     

    Else

     

      - if your custom logic has converted the Copybook back to a String already, remove the Copybook DPH from the Response Filters -- your logic converted it from XML back to string so you do not want DevTest to do this.

     

    NOTE: If you are passing Copybook XML to the Response Side DPH, the DPH requires that you have configured the copybook mapping so that DevTest knows which mapping to apply when it converts XML back to string. 



  • 7.  Re: XML Payload To Fixed Width Flat File

    Broadcom Employee
    Posted Jan 30, 2019 02:07 AM

    Hi Manik,

     

    As you know there are always multiple solutions to implement the same set of requirements. Joel offers various avenues of thoughts which you might consider. I will just comment on the specific solution path that I already outlined.

     

    If this is a .vsm then indeed your earlier remark was correct: last step in the workflow loop is the respond step, and for this step you add the Data Protocol filter-->Copybook Data Protocol, and you configure it to Process Responses.

     

    So, the scripting is now reduced to below, but some additions in bold:

    • add an "Execute Script" step
    • inside the script, create an ArrayList object eg. "responseArray"
    • Create a transient response object, ie "response"
    • Set the body text of the response object to your copybook payload xml, eg. response.setBodyText("<yourCopybook PayloadXml");
    • Set the necessary metadata in the response to satisfy the transport protocol requirements, eg. for HTTP/S the minimum is Http-Response-Code and Http-Response-Code-Text
    • Add the response object to the responseArray, eg. responseArray.add(response);
    • Store the arraylist as the lisa.vse.response property, eg. testExec.setStateObject("lisa.vse.response",responseArray);

     

    With the above steps the lisa.vse.response object is available in the environment to be processed by the CopyBook DPH in the respond step.

     

    Cheers,

    Danny

     

     

     



  • 8.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 30, 2019 04:29 AM
      |   view attached

    Hi Danny/Joe,

    Thanks for your valuable inputs. I have implemented the steps as mentioned by you.

    At present I am getting an error as attached after applying the below code which sends the output to Responder step for forming the copybook stream using the copybook and the payload mapping file. Please let me know if you want to view them also.

    Here the output xml is containing the copybook payload data in xml format.

     

    Response response = new Response();
    response.setBody(testExec.getStateObject("outputxml").toString());
    ParameterList metadata = new ParameterList();
    metadata.addParameters("HTTP-Response-Code=200&HTTP-Response-Code-Text=OK&Content-Type=application/xml");
    response.setMetaData(metadata);
    TransientResponse transRsp = response.createTransientCopy();
    testExec.setStateObject("lisa.vse.response", transRsp);
    return;

     

    Kindly advise me here.

     

    Regards,

    Manik

    Attachment(s)

    zip
    ErrorLog.txt.zip   1 KB 1 version


  • 9.  Re: XML Payload To Fixed Width Flat File

    Broadcom Employee
    Posted Jan 30, 2019 07:20 AM

    It looks like the lisa.vse.response object which was created by script isn't complete. When I wrote out the high-level steps I wasn't thinking through how the response object looks for a cobol payload. I now recall that the response contains metadata that is used by the DPH to convert the cobol XML payload back into a flat cobol data string.

     

    My suggestion to approach this. Create a request response pair from request and response flat cobol data strings.

    Use Create Image from Request Response pairs to create an out of the box .vsm and .vsi from this. It also allows you to validate that your Cobol DPH configuration is working correctly.

     

    Now inside the generated .vsi you can have a look what metadata properties are needed, and you can choose to create them as such.

     

    However, another approach can be to use this generated .vsi to let the vsm create you a template lisa.vse.response object.

    Here's what you would do in your vsi:

    • Remove the specific transaction, now the vsi only contains one meta transaction
    • Set the operation name to "template"
    • Set the matching to "Operation"
    • Replace the response with a property eg. {{cobolXmlPayload}}
    • Look in the response metadata properties, any properties that are not generic but specific to your req-rsp payload and which would change for other req-rsp pairs you would have to replace with properties

    And in your vsm you would:

    • Add in a "VS Image Response Selection" step, configured for above vsi
    • You already have scripting to retrieve data from lisa.vse.request and create your cobol xml payload, do this scripting before the "VS Image Response Selection"
      • Store your response cobol xml payload in property "cobolXmlPayload"
      • Make sure you store values for all your response meta data properties
    • At the end, before the "VS Image Response Selection" step, set the operation name of lisa.vse.request to "template"
    • Now, you do not have to do anymore scripting AFTER the "VS Image Response Selection" step, just let the workflow go to the Respond which has the with the response-side Copybook DPH configured

     

    Cheers,

    Danny



  • 10.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 30, 2019 08:32 AM

    I believe creating a sample VSI is the best approach. I would argue that using the VSI Selection to create the response object is also the best approach to reduce your need to set up the framework objects correctly. Your code just overrides the response body.

     

    Question: What does your mapping.xml document look like? Specifically, what did you configure there so the response side processing knows which copybook to use for the response mapping?

    <payload type="response" name="***" key="yyy"> <section name="body"> <copybook>abcde.***</copybook>

     

    Question: This virtual service is not a mainframe virtualization using the mainframe LPAR and CICS Agents, correct? 



  • 11.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 30, 2019 08:39 AM

    Hi Joe,

    No we are not using LPAR and CICS Agents to virtualise the mainframe. The basic intention of doing the POC is to prove the response coming from MQ to be made dynamic depending on the request data.

    P.S- We are using proxy queues here for virtualising the mainframe calls. Any inputs to achieve this would be helpful .

     

    Regards,

    Manik



  • 12.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 30, 2019 08:55 AM

    OK, good. No LPAR / CICS properties to worry about. That would open a whole new can of worms.

     

    I was unable to locate where the exception is thrown in the actual code. And, The attached file contains syntactically correct XML. So, it makes me wonder if the exception is related to the response-side mapping.xml document not having a response definition that identifies which copybook to use for converting the XML back into a stream.

     

    The response side copybook needs to locate a mapping in the mapping.xml just like the request side copybook does. So, I was curious how the mapping.xml is annotated for locating the response copybook member to use during mapping. 

    I would have expected the mapping.xml to contain a <payload type="request" .... and a <payload type="response"... mapping with instructions on which <copybook> to use for the mapping. I do not know the specific content for your scenario.



  • 13.  Re: XML Payload To Fixed Width Flat File

    Posted Jan 30, 2019 09:05 AM

    Hi Joe,

    Providing you the mapping.xml contents FYR.

     

    <?xml version="1.0" encoding="UTF-8"?>
    <payloads>
    <payload type="response" name="TEST" matchType="payload" key="TEST" value="TEST" keyStart="1" keyEnd="8">
    <section name="Section">
    <copybook>TEST_RequestCopyBook.txt</copybook>
    </section>
    </payload>
    </payloads>

     

    Regards,

    Manik