Live Invocation Mode not Working with Copybook Data Protocol in Place

[Saman Aghamir-Baha]

Hi,

 

We recently bumped into an interesting observation while working with DevTest 9.1.

 

The transport and data protocols that we are using are IBM MQ Native Transport protocol and copybook data protocol respectively.

 

In order to create virtual services, we used the recorder and were able to connect to Queue Manager successfully and record traffic. However once we deployed the service, the Live Invocation mode didn't work. The Live Invocation worked once we removed the copybook data protocol. It appears the data protocol handler is modifying the original request property, preventing the original request from being passed in the passthrough mode. Instead the XML message created by the data protocol handler is being passed to the live system, which the live system can't handle. We noticed none of the other data protocol handlers do this, so it's likely an issue with the data protocol handler. We have come up with a couple work arounds but wanted to check with the community to see if anyone else has hit this issue.

 

One thought is to have a duplicate service without the DPH and use that for live invocation mode. However a more elegant solution to accommodate for all modes in one service is more desirable.

 

Another thought is we can put a patch in the model to save the original request into a temporary parameter prior to the data protocol handler modifying it. Then in the live system step, we can pass the original request instead of the modified request. We're hoping not to write a custom filter to pull the body out of the request. Does anyone know if the body is parsed into a property prior to pushing it through the data protocol handler?

 

Any other solutions welcome!

 

Regards,
Saman

[Kevin.Bowman]

Unfortunately, this is a more general problem with any transport protocol and a data protocol handler that overwrites the request body.  The original request body is lost almost immediately due to the data protocol filter on the Listen step itself.  What gets saved to the 'lisa.vse.request' and 'LASTRESPONSE' properties is the VSE request *after* data protocol handlers have been run.

 

The best option is if the original request object was preserved intact, along with all its meta-data, attributes, etc.  You can do this in 9.1, but it requires some surgery on your VSM.  You have to separate your 'Listen' step from its DPH filters, moving them to a new step where you can create a copy of the request object before the DPHs are run.  Then you need to insert a step before your 'Live Invocation' step to "revert" the 'lisa.vse.request' property to the copy of the original you made. 

 

I tried this out and it works, but you have to be careful with step connections.  Here's what my VSM looks like post-surgery:

 

I added a script step after the 'Listen' step, and *moved* all of the 'Listen' step's filters to the new step.  You can click and drag filters to a new step to copy them, and then you have to delete the originals from their original step.  I also had to make sure the 'If being efficient' assertion on the 'Listen' step went to this new step, and I copied that assertion to the new step as well. 

 

Here are the contents of this first script step:

// get the request before any DPH is run
request1 = testExec.getStateObject("lisa.vse.request");

// create a new request and copy the original data over
request2 = new com.itko.lisa.vse.stateful.model.Request();

// there's no convenient copy constructor, so this has to be
// done piece by piece
request2.setOperation(request1.getOperation());
request2.setMetaData(request1.getMetaData());
request2.setArguments(request1.getArguments());
request2.setAttributes(request1.getAttributes());
// how to copy the body depends on whether it's binary or not
if (request1.isBinary()) {
    request2.setBody(request1.getBodyAsByteArray());
    request2.setBinary(true);
} else {
    request2.setBody(request1.getBodyAsString());
    request2.setBinary(false);
}

// save the request copy to a new testExec property
testExec.setStateObject("lisa.vse.request.backup", request2);

 

Before the 'Live Invocation' step I inserted another script step.  I didn't have to mess with connections here; the way it arranged itself by default is good enough.  Here are the contents of this second script step:

// get the original request
request2 = testExec.getStateObject("lisa.vse.request.backup");

// revert the testExec's request object to the old one.
testExec.setStateObject("lisa.vse.request", request2);

 

I wish I had a better answer but I this is the best I can come up with for now.  We definitely need to take a look at this scenario for the next release and see if we can make it "just work".  I'm seeing this issue pop up a lot now that we have an IBM MQ protocol that supports live invocation and lots of IBM MQ shops also dealing with Copybook.

[Anon Anon]

Hi Kevin,

 

Thanks! Can you post the sample model here? I believe it will be very helpful for those trying to get live invocation working with MQ.

 

You may also want to take a look at how the other transport protocol handlers are doing this. With a quick test for example, I can create a web service based virtual service using multiple data protocol handlers, including custom built data protocol handlers. In live invocation mode, the original request is still sent to the live system as expected. This suggests the MQ transport protocol handler may need to be updated to store the original request for use in the live invocation step prior to allowing any data protocol handlers from modifying the request. It could be as simple as modifying the listen step to capture and store the request as soon as it's received. Then use that property in the live invocation step. Basically you could do exactly the same thing as above, but it would be done by the transport protocol handler instead of requiring manual tweaking of the model. Thoughts?

 

Sincerely,
Casey Best

[Kevin.Bowman]

Yeah, it looks like the HTTP and TCP protocols do their own special things to work around this problem.  They don't copy the VSE request object, but they do save the original request data in various formats to their own properties. Let me see what I can do with the messaging protocols...

 

Thanks for the heads up.

[Anon Anon]

Hi Kevin,

 

Thanks for looking into this. I did a quick test to narrow down the problem. I created a custom data protocol handler which converts an ISO based message request to XML, which is used by the service image. I tried it with the MQ transport protocol handler - it worked as expected in most efficient mode and live passthrough mode. This suggests the MQ transport protocol handler is designed correctly, but the copybook protocol handler is doing something to permanently change the original request, instead of just changing the request body to XML so it can be used by the service image. Hope this helps you find the underlying issue.

 

Sincerely,

Casey Best

[Kevin.Bowman]

> the copybook protocol handler is doing something to permanently change the original request, instead of just changing the request body to XML

 

That's the same thing.  There is only one request object and one request body.  You must be doing something else weird, because I can reproduce this problem in 9.1/9.5 with just the Request Data Manager DPH, or just a script step that changes the request body before it gets to the Live Invocation step.

[Saman Aghamir-Baha]

Hi Kevin,

 

I followed your instructions and could get live invocation and most efficient mode working in one service.

 

Thanks a lot for your help.

 

Regards,

Saman

[Legacy User]

Follow up question RE transports, please...  Related to DevTest 9.x versions... 

 

Can anyone confirm whether or not the CICS and IMS Connect transports, when the Copybook DPH is used and the service needs Live Invocation, affected by this issue?