I have a requirement where I need to run multiple virtual services on single request and response queues. I have created some 4 services which are listening to same request queue and respond on the same response queue.
while creating I haven't kept any operation name which I believe will allow the request to be picked up by the relevant virtual service where the inbound message is matching with the VSI requests.But the issue I'm seeing is, I get the valid response after multiple hits. Looks like the request is being picked up in the order in which the virtual service are deployed and I have to hit the same request multiple times to get the response from a valid VSI.
I cannot merge the images into a single image as I'm using copybooks for the each individual virtual service and cannot have a single VSM with multiple copybooks.
For clarification to help us, I have a couple of questions:
1) What type of JMS transport are you using (JMS, IBM MQ, RabbitMQ, etc.)?
2) You commented that you "cannot merge the images into a single image as I'm using copybooks for the each individual virtual service and cannot have a single VSM with multiple copybooks". Is this because the headers or payloads do not contain any information on which to identify the copybook used in mapping? I have single services that applies different copybooks to request / response payloads.
3) What types of queues are you using and what technique are your VSMs using to ensure they only select messages to which they can respond?
Please find the clarifications for your questions below.
1.What type of JMS transport are you using (JMS, IBM MQ, RabbitMQ, etc.)?
A) The transport protocol is JMS only and the MQ server is IBM Websphere MQ
2.You commented that you "cannot merge the images into a single image as I'm using copybooks for the each individual virtual service and cannot have a single VSM with multiple copybooks". Is this because the headers or payloads do not contain any information on which to identify the copybook used in mapping? I have single services that applies different copybooks to request / response payloads.
A) Yes Joel. The only thing I fell can be helpful is ,when I checked the request in the inspection View, under the lisa.vse.request ,I see msg.props.transactionName as the only property which has information related to the transaction being call. Can we use this to redirect the request to the respective VSM?
Is it possible to share the model which you have that applies different copybooks to req/res payloads plz.
3. What types of queues are you using and what technique are your VSMs using to ensure they only select messages to which they can respond?
A) If I understood your question correctly, we use dedicated req and res queues for individual virtual services and while creating , we use the request data manager to identify certain unique field as operation in the VSI . Even though some of the other VSM's have same operation name, since we are using dedicated queues , request will always be picked up by the concerned model file and looked for the response in the mapped VSI.
Please let me know if you need any further information. Appreciate for you help.
Have you tried using some type of correlation mechanism? This could help determine which virtual service will service a given request, particularly in a situation where you have multiple virtual services listening on the same queue. Keep in mind that, when one of the virtual services picks up the request from the request queue, there's no way, as far as i know, for it to put the request back on the request queue in case it doesn't find a matching response and let the next virtual service service the request.... Instead, it will either put a meta response or an unknown response to the response queue.
I did enabled the message ID to correlation ID mapping scheme while creating the virtual services and I believe the correlation mechanism will come into play once the matched transaction is identified in the VSI. The msg ID from the request will up updated in response for the client application to pick the right response.
In my case, the issue is , the request is not being picked up by the right virtual services. What I observed is when I first hit the request and assuming 3 services are listening on the same request queue, the first deployed service is picking up and incase doesn't match will send the request doesn't match the response and stops. Second time when I hit, request is being picked up by the second deployed service and so on..My assumption is all 3 services should pick the request and send the response (2 invalid and 1 valid) and it is upto the client application to pick the right response using the correlation ID which is not happening.
Not sure whether DevTest supports running multiple services on same req and response queues.
I kind of created a work around to handle my requirement. I have created a proxy VS which will listen on single request and response queue but internally based on the transaction name, will route the request to the actual virtual service and get the response. From the application end, it will look like they are publishing on one req and response queue and internally we are managing to route to the individual virtual services running on different queues.
However I see one issue, Below is how my model is.
1. The JMS VSE Listen step will take the inbound request and based on the transaction name in the header, I have assigned the transaction specific queues to properties
2. In the JMS send receive, the publish and subscribe queue names are replaced with the properties in the previous step
This when executed, will send the request to the respective virtual service and gets the response. The response is stored in a property
3. In the jave step, I have added the script which will take the response from previous step and convert that into a transient response (lisa.vse.response)
4. The JSM VSE Response publishes the transient response to the listen step from where the application picks up.
This is working fine without a correlation mechanism in place but when I enable the correlation schema ,the response comes but because of incorrect correlation mapping I believe, the application is not able to pick the response.
What I even further tried is, in the listen step, I have captured the "lisa.jms.Recv.HeaderMap.JMSMessageID" and stored it in a property and in the java step, where I'm setting the transient response, I have added this message ID to "lisa.jms.Send.HeaderMap.JMSCorrelationID" in the meta data.
If I look at the ITR , the java step response, lisa.vse.response has the correlation ID which i have added from the listen step but when I look at the VSE Respond step, the correlation ID is shown as the Message ID from the JMS recieve of the send receive step.
Not sure where I'm doing wrong. Can anyone throw some light incase if you have worked on similar requirement earlier.
Thanks in advance!!
> My assumption is all 3 services should pick the request and send the response (2 invalid and 1 valid) and it is upto the client application to pick the right response using the correlation ID which is not happening.
> Not sure whether DevTest supports running multiple services on same req and response queues.
You can run multiple services listening on the same request queue. However, they do not share the same *listener*. Each one has its own connection to the queue. Unless you are using message selectors to control which service receives which request, then the particular listener that receives each request message is up to the JMS provider, and only that listener will receive it. The only way around this without using message selectors is to use a Topic instead of a Queue. Depending on the version of MQ you're using some level of support for Topics may be present, but I've never actually gotten it to work.
> based on the transaction name
Is this transaction name stored in the message body, or is it in a message header or custom property that you can access with a message selector? If it's in a property then we can support that now. You will have to build a JMS Consumer asset with an appropriate message selector for each of your services and make sure it's selected in the 'JMS Consumer' field in each of your VSM's Listen steps.
If your transaction name is in the body then we don't currently support doing filtering based on that from the VSE Listen step. We support that kind of thing in a client context as a correlation scheme; when a request is sent from the Send Receive step you can filter on some part of the response body when receiving the response. However, there's currently no way to take just the filtering part of that and use it from the VSE Listen step.
> What I even further tried is, in the listen step, I have captured the "lisa.jms.Recv.HeaderMap.JMSMessageID" and stored it in a property and in the java step, where I'm setting the transient response, I have added this message ID to "lisa.jms.Send.HeaderMap.JMSCorrelationID" in the meta data.
> Not sure where I'm doing wrong. Can anyone throw some light incase if you have worked on similar requirement earlier.
I think the meta-data property you have to set is called 'msg.correlationid', not 'lisa.jms.Send.HeaderMap.JMSCorrelationID'. However, there's a better way that doesn't bypass the correlation scheme on the Respond step.
It's tough to tell from just your screenshot, but I'm pretty sure the Send Receive step in between your Listen and Respond steps is overwriting a testExec property that the Respond step needs in order to successfully operate its correlation scheme. That property is 'LASTRESPONSEPAYLOAD'. Try this:
1. Add a filter to your 'Listen' step that copies 'LASTRESPONSEPAYLOAD' to some other property, like 'LASTRESPONSEPAYLOAD_COPY'.
2. Add a filter to your 'Send Receive' step that copies 'LASTRESPONSEPAYLOAD_COPY' back to 'LASTRESPONSEPAYLOAD'.
Could you please share this project. As i need to implement same thing in my project.
Could you please let me know what exactly you are looking for. I may not be able to share you the project but I can help you in setting up one for running multiple vs on same queue.
Thanks and Regards,
Thanks for the detailed insight Kevin, I will try the option you have suggested. Meanwhile I think I have solved the issue by changing the correlation scheme. Other than the listen step, for all the other steps I have changed the correlation scheme to JMSCorrealtionID instead of MessageID to CorrelationID . In the listen step , the Message ID which I have captured into a property , I'm writing that to the send receive step correlation ID scheme manually and the same is passed on to the subsequent steps.
This way , the messageID published in the listen step is coming as a correlation ID in the respond step and the application is able to consume the response.I have tired that option for one single service and hoping to seeing it working for multiple services as well.
Regarding the message selectors which you have suggested, I see the transaction Name coming in the request header which I believe DevTest supports. Can you throw some light on how to configure the JMS Consumer asset with an appropriate message selector or any documentation around that.
I tried the JMS Consumer asset with the message selector and looks like its working as expected. I tried locally by sending the transaction name in the custom properties in the JMS send step and have the message selector like below
TransactionName LIKE 'CUSTOMER_CREATE'
and the request is being picked up by the right virtual service even when multiple services are listening on the same queue. I will try out the same from the application end.
Thanks a ton Kevin for the insight!!