Service Virtualization

 View Only
  • 1.  java.net.SocketException: Connection reset || Error in VSE log

    Posted Jul 11, 2019 12:21 PM
    Hi Team,

    In Portal, our Virtual Services are displaying as up and running but when we triggered any requests, getting the read time out errors.
    So we checked in the VSE logs for errors and found the the below errors are for few of the services.

    Please explain me on the below points.

    • When will this error occur ?
    • What's the resolution to this error ?

    2019-07-10 16:10:44,281Z (18:10) [Lotte_CB_1 [VS_ZSMART_Volume_Lotte_CB_1_Run]/27] ERROR com.itko.lisa.ws.nx.NxWSStep   - Unable to invoke request: ; nested exception is: 2019-07-10 16:10:44,281Z (18:10) [Lotte_CB_1 [VS_ZSMART_Volume_Lotte_CB_1_Run]/27] ERROR com.itko.lisa.ws.nx.NxWSStep   - Unable to invoke request: ; nested exception is:  java.net.SocketException: Connection resetAxisFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException faultSubcode:  faultString: java.net.SocketException: Connection reset faultActor:  faultNode:  faultDetail:  {http://xml.apache.org/axis/}stackTrace:java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:209) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160) at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84) at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) at com.itko.lisa.ws.axis.LisaHTTPSender.doInvoke(LisaHTTPSender.java:492) at com.itko.lisa.ws.axis.LisaHTTPSender.invoke(LisaHTTPSender.java:267) at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32) at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118) at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83) at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165) at org.apache.axis.client.Call.invokeEngine(Call.java:2837) at org.apache.axis.client.Call.invoke(Call.java:2820) at com.itko.lisa.ws.nx.NxWSStep.executeCall(NxWSStep.java:384) at com.itko.lisa.ws.nx.NxWSStep.execute(NxWSStep.java:289) at com.itko.lisa.test.TestNode.executeNode(TestNode.java:984) at com.itko.lisa.test.TestCase.execute(TestCase.java:1297) at com.itko.lisa.test.TestCase.execute(TestCase.java:1198) at com.itko.lisa.test.TestCase.executeNextNode(TestCase.java:1183) at com.itko.lisa.test.TestCase.executeTest(TestCase.java:1124) at com.itko.lisa.coordinator.Instance.run(Instance.java:208)
    {http://xml.apache.org/axis/}hostname:SWNC7R1074
    java.net.SocketException: Connection reset at org.apache.axis.AxisFault.makeFault(AxisFault.java:101) at com.itko.lisa.ws.axis.LisaHTTPSender.doInvoke(LisaHTTPSender.java:665) at com.itko.lisa.ws.axis.LisaHTTPSender.invoke(LisaHTTPSender.java:267) at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32) at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118) at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83) at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165) at org.apache.axis.client.Call.invokeEngine(Call.java:2837) at org.apache.axis.client.Call.invoke(Call.java:2820) at com.itko.lisa.ws.nx.NxWSStep.executeCall(NxWSStep.java:384) at com.itko.lisa.ws.nx.NxWSStep.execute(NxWSStep.java:289) at com.itko.lisa.test.TestNode.executeNode(TestNode.java:984) at com.itko.lisa.test.TestCase.execute(TestCase.java:1297) at com.itko.lisa.test.TestCase.execute(TestCase.java:1198) at com.itko.lisa.test.TestCase.executeNextNode(TestCase.java:1183) at com.itko.lisa.test.TestCase.executeTest(TestCase.java:1124) at com.itko.lisa.coordinator.Instance.run(Instance.java:208)Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:209) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160) at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84) at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) at com.itko.lisa.ws.axis.LisaHTTPSender.doInvoke(LisaHTTPSender.java:492) ... 15 more


  • 2.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Broadcom Employee
    Posted Jul 15, 2019 05:54 PM
    Looks like you have a Web Service execution step in your VSM and unable execute that request and thrown the below error.

    ERROR com.itko.lisa.ws.nx.NxWSStep   - Unable to invoke request: ; nested exception is: 2019-07-10 16:10:44,281Z (18:10) [Lotte_CB_1 [VS_ZSMART_Volume_Lotte_CB_1_Run]/27] ERROR com.itko.lisa.ws.nx.NxWSStep   - Unable to invoke request: ; nested exception is:  java.net.SocketException: Connection resetAxisFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException faultSubcode:  faultString: java.net.SocketException: Connection reset faultActor:  faultNode:  faultDetail:  {http://xml.apache.org/axis/}stackTrace:java.net.SocketException: Connection reset at 

    You can validate the step manually and make sure it works then deploy the VS and test it.


  • 3.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Posted Jul 16, 2019 10:44 AM
    Hi Premalatha,

    Yes. I've Web Service execution step in my VSM.

    This error is occurring sometimes only and during that time we observed, Virtual Services are becoming unresponsive.



  • 4.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Posted Jul 17, 2019 10:12 AM
    Hi,

    Similar behaviour has been reported before. Often it is due to a vulnerability scanner (port scanner) deployed within the organisation which sends unexpected payloads to the virtual service port.

    What is the base path of your virtual service? If it is "/" then assign a more specific base path. With a more specific base path the VSE will just discard the incoming request and will not pass it to the virtual service.

    Cheers,
    Danny

    ------------------------------
    Cheers,
    Danny
    ------------------------------



  • 5.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Posted Jul 17, 2019 11:36 AM
    Yes Danny, the same issue was reported before. Please guide me to verify the below 

    "Often it is due to a vulnerability scanner (port scanner) deployed within the organisation which sends unexpected payloads to the virtual service port.". 

    1. How to check the what vulnerability scanner (port scanner) that is deployed ?
    2. Is there any way to stop sending the unexpected payloads to the virtual service port ?

    We're using full base path name, instead of "/".
    i.e. /Virtualize/{{SERVICEPATH}}/{{SERVICENAME}}



  • 6.  RE: java.net.SocketException: Connection reset || Error in VSE log
    Best Answer

    Posted Jul 18, 2019 05:43 AM

    Hi,

     

    Re. 1. How to check the what vulnerability scanner (port scanner) that is deployed ?

     

    You will have to ask your IT Management and/or Operations team if such a solution is in use within your organization. From the side of the VSE there is no knowledge what type of client and/or solution is sending something to a listening port

     

    Re. 2. Is there any way to stop sending the unexpected payloads to the virtual service port ?

     

    There is not really a way to stop the VSE to transfer the incoming request to the virtual service. Since your VS has a detailed basepath it appears that the VSE was able to extract the HTTP command from the incoming request and pass the request to the virtual service.

     

    Do you have certainty that the incoming requests are NOT coming from your SUT? It could be that your SUT has an issue and sometimes sends out garbage?

     

    Does the VSE monitor shows an error count in the portal? Is the VS still running but unresponsive or did it stop? In the case it stops due to errors then you can adapt the Listen step in the VSM to not fail when there is an "If Environment Error" assertion fired but instead let the assertion loop back to the Listen step itself again to process the next request.

     

    Cheers,

    Danny

     

     

    ::DISCLAIMER::
    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------





  • 7.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Posted Aug 01, 2019 10:11 AM
    Hi Danny,

    Re 1:  I'll check with the team on the vulnerability scanner part.

    Re 2: Since, we've secured firewall connection to the DevTest server, I assume all the request are coming from SUT.

    I don't see any errors in the portal. VS are showing as running but still unresponsive.

    Currently, we're restarting the DevTest server - for VS to respond. 



  • 8.  RE: java.net.SocketException: Connection reset || Error in VSE log

    Posted Dec 17, 2019 08:07 AM
    Connection reset socket error occurs when the opponent is forcibly terminated without calling close(). This simply means that a TCP RST was received. TCP RST packet is that the remote side telling you the connection on which the previous TCP packet is sent is not recognized, maybe the connection has closed, maybe the port is not open, and something like these. A reset packet is simply one with no payload and with the RST bit set in the TCP header flags. There are several possible causes.

    • The other end has deliberately reset the connection, in a way which I will not document here. It is rare, and generally incorrect, for application software to do this, but it is not unknown for commercial software.
    • More commonly, it is caused by writing to a connection that the other end has already closed normally. In other words an application protocol error.
    • It can also be caused by closing a socket when there is unread data in the socket receive buffer.