I have created virtual service for Performance testing purpose and deployed in PERF VSE. Virtual Service simple one having one request and response pair. Web Application is under performance test which uses Virtual Service in back end. Service will be hit by 60 TPS. Virtual Service is responding properly for all transactions.
But when i checked "Virtual Service Response Time" Report from portal, I could see Spikes intermittently during the load test. Not able to find any error or info regarding why Spikes occurred. Below is an example.
In the above graph, During 8hr run, few spikes occurred.
How to identify what caused these spikes in response times and what might be the reasons.
We have high configuration server, 24 core processor and 64 GM RAM.
Some random thoughts on this:
I have also experienced similar behaviour during performance tests- Do you have anything special in your virtual service like match scripts or custom coding?
Thank you Jamesdunn and Danny,
Added my comments. Could you please check and advice.
Average transactions per second graph is steady during the spike time.
8 GB Memory is configured for VSE process. And i am not sure how to check is there any JVM garbage collection activity occurred during that time.
I have consulted Admin team and got confirmation that no other process kicked in during that time.
I have seen this kind of behaviour very frequently during performance test runs. I could not see any details in logs also.
This particular service doesn't contain any thing like match script or Custom Coding. Can you please let me know, if virtual service contains any Match Script or Custom Coding, will it affect the performance.
I am getting similar behaviour with some of our services and it's also causing issues:
Transactions per Second: