We are doing a performance test on API Gateway. We scheduled a load for 10 Mins and what we have noticed is irrespective of load (no. of threads concurrent threads and Jobs) Gateway is not taking more than 80K requests.
Can someone help use to understand which parameter is restricting this traffic and how to increase it?
There is not really any parameters to throttle the load as such, sounds like you have already adjusted the max ae http threads. You can adjust the connections per backend but by default these are not set so that won't apply. Can you explain more about this test, for example are there any backend systems involved?
We are doing performance testing on Gateway and we are using JMeter for this.
In JMeter we have configured three parameters as listed below:
1. No. of Threads (Users): 100
2. Ramp-Up period (in Secs): 1 -> The duration at which all threads are activated
3. Schedulers: Start Time and End Time - We gave for 10 Mins
No. Requests Gateway Processed: around 81K.
1. No. of Threads (Users): 200
1. No. of Threads (Users): 1000
This time JMeter configured in two separate Desktops with same configurations as shown below which will run during the same period.
In each machine No. of Requests Processed are 40K
Total No. Requests Gateway Processed in Gateway: around 81K. (around 40K*2)
This time JMeter configured in three separate Desktops with same configurations as shown below which will run during the same period.
In each machine No. of Requests Processed are 27K
No. Requests Gateway Processed: around 81K. (27K * 3)
if you look at above results, gateway is not processing more than 81K irrespective of the load. So what is holding gateway to process more request? Any help is appreciated.
Can someone answer this, please?
The load may be opening a ton of connections that remain open and are affecting performance. You can try to enable pool management and timeout idle sessions:
- add the following to the /opt/SecureSpan/Gateway/node/default/etc/conf/system.properties file:com.l7tech.server.policy.assertion.ServerHttpRoutingAssertion.statePool.enable=true
- modify this cluster-wide property from 0s to 5sio.httpConnectionIdleTimeout
Restart the gateway after making these changes.Thanks
Hello Kareem.shaik7 ,
First of all, we should understand the delay is on gateway or on backend. Could you please attach a sceenshot of Dashboard?
on policy manager -> menu View -> Dashboard -> select Resolution as Daily -> click on the bar when performance test occurred.
Here is the screenshot of Dashboard
Dear Kareem.shaik7 ,
gateway avg time is 3701-1155=2546ms, backend time is 1155ms. bottleneck is on gateway side, but backend also not in good performance.
1. reduce DB operations as much as possible, ie. ensure the audit level threshold is WARNING or above(cluster-wide property audit.detailThreshold), remove or disable all Audit Messages In Policy assertion, disable all the audit detail assertion, or set them as INFO level.
2. set as Kemal Ajan suggested, particularly, you need to set io.httpConnectionIdleTimeout (to 5s)
3. you can increase http concurrent settings, but ensure you have enough cpu and memory on gateway server (8 cpu, 16GB is recommended) , and ensure the backend can accept the concurrent requests as high as io.httpMaxConcurrnecy
As your set jmeter as 1000 threads, you may try following settings,
io.httpCoreConcurrency = 1000
io.httpMaxConcurrnecy = 1250
(before restart mysql, stop all gateway nodes first)
ping from jmeter to gateway, and ping from gateway to backend, ensure all network latency less than 10ms