VSE_METRICS_PERIODICS Table Details

[Rahul Verma]

I am trying to get some information from VSE_METRIC_PERIODICS table to genarate some useful report out of it But not clear about below things. Kindly provide details how this works. All Vse metric properties in Lisa.properties file are kept as default. 

Version used: 8.0

Databse : Derby

- Is metric value for response time in millisecond and is this averaged for all the trnsactions recived 

- By default VSE logs in table every 5 minutes . So is it like it only logs trnsaction count recived at that particular time or sum of all the trnsactions recived within that 5 minutes .

- I want to get the transactions at particular second with response time and genrate something useful.

[Rahul Verma]

MarcyNunns deoma03  Kindly help here Bit urgent

[Marcy Nunns]

I will have to find out how the data is stored.

 

On the second point; the VSE will collect the details in memory and then write out the details to the database every 5 minutes for these tables:
VSE_METRICS_EVENTS
VSE_METRICS_PERIODICS
VSE_METRICS_TXN_COUNTS 

 

On the last point; it is suggested to do a test run for at least 15 minutes (so there should be at least 2 * 5 minute intervals covered) and then check the contents of the VSE_METRICS_PERIODICS table. Note that the VS_NAME lists the name of the Virtual Service and PERIOD_TS should show the Time Stamp. You could export the table to a csv file and then further manipulate the rows in a spreadsheet to calculate the average of a specific run. 

 

Hope this helps.

[Marcy Nunns]

I found a bit more information so might be able to answer your first question:

 

PERIOD_LENGTH is the sampling time (300013) would be 30 seconds 13 ms. 

[Rahul Verma]

Thanks Mercy. I will test this for Load /Endurance test since we need to get actual trnsactions at micro level . 

TPS we recieve as of now varies from 50-200 . I will post my observations here . kindly check for more information.

[Rahul Verma]

Hi Mercy . I tested one service with 100 Vusers for 15 minutes test . Below i got in Periodics table.

 

 

 

VS_NAME	METRIC_MAX_VALUE	METRIC_MIN_VALUE	METRIC_NAME	METRIC_VALUE	PERIOD_LENGTH	PERIOD_TS
Test_sv	56	0	Response Time	1	300000	11/27/2018 22:45
Test_sv	150	19	Transactions/Second	133	300000	11/27/2018 22:45
Test_sv	74	0	Response Time	1	300017	11/27/2018 22:50
Test_sv	147	96	Transactions/Second	133	300017	11/27/2018 22:50
Test_sv	100	0	Response Time	1	300010	11/27/2018 22:55
Test_sv	176	79	Transactions/Second	133	300010	11/27/2018 22:55

 

 

 

 

 

 

 

 

 

 

 

What i can interpret here is that response time is avergaed to 1Millisecond but Avergae response time is noticed in Test Run during statging is different . 

 

I want to genearte a report for particular service that shows tps along with Avg response time by fetvhing data from Perioidc count table . How can i do that at micro level . 

 

i also tried setting lisa.vse.metrics.sample.interval=1s and 1 minute but that averges to same . just huge records i get that is expected .

[Marcy Nunns]

I will do more research.

[Rahul Verma]

Thanks Marcy . I will wait for your reply

[Will_Truong]

Hi Rahul,

 

Can you share some details on how you are executing your load test? Are you directly load testing the virtual service? Or do you have a system in between your load test and the virtual service? Knowing the architectural setup will help us better understand your performance testing scenario.

 

Can you also share how you intend to use the metrics that you're collecting? Is it for auditing purposes where you collect metrics around the virtual services to see how they are being utilized? Or is it for another purpose? If so, please share these details as well, as there may be a better way to do what you're intending to do.

[Rahul Verma]

Hi William,

 

Here i am directly hitting the virtual service just to check the table data how it behaves. But actual scenario is below what i am trying to achieve.

We have virtualized different vendor systems to be utilized for performnace testing which are working well enough . Between front end (UI) and Vendor system there is a middleware that hits different vendors (virtulized) and consumes responses . TPS varies from 150-200. in version 8.0 we don't have reporting templates where i can get TPS and response time for each service at microlevel so that we can have something useful for testing team and internal use as well. 

Then i came across lisa internal tables that stores VSE metrics at service level ,something useful to me. i am able to extract information from tables using SQL queries i have designed as per my requirment but how some fields are storing data is still not clear to generate a useful report and lisa.vse.metrics.sample.interval property is not handling value less than 1 sec if i want metrics sampling after few milliseconds ,

[Will_Truong]

Hi Rahul,

 

Thanks for clarifying the performance test setup/scenario.

 

The difference that you see in the response time could be the fact that the virtual service only records how long it takes for it to respond, whereas the average response time in the staged test run result factors in the entire RTT (round trip time), meaning it records from the time the request is sent plus the time for the service to respond plus the actual time for it to be sent back over the wire. So you're comparing two different type of metrics.

 

So are you just concerned about the discrepancies of the results from the staged test run results and the VSE database metrics table? Or are you not able to get the level of detail of metrics from the VSE database?

 

I'm not sure if there's a way to lower the value of the lisa.vse.metrics.sample.interval property. It might be this way by design as anything lower than 1sec may overload the database connection.

 

If you really need a workaround for this, then one suggestion would be to customize the VSM to calculate the information that you're looking for. For example, if you want to collect the response time then you can add a Timestamp filter on the Listen step and on the Responder step and then add a separate scripting step after the Responder step to calculate the response time using the two timestamp values from the listen and responder step.

 

I'm attaching a sample project with the above example implementation for your reference.

 

Timestamp filter on Listen step:

 

Timestamp filter on Responder step:

 

Scripting step after the Responder step:

Attachment(s)

VSEMetrics.zip   17 KB 1 version

[Rahul Verma]

Hi William,

Thanks for your reply.

I would like to get actual metrics that are stored in VSE table for all the services which i am able to get using SQL queries on VSE Periodics table . 

I have set lisa.vse.metrics.sample.interval=1s to get metrics at each second . But i have same question .

 

For METRIC_NAME "Response Time" . Are values stored as milliseconds . I believe those are not in seconds or anyhting else because TPS is much high and Concurrency i have set to 13 . Kindly confirm on this. 

Below is extract from VSE_PERIODIC table tested for a service keeping Sample interval to 1 second. 

 

I would generate a dynamic graph from this that shows TPS and Response time so that we can leverage this for reporting purpose . Also let me know do i need to perform any averaging or anyhting else on this or this data is good to go . 

 

Extract from VSE_PERIODCS. Service name and VSE Name is replaced with soemthing dummy. Rest values are actual.

 

PERIODIC_METRIC_ID	VSE_NAME	VS_NAME	METRIC_MAX_VALUE	METRIC_MIN_VALUE	METRIC_NAME	METRIC_VALUE	PERIOD_LENGTH	PERIOD_TS
71005	Vse_Link	Test_Service	125	125	Transactions/Second	125	1001	11/27/2018 23:41
71009	Vse_Link	Test_Service	112	5	Transactions/Second	58	1079	11/27/2018 23:41
71003	Vse_Link	Test_Service	85	85	Transactions/Second	85	1006	11/27/2018 23:41
71004	Vse_Link	Test_Service	145	3	Response Time	29	1006	11/27/2018 23:41
71010	Vse_Link	Test_Service	246	1	Response Time	8	1079	11/27/2018 23:41
71006	Vse_Link	Test_Service	49	2	Response Time	7	1001	11/27/2018 23:41
71011	Vse_Link	Test_Service	149	149	Transactions/Second	149	1013	11/27/2018 23:41
71007	Vse_Link	Test_Service	122	122	Transactions/Second	122	1005	11/27/2018 23:41
71008	Vse_Link	Test_Service	53	2	Response Time	4	1005	11/27/2018 23:41
71012	Vse_Link	Test_Service	59	1	Response Time	3	1013	11/27/2018 23:41
71022	Vse_Link	Test_Service	5	1	Response Time	1	1000	11/27/2018 23:41
71018	Vse_Link	Test_Service	59	1	Response Time	3	1001	11/27/2018 23:41
71019	Vse_Link	Test_Service	134	134	Transactions/Second	134	1001	11/27/2018 23:41
71016	Vse_Link	Test_Service	34	1	Response Time	2	1002	11/27/2018 23:41
71013	Vse_Link	Test_Service	140	140	Transactions/Second	140	1003	11/27/2018 23:41
71017	Vse_Link	Test_Service	132	132	Transactions/Second	132	1001	11/27/2018 23:41
71014	Vse_Link	Test_Service	15	1	Response Time	2	1003	11/27/2018 23:41
71021	Vse_Link	Test_Service	135	135	Transactions/Second	135	1000	11/27/2018 23:41
71015	Vse_Link	Test_Service	130	130	Transactions/Second	130	1002	11/27/2018 23:41
71020	Vse_Link	Test_Service	16	1	Response Time	1	1001	11/27/2018 23:41
71030	Vse_Link	Test_Service	5	1	Response Time	1	1000	11/27/2018 23:41
71023	Vse_Link	Test_Service	125	125	Transactions/Second	125	1001	11/27/2018 23:41
71026	Vse_Link	Test_Service	10	1	Response Time	1	1010	11/27/2018 23:41
71024	Vse_Link	Test_Service	14	1	Response Time	1	1001	11/27/2018 23:41
71032	Vse_Link	Test_Service	7	1	Response Time	1	1007	11/27/2018 23:41
71028	Vse_Link	Test_Service	12	1	Response Time	1	1002	11/27/2018 23:41
71029	Vse_Link	Test_Service	130	130	Transactions/Second	130	1000	11/27/2018 23:41
71027	Vse_Link	Test_Service	129	129	Transactions/Second	129	1002	11/27/2018 23:41
71025	Vse_Link	Test_Service	144	1	Transactions/Second	72	1010	11/27/2018 23:41
71031	Vse_Link	Test_Service	131	131	Transactions/Second	131	1007	11/27/2018 23:41
71038	Vse_Link	Test_Service	6	1	Response Time	1	1000	11/27/2018 23:41
71036	Vse_Link	Test_Service	5	1	Response Time	1	1000	11/27/2018 23:41
71034	Vse_Link	Test_Service	13	1	Response Time	1	1000	11/27/2018 23:41
71040	Vse_Link	Test_Service	5	1	Response Time	1	1011	11/27/2018 23:41
71037	Vse_Link	Test_Service	129	129	Transactions/Second	129	1000	11/27/2018 23:41
71041	Vse_Link	Test_Service	134	134	Transactions/Second	134	1000	11/27/2018 23:41
71035	Vse_Link	Test_Service	130	130	Transactions/Second	130	1000	11/27/2018 23:41
71039	Vse_Link	Test_Service	137	137	Transactions/Second	137	1011	11/27/2018 23:41

 

 

 

[Will_Truong]

Hi Rahul,

 

I'm not familiar with this metrics table and we will probably need confirmation from support/development team MarcyNunns

 

But my guess is that the metrics are in milliseconds and the response time is the average response time for the duration, which is 1sec in this case. Again this needs to be further confirmed..

[Rahul Verma]

No prob William. She is working on the same . Thank you

[Marcy Nunns]

Rahul,

 

 

From the research I have done as well as asking other engineers, all of the data should be stored as milliseconds.

 

~Marcy

[Rahul Verma]

Thanks MarcyNunns . I was able to capture total Count as well . Now i have to play with SQL queries to get meanigfull data . 

Thanks everyone for your inputs.