Javascript Calculator: Does Introscope stop a JavaScript if the execution time is too long or longer than the defined frequency

[Anon Anon]

Hello guys,

 

at one of our customers we will use a JavaScript Calculator in the near future. The Calculator is executed every 15 seconds and we assume that the processing will always be less than 2 - 3 seconds. Nevertheless, the customer and I would like to know how Introscope behaves if the Calculator needs more time. That's why I included a small hack in the script that simulates a longer execution time. I assumed that Introscope will log something if the script needs longer than the frequency, here 15 seconds. However, even if I delay the execution by 2 minutes Introscope does not stop the script from running. There is nothing written in the log. Maybe it takes quite some iterations. I will run it the next days on a test instance to see if Introscope stops executing the script.

 

The Calculator writes some metric back to Introscope. It looks like that Introscope just uses the provided metric value for all passed script iterations, meaning if value 10 is returned after 2 minutes execution time, I can see the value 10 for that metric for start, start + 15s, start + 30s, ... end of script (~2 minutes). Quite interesting this behaviour.

 

Does anyone know how Introscope deals with long lasting JavaScript Calculators? Yet, I could find something in the forum or documentation.

 

Thanks for your responses.

 

Greetings,

Matthias

[Anon Anon]

Update:

 

I performed a long running test over the weekend. I don't find any log entry which directly indicates a problem  due to the long running JavaScript. The below snippet shows two log entries from my script (start and end) which logged 2 minutes after another (2 minutes delay, perfect!). Then a warning tells me that the timeslice processing was delayed and that timeslices will be combined. JavaScript still running!!

 

12/01/14 03:38:29.329 PM CET [INFO] [Harvest Engine Pooled Worker] [Manager.JavaScript|MBConnect-Alerts.js] WAIT

12/01/14 03:40:29.330 PM CET [INFO] [Harvest Engine Pooled Worker] [Manager.JavaScript|MBConnect-Alerts.js] END

12/01/14 03:40:29.819 PM CET [WARN] [master clock] [Manager.Clock] Timeslice processing delayed due to system activity. Combining data from timeslices 94496314 to 94496321

 

The JavaScript writes a metric to Introscope. Prior to the test, I thought that the value will be used for all timeslices between start and end of the JavaScript. This is not true because I see in the Investigator a metric value every 2 minutes.

 

My current conclusion:

- Long-lasting JavaScripts are not stopped.

- No information that the JavaScript delayed the system activity.

- Timeslice processing delayed due to system activity. Combining data from timeslices ... => can be a clue that the JavaScript takes too much time

- Long-lasting JavaScripts can falsify metrics because timeslices will be skipped

[Guenter DI Grossberger]

Hi Matthias,

 

I have neither tried it out nor do I know 100% how the scripts are executed and what happens in parallel and what not.

 

Are other metrics processed normally while the script is running for 2 minutes? Or don't you get any metrics at all while your script is sleeping?

 

Of course you don't get any metrics reported from your script while it is sleeping. EM is a real-time process. Once once computation cycle is over after 15s you cannot change what is reported in that interval. Metrics from agents or scripts are always only reported, stored and use for other computations (calculators, alerts, ...) in the current interval. If for some reason the computation for that interval is not finished it will be combined with the next interval. Therefore you get the "combining timeslices" warning.

 

If your script takes more than 15s it is not suited for running in the EM!!! Actually it should be completed in a few milliseconds.

 

Why do you think or what are you doing that your scripts should take several seconds?

 

Ciao,

Guenter

[Anon Anon]

Hello Guenther,

 

thanks for your response.

 

> Are other metrics processed normally while the script is running for 2 minutes? Or don't you get any metrics at all while your script is sleeping?

All the metrics I had a look at are only reported every 2 minutes. 'All' includes also metrics which are not processed or written by the javascript. To be fair, I'm not 100% sure if the sleep mechanism was suitable to test a long-lasting javascript behavior. That's why I do not take this behavior to serious.

 

> Of course you don't get any metrics reported from your script while it is sleep ...

Thanks for the information which help a lot to understand how Introscope works and what the specific log warning means.

 

> If your script takes more than 15s it is not suited for running in the EM!!! Actually it should be completed in a few milliseconds. Why do you think or what are you doing that your scripts should take several seconds?

I don't think that the script will ever take longer than 2-3 seconds. It should be less then 1 sec or even less than 300-500 ms. But you never know if due to unknown circumstances the script will take more than 15 seconds. Therefore, it is always good to know how a tool will handle this situation. Especially if this could cause problems for your whole monitoring environment.

 

Greetings,

Matthias

[Guenter DI Grossberger]

Hi Matthias,

 

always glad to help.

 

To reduce the impact of the script iterate over as few metrics as possible. Specify the regular expressions in getAgentRegex() and getMetricRegex() as strictly as possible - which requires a naming convention for agents and metrics which is a Best Practice anyways.

Ciao,

Guenter

[Anon Anon]

Hello Guenter,

 

thanks for pointing this out. We follow already this Best Practice. Agent names and metric use a naming convention. The regular expressions are as strict as they could be.

 

Greetings,

Matthias