How can we modify the min/max heap settings for this agent ?We dont want to rebuiild the agent.
Looking at the startup scripts we can see that the min/max values can be passed as params, but how to do this ?
As we have a large estate the agent is crashing due to default heap of 16/256 being too small
./APMIAgent.sh startwithlogs: APM Infrastructure Agent started
APM Infrastructure Agent Start is Completed
Tue Apr 10 11:41:45 UTC 2018 [INFO] APM Infrastructure Agent Start is Completed
4/10/18 11:41:51 AM UTC [INFO] [IntroscopeAgent.OSEMetadataInfo] Operation Name = null
4/10/18 11:41:58 AM UTC [INFO] [IntroscopeAgent.IntelligentInstrumentationService] Intelligent Instrumentation Service activated
4/10/18 11:42:43 AM UTC [ERROR] [IntroscopeAgent.InfrastructureAgent Agent] Error on getting last deployment status for heapster
4/10/18 11:42:48 AM UTC [INFO] [IntroscopeAgent.InfrastructureAgent Agent] AutoProbe has reloaded all directive configuration. This is because a change to a directive configuration file was detected.
4/10/18 11:43:02 AM UTC [ERROR] [IntroscopeAgent.OutgoingRouteConnector] Closing outgoing route connector due to unrecoverable error.
java.lang.OutOfMemoryError: GC overhead limit exceeded
4/10/18 11:43:02 AM UTC [ERROR] [IntroscopeAgent.InfrastructureAgent Agent] The thread named "UnknownHub Hub Transmit 2" has unexpectedly terminated because of java.lang.OutOfMemoryError GC overhead limit exceeded
4/10/18 11:43:09 AM UTC [ERROR] [IntroscopeAgent.InfrastructureAgent Agent] The thread named "Thread-3" has unexpectedly terminated because of java.lang.OutOfMemoryError GC overhead limit exceeded
The pod remains Running. We have attempted a couple of restarts, all produce the same results.
When reviewing the container, I can see the /usr/local/openshift/apmia/run.sh calls apmia-ca-installer.sh which in turn call bin/APMIAgent.sh
APMIAgent.sh takes default min/max heap from within the script as it expects them to be passed as arguments when being executed ($2 and $3)
Is it possible to set these as ENV variables?
Funnily enough I was looking at the same thing for the collector agent. Yes you can is the answer
If you define a couple of variables MIN and MAX than call teh script with them as follows
# export MAX=512# export MIN=128 ./*sh start $MIN $MAXStarting /collectors/CollectorAgent-10.5.2/jre/bin/java -server -Xms128m -Xmx512m -cp lib/CollectorAgent.jar:lib/gson.jar:lib/Agent.jar -XX:ErrorFile=/collectors/CollectorAgent-10.5.2/logs/jvm_error.%p.log com.wily.introscope.agent.collector.CollectorAgent./CollectorAgent.sh start: CollectorAgent (pid 15426) started
If they are not set and you still call it with those arguments it just uses the default as the $2 $3 are missing
# unset MIN# unset MAX# ./*sh start $MIN $MAXStarting /collectors/CollectorAgent-10.5.2/jre/bin/java -server -Xms16m -Xmx256m -cp lib/CollectorAgent.jar:lib/gson.jar:lib/Agent.jar -XX:ErrorFile=/collectors/CollectorAgent-10.5.2/logs/jvm_error.%p.log com.wily.introscope.agent.collector.CollectorAgent./CollectorAgent.sh start: CollectorAgent (pid 15478) started
hope this helps
Hi Mike, its not what we want. As the supplied "run "script is already running inside the Docker image we cant pass any parameters to do it without rebuilding the Docker image locally, which we would rather not do. Ideally we thought the min/max values might be able to get picked up from the Openshift environment where we can set them, rather than as command line args that we can't set
I wont pretend to be an Openshift/Docker expert but if you have a script in the docker image that references variable names like in my example MIN MAX then you can set these externally to values or not. I admit this would need a rebuild to get it to be able to read the variables in the first place, but only once to set the mechanism up, you could then change the variable values in Openshift (if I understand you correctly).
If I'm still missing something - apologies for that.
Hi Mike, it looks to us like the actual agent startup script that is supplied can only take the values as command line arguments. It doesn't appear to try to get them from the environment. So if there is no way to get them from the environment, we are going to have to modify the supplied image and rebuild that. We think we have got a workaround, but just wanted to know from CA if we were missing something.
Mike is saying that you will need to edit the OOTB APMIAgent.sh script to be able to process the values of MIN & MAX as environment variables. I do agree that you would also need to disable the argument checking for the heap values. The following changes work for me:
# changes for passing heap values in argumentsMIN_HEAP_VAL_IN_MB=16MAX_HEAP_VAL_IN_MB=256
MIN_ARG_PRESENT=trueif [ -z "$2" ] then MIN_ARG_PRESENT=falsefi
MAX_ARG_PRESENT=trueif [ -z "$3" ] then MAX_ARG_PRESENT=falsefi
# modified to pass in heap values as environment variablesMIN_HEAP_VAL_IN_MB=16MAX_HEAP_VAL_IN_MB=256
if [[ "$MIN" ]]; then echo "Setting MIN_HEAP_VAL_IN_MB to environment variable value for MIN" MIN_HEAP_VAL_IN_MB=$MINfi
if [[ "$MAX" ]]; then echo "Setting MAX_HEAP_VAL_IN_MB to environment variable value for MAX" MAX_HEAP_VAL_IN_MB=$MAXfi
# disable heap argument checking
#MIN_ARG_PRESENT=true#if [ -z "$2" ]# then# MIN_ARG_PRESENT=false#fi
#MAX_ARG_PRESENT=true#if [ -z "$3" ]# then# MAX_ARG_PRESENT=false#fi
Hope that helps
Thanks Lynn and Mike
Ok we understand what is required now. We've spoken to Nishant also, and we understand that there will be a mod to the script released at some point so the heap size can be picked up from the environment without having to amend the supplied script.