Clarity

Expand all | Collapse all

System Errors in app service

  • 1.  System Errors in app service

    Posted 01-27-2017 07:20 AM

    Hello everyone,

     

    There is somewhat a recurring issue with one of our productions environments with ca ppm 14.3.0.298 (with sql server 2012), in which the app service seems not to be working properly, when its like this it doesn't even show the login screen. The average user load is around 300-400 users logged in, during peek times.

    The application seems to work fine after a service restart, but this is causing some issues in the user experience having to restart the services every time this happens so i was wondering if there is any hints on what the issue might be. I attached the logs from the app-ca.log and app-system.log.

     

    Is this a problem with the database connection? or some kind of configuration of the application? Do i need to schedule a service restart for the app service?

     

    Thank you!

     

    Attachment(s)

    zip
    app-ca.log.zip   855B 1 version
    zip
    app-system.log.zip   998B 1 version


  • 2.  Re: System Errors in app service

    Posted 01-27-2017 07:25 AM

    Can you check if this helps?

     

    13.1 database compatibility query - 14.1

    https://communities.ca.com/message/241732943  

     

    NJ



  • 3.  Re: System Errors in app service

    Broadcom Employee
    Posted 01-27-2017 08:54 AM

    Hi Tiago,

     

    You can check the app utilization memory and CPU, you also need to see if your system is properly sized or not.

     

    Regards

    Suman Pramanik



  • 4.  Re: System Errors in app service

    Posted 01-31-2017 10:37 AM

    The APP service is averaging around 4.5 - 4.7 MB in memory utilization, the VM that is runing in has 16GB of memory with the CPU having 4 cores at 2.5GHz.

    There are also 3 BG services running all averaging at 2.2 MB of utilization each.

    In total the VM is using ~81% of memory.

     

    Is this acceptable in terms of resources for this amount of users? Is this memory utilization what is causing the issue?

     

    I also found this https://www.ca.com/us/services-support/ca-support/ca-support-online/knowledge-base-articles.tec1241682.html. But my DB is MSQL Server 2012.

    Can i also consider on applying the patch indicated, and hope to resolve this issue?

     

    Thank you.



  • 5.  Re: System Errors in app service

    Broadcom Employee
    Posted 01-31-2017 11:15 AM

    If you stop the ppm services what is your VM utilization? Also as you said you have 3 BG, does all run Process engine? Also What is the APP JVM and BG JVM in properties.xml?



  • 6.  Re: System Errors in app service

    Posted 01-31-2017 01:09 PM

    I cant stop the services now but i'm gessing its a prety low value since the VM is only running PPM.

    Here are the settings for the services:

    APP
    -Xms4096m -Xmx6144m  -Dsun.lang.ClassLoader.allowArraySyntax=true -Duser.country=US -Duser.language=en -XX:-UseGCOverheadLimit

     

    BG (Run Job Scheduler)
    -Xms4096m -Xmx4096m  -XX:-UseGCOverheadLimit

     

    BG 2 (RUN Process Engine)
    BG 3 (RUN Process Engine)



  • 7.  Re: System Errors in app service

    Broadcom Employee
    Posted 01-31-2017 01:17 PM

    Hi Tiago,

     

    A simple math during a peak load

     

    APP consumes 6 GB at max 

    BG consumes 4GB * 3 BG = 12 GB 

     

    Total = 18 GB and you have just 16 GB RAM, hence it will go and ask for swap memory and will cause performance issues 

     

    You should never allocate application more than 80% capacity of RAM.

     

    Hope this helps in resizing.

     

    Regards

    Suman Pramanik 



  • 8.  Re: System Errors in app service

    Posted 02-01-2017 05:36 AM

    I have corrected the values before posting, i did not realize it was configured that way. Could this configuration be the cause for the lost connection to the database by the app service?

     

    Thank you.



  • 9.  Re: System Errors in app service

    Broadcom Employee
    Posted 02-01-2017 07:16 AM

    Lost connection to the database is nothing to do with your JVM configuration, you have said that your VM utilization is high and what I suspect is due to your sizing. If this is not the correct one then please let me know whats your JVM configuration