Clarity

Expand all | Collapse all

Apache Tomcat Log Analysis in v14.1

Jump to Best Answer

Anon Anon10-24-2014 05:03 PM

  • 1.  Apache Tomcat Log Analysis in v14.1

    Posted 10-24-2014 06:34 AM

    Hi Friends,

     

    We completed a clarity upgrade from v13.0 to v14.1. So glad to see all those smart sections working like expand/collapse switches. We were also excited about the new performance analyze feature added for Tomcat users in v14.1. CA has added 5 new tables and 2 jobs for this new feature, as shown below:

     

    ■ LOG_DETAILS

    ■ LOG_FILES

    ■ LOG_REPORTDEFS

    ■ LOG_SESSIONS

    ■ LOG_SUMMARY

    and the 2 jobs being:

    1) Tomcat Access Log Import/Analyze Job

    2) Delete Log Analysis Data Job

     

    Two days after the upgrade (once we were sure that we got some access logs to analyze the performance), we ran the first job to import logs and fill the above log tables. But to our surprise, only the LOG_SESSIONS and LOG_REPORTDEFS tables are being populated. No data in the other 3 tables

     

    Did we miss anything here? Any other configuration to be done to activate this performance analysis feature? Please let us know if you were able to populate the other log tables as well.

     

    P.S. The upgrade to v14.1 was so smooth. Thanks CA

     

    Regards,

    Georgy



  • 2.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-24-2014 05:03 PM

    You are way too ahead buddy.......



  • 3.  Re: Apache Tomcat Log Analysis in v14.1

    Broadcom Employee
    Posted 10-27-2014 05:52 AM

    Hi Georgy,

     

    Yes we added few tables for detailed log analysis and we will be soon shipping a portlet package which you can install using content addin and use. So watch out the community space


    Regards

    Suman Pramanik



  • 4.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-28-2014 01:54 AM

    Hi Suman,

     

    Thanks a lot for letting us know about the upcoming portlet package. I guess this future portlet also will be looking in the above newly added log tables for displaying the log analysis data, rt?

     

    Our issue here is that even after runing Tomcat Access Log Import/Analyze job, these 3 tables (LOG_DETAILS, LOG_FILES, LOG_SUMMARY) are empty. Is there anything else to be done other than running this job?

     

    Regards,

    Georgy



  • 5.  Re: Apache Tomcat Log Analysis in v14.1

    Broadcom Employee
    Posted 10-28-2014 02:02 AM

    Hi Georgy,

     

    Yes those portlet will pull the data from the tables mentioned,

     

    Tomcat Access Log Import/Analyze

    • §Job Function

    This job imports the tomcat access logs into the Clarity db schema.

    • §Job Specifications

    Takes an optional date argument specifying which date to retrieve log files for

    Date defaults to previous day if not specified.

    Dynamically discovers the app services within the Clarity cluster and fetches/imports tomcat access logs for those services.

    Inserts detailed information and summary information into the Log Analysis db schema tables

    • §Recommended Configuration

    Schedule after midnight in the same timezone as the app servers.

    This will automatically retrieve/import the previous days logs just after Tomcat stops writing to them at the end of the day.


    Hope this will help


    Regards

    Suman Pramanik



  • 6.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-28-2014 02:36 AM

    Hi Suman,

     

    Yes, I also thought the same. As you said, this job should be inserting detailed and summary information into the log analysis db schema tables which its not doing now

     

    Regards,

    Georgy



  • 7.  Re: Apache Tomcat Log Analysis in v14.1

    Broadcom Employee
    Posted 10-28-2014 02:49 AM

    Hi Georgy,

     

     

    This job design is to externally bulk import logs from other Clarity environments , so i believe it will populate when you import the logs from other environment.

     

     

    However i will again test myself and will let you know.

     

     

    Regards

    Suman Pramanik



  • 8.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-28-2014 02:56 AM

    Hi Suman,

     

    Great! thanks a lot for testing this. Will be waiting for your results.

     

    Regards,

    Georgy



  • 9.  Re: Apache Tomcat Log Analysis in v14.1
    Best Answer

    Broadcom Employee
    Posted 10-28-2014 09:29 AM

    Hey Georgy,

     

    The problem i had with my system was my beacon was not working properly, so i stopped the beacon, added again and started and it works like charm.

     

    select * from LOG_FILES

    app-access-2014-10-28.logapp11128-OCT-14local10.130.111.20228-OCT-14

     

    I ran only for today and it populated. Try the workaround and let me know.

     

    Regards
    Suman Pramanik



  • 10.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-28-2014 10:45 AM

    Hi Suman,

     

    wow, that worked  Just restarting the beacon didn't fix it. Also when we go to the new Process Monitor page in CSA, we had an error there suggesting us to put a proper multicast address and a bind address.

    process monitor.png

    As all our DEV, UAT, TRAIN, STG systems were using the same default multicast address, among which only the DEV is in v14.1, I guess that caused this issue. We gave different values for the DEV multicast address and multicast port. Also gave the DEV server IP as the Bind Address in CSA as our DEV env only has one server, no cluster.

     

    After all this, removing the beacon, adding it back again, starting it  fixed both the issues. The error in the Process Monitor page in CSA is gone and all the 3 log tables are now geting populated with data We ran the populate job two times giving yesterday and today as the Log Date parameter.

    log files.png

    Thanks a lot for your time in fixing this issue...really appreciated.

     

    Regards,

    Georgy



  • 11.  Re: Apache Tomcat Log Analysis in v14.1

    Broadcom Employee
    Posted 10-28-2014 10:57 AM

    Hi Georgy,

     

    You are welcome , Yes the log analysis & Process Engine depends on the correct configuration of beacon.

     

    Regards
    Suman Pramanik



  • 12.  Re: Apache Tomcat Log Analysis in v14.1

    Broadcom Employee
    Posted 10-28-2014 08:14 AM

    Wow Mr. Joseph,

    Very impressive to see such a smooth upgrade to 14.1 just 3 weeks after the GA date for this new release of CA Clarity PPM. Says a lot about CA, the upgrade process, and the maturity/agility of both CA and your organization. Are you working on CA Clarity on-premise or on-demand (SaaS)? Also, are you supporting one large customer or several customer instances of CA PPM? Just want to get a sense of what your challenges are day to day.

     

    Kind Regards,
    Damon

    CA Technologies



  • 13.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 10-28-2014 09:02 AM

    Hi Damon,

     

    Yea, the upgrade went smoothly. Happy to get this done so fast in our DEV environment . We are yet to plan our Production upgrade as we guess v14.2 will be out before the end of this year. Our customer has CA clarity installed on-premise and they only have one large instance of CA PPM.

     

    Regards,

    Georgy



  • 14.  Re: Apache Tomcat Log Analysis in v14.1

    Posted 09-19-2016 11:51 PM

    Hi

    I hope some one else already faced this issue and have some resolution in place.

    We have a audit requirement to maintain all the logs of client level event / activities.

    Apart from the Date, time, URL Actioned we need following information also.

    1. User of Application / System

    2. Source IP Address

    3. Success or Failure of events

     

    For item 3 we can use browser status, but we cant find Source IP address information. Is this possible ?

    Also for User of Application / System Account, we thought we would use session_cookie info from LOG_DETAILS  and link with LOG_SESSIONS table to find the actual user id, but this field (sesion_cookie) is always blank.

    Note: I am working on SaaS environment. CA PPM 14.4.0.234.

    I did check with  another environment on 14.3.0.29, i don't have the same issue. Is this an issue in 14.4 ?

     

    What are all the scenarios this session cookie is updated as "-"  ?

     

    Any suggestion / advice please..

     

    Regards

     

    Venky