DX Application Performance Management

Expand all | Collapse all

CEM - TIM not parsing transactions anymore

  • 1.  CEM - TIM not parsing transactions anymore

    Posted Apr 18, 2017 11:04 AM

    Hi all,

     

     

    I've changed some CEM users/gropus, and I completely lost transaction statistics.

     

    I'm checking TIM status and I see that

    Total TranComps      Total TranUnits and Total TranSets are 0 on TIM.

     

    Does this mean that TIM is not parsing that traffic anymore?

    I don't see anything in CEM Console about my transactions, but Introscope keeps seeing them (Avg response time, Responses per interval, etc ... )

     

    How can I check if tim is correctly configured, or why it is not matching Tran definitions? I downloaded conf and log files and I don't see anything strange.

     

    TIM is on phisical RHEL

     

    Regards,

    Roger



  • 2.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 18, 2017 11:47 AM

    It looks like a permissions issue when you make changes to your uers/groups. Can you revert back those changes and see if you are able to see everything you want to?

    Are you using EEM for authentication or just the MoM's user's and domain xml files?



  • 3.  Re: CEM - TIM not parsing transactions anymore

    Broadcom Employee
    Posted Apr 18, 2017 12:31 PM

    Dear Rogelio:

       You didn't say how you removed the user and groupsIf done from the database side only, it could break pointers with other tables. Key is to check what is being populated in the database tables. 

     

    Are the Business transactions enabled to give defects and statistics ?

     

    Manish's points on permissions is good . But I would expect that to be logged somewhere

     

    Would need a webex to explore further. This type of issue is best solved with a case.

     

    Thanks

    Hal German



  • 4.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 18, 2017 12:36 PM
    Total TranComps      Total TranUnits and Total TranSets are 0 on TIM.

    And you don't see any transactions on CEM Console.

    It looks like your TIM is not reading/processing traffic.

    The edits of CEM user/user group configurations on CEM Console should not be a factor causing this issue.

    Has your TIM been working before, or it has never been?  When was the last time it is upgraded.

    A support ticket seems to be in order in this case.

    Francis


  • 5.  Re: CEM - TIM not parsing transactions anymore
    Best Answer

    Broadcom Employee
    Posted Apr 18, 2017 06:29 PM

    Yes if no data is being matched by TIM the root cause looks to be unrelated to user permissions and more a coincidence it occurred at the same time. NOTE: Those status values would also get reset to 0 when the TIM is restarted.

    If you are using SSL check the SSL Server Status page for any increase in decode failures which may be due to expired web server keys or change in cipher suite to something that the TIM cannot support.

     

    When you said "I don't see anything in CEM Console about my transactions, but Introscope keeps seeing them (Avg response time, Responses per interval, etc ... )"

    Are those visible metrics from agents or are they Customer Experience Metrics?

    https://docops.ca.com/ca-apm/10-5/en/using/apm-metrics/customer-experience-metrics

    Based on 0 data reported by TIM Status I would not expect CE metrics to be visible either (they come from TIM generated BtStats data via TIM Collection Service RTTM child process), but it is worth clarifying.

     

    Thanks,

     

    Lynn



  • 6.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 18, 2017 09:12 PM

    Hi everybody,

     

    thanks for answering.

     

    The fact is that users were moved by updating the database; as we considered data corrupted, we decided to install a new database (with the schemas scripts) and import the transactions again. Note that the transactions were exported when we already received no CE data.

     

    TIM was working fine until the users were moved.

     

    Since then we only see Custom Experience Management Mentrics as Lynn's link expose:

     

    *SuperDomain* | Custom Metric Host |Custom Metric Proces...|Custom Business Appl...| By Business Service |Service Name|Transaction|Component|Average Response Time (ms)

     

    and for each Agent hosting components for that transaction, we see the "Business Segment" subtree.

     

     

     

    Besides, as TIM Status keeps acumulating Total Packets and Total Components, we consider its getting traffic, but not matching the transactions definitions. If I record transactions, I see the components.

     

    When I check Tess Collector logs, I see things like "defect aggregation received - size: 0 bytes".

     

    I have a support ticket, but we are not getting to the core of the solution.

     

     

    Regards,

    Roger



  • 7.  Re: CEM - TIM not parsing transactions anymore

    Broadcom Employee
    Posted Apr 18, 2017 10:30 PM

    Hi Roger,

    I found your ticket and I will make some suggestions to Fernando fjvargas to try to help with root cause analysis.

    Regards,

    Lynn



  • 8.  Re: CEM - TIM not parsing transactions anymore

    Broadcom Employee
    Posted Apr 19, 2017 09:05 AM

    Dear Rogelio"

         If a Webex is held on this, I would like to attend to see if I can help fjvargas please note. Marking as answered since there is a case on this. If a resolution is discovered, please post to help others that may have this situation. There may be two issues here:

     

    1) Definitions not matching

    2) Not seeing defects/statistics

         Thanks

    Hal German



  • 9.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 19, 2017 03:15 PM

    Thanks! I'm trying to contact Fernando, but , check this from the collector log:

     

    4/19/17 04:01:00.322 PM ART [INFO] [StatsConversionPool.Thread2] [Manager.com.timestock.tess.services.collectors.StatsConversionTask] Stats conversion task starting
    4/19/17 04:01:00.322 PM ART [INFO] [StatsConversionPool.Thread3] [Manager.com.timestock.tess.services.collectors.StatsConversionTask] Stats conversion task starting
    4/19/17 04:01:00.322 PM ART [INFO] [StatsConversionPool.Thread4] [Manager.com.timestock.tess.services.collectors.StatsConversionTask] Stats conversion task starting
    4/19/17 04:01:00.322 PM ART [INFO] [StatsCollector.Thread1] [Manager.com.timestock.tess.services.collectors.StatsCollector] Downloading stats for 2017-04-19 15:00
    4/19/17 04:01:00.322 PM ART [INFO] [StatsConversionPool.Thread5] [Manager.com.timestock.tess.services.collectors.StatsConversionTask] Stats conversion task starting
    4/19/17 04:01:00.330 PM ART [INFO] [StatsCollectionPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'timc-2017-04-19-18-00.1492624800.stats' read, length=0
    4/19/17 04:01:00.330 PM ART [INFO] [StatsCollectionPool.Thread1] [Manager.com.timestock.tess.services.collectors.StatsDownloadTask] Processing file 'timc-2017-04-19-18-00.1492624800.stats' from monitor 'timc' (10.250.4.170); length = 0...
    4/19/17 04:01:00.330 PM ART [INFO] [StatsCollectionPool.Thread1] [Manager.com.timestock.tess.services.collectors.StatsDownloadTask] Processed 0 records in 0 secs

     

    TIM is sending empty stats ... so TIM is not matching definitions ... but stopped matching them just after data corruption in the database.

     

     

    Regards,

    Roger



  • 10.  Re: CEM - TIM not parsing transactions anymore

    Broadcom Employee
    Posted Apr 19, 2017 03:51 PM

    would be interested what your timfiles log looks like 



  • 11.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 19, 2017 05:07 PM

    Hi Hallet,

     

    logs.tgz is attached to the case 00711490: CEM not creating user/group stats

     

    Regards,

    Roger



  • 12.  Re: CEM - TIM not parsing transactions anymore

    Posted Apr 19, 2017 11:41 AM

    thanks rogelio.dipasquale for providing that feedback/update and to Lynn_Williams for the explanation on what could be wrong and that you are getting the help you need from Support. All the best

     

    Manish