DX Application Performance Management

Expand all | Collapse all

Webservice calls make APM slow

Jump to Best Answer

Haruhiko Davis12-01-2014 12:13 PM

  • 1.  Webservice calls make APM slow

    Posted 12-01-2014 11:50 AM

    Hi all!

     

    Here at my company, we are trying to integrate Introscope with other tools and to do it we are calling some APM WS at each 30s. We noticed that, when it reaches approximately 8 simultaneous calls, APM goes very slow.

     

    I think that there should be some tuning procedures to makes the responses faster, because every call took more than 8s to query the last 30s of a specific metric and agent. It's only one value retuned... I suppose the answer might be very quick.

     

    We are calling http://mti-introscope.sicredi.net/introscope-web-services/services/MetricsDataService, method getMetricData

     

    XML:

    <soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:met="http://metricsdata.webservicesimpl.server.introscope.wily.com">
       <soapenv:Header/>
       <soapenv:Body>
          <met:getMetricData soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
             <agentRegex xsi:type="xsd:string">(.*)\|CEM\|TESS Agent</agentRegex>
             <metricRegex xsi:type="xsd:string">Business Service\|REDE COLABORATIVA\|Business Transactions:Total Bussiness Transactions Per Interval</metricRegex>
             <startTime xsi:type="xsd:dateTime">2014-12-01T13:10:00.000Z</startTime>
             <endTime xsi:type="xsd:dateTime">2014-12-01T13:10:30.000Z</endTime>
             <dataFrequency xsi:type="xsd:int">30</dataFrequency>
          </met:getMetricData>
       </soapenv:Body>
    </soapenv:Envelope>
    

    Any ideas?

    Thanks!



  • 2.  Re: Webservice calls make APM slow

    Posted 12-01-2014 12:13 PM

    What version of APM are you using?



  • 3.  Re: Webservice calls make APM slow

    Posted 12-01-2014 12:15 PM

    Hi Haruhiko!

    I'm using APM 9.5.3.0.

    Thanks.



  • 4.  Re: Webservice calls make APM slow

    Posted 12-01-2014 12:25 PM

    Thanks for replying.

     

    I would also like to ask:

     

    • Is your environment clustered (APM cluster, not high availability)
    • What language are you using
    • Is it possible for you to share the code with us on the APM-Dev group (haven't signed up/not heard? We're located here CA Developer APM). It may be a better location to discuss this with you in detail and help troubleshoot.

     

    Please take a look at the materials posted. I know it may seem like a lot of reading, but it really is not much and should only take a few minutes. Discussions on the Dev subcommunity will be more focused on these types of discussions versus general APM queries/troubleshooting requests.



  • 5.  Re: Webservice calls make APM slow

    Posted 12-01-2014 02:09 PM

    1) Yes. Our environment is composed of two physical machines (IBM Blade, 23CPU, 145GB RAM). There are 4 collectors in each one plus a collector to CEM, in machine 1.

     

    2) Actually I'm using an ETL tool called Kettle. It's a component of Pentaho Suite. It makes a webservice call, transform the data and send it to an Oracle DB.

     

    3) As it is not a code I don't know if sending you the job will help. I'll join to this community you suggested and see if I find something useful to my case.

     

    Thank you very much!



  • 6.  Re: Webservice calls make APM slow

    Posted 12-01-2014 04:05 PM

    I will take a look at Kettle and let you know.

     

    Since you are running your cluster on two physical servers, I now need to ask about your SmartStor configuration.

     

    • For each collector, is the SmartStor on a dedicated disk with its own I/O path? If you did not do this, then be prepared for slowness while querying from the MOM.
    • Are you running your queries directed at the MOM and not the collectors?
    • Can you estimate how much data or how many rows you're returning at each polling?


  • 7.  Re: Webservice calls make APM slow

    Posted 12-03-2014 01:05 AM

    There is also a note in the guide about the use of these web services.

     

    "Note: Keep in mind that the more generic your regular expressions are, the more metric statistics are returned by the Enterprise Manager. That can cause OutOfMemory errors both on your Enterprise Manager and on your web services client. Such large queries can also cause significant slowdown on the Enterprise Manager. To avoid these situations, Enterprise Manager clamps the maximum number of metrics that are matched for one web service query. The default is 10,000, and it can be configured on the Enterprise Manager."



  • 8.  Re: Webservice calls make APM slow

    Posted 12-01-2014 02:39 PM

    Maybe if I run only one call to APM Webservice using the method getBulkMetricData, instead of several calls, should be better.

    Anyone here already used this method? How can I use array in allMetrics field?

     

    <soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:met="http://metricsdata.webservicesimpl.server.introscope.wily.com" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
      <soapenv:Header/>
      <soapenv:Body>
          <met:getBulkMetricData soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
            <allMetrics xsi:type="urn:ArrayOfAgentMetricName" soapenc:arrayType="urn:AgentMetricName[]" xmlns:urn="urn:ca.wily.introscope.webservices"/>
            <startTime xsi:type="xsd:dateTime">?</startTime>
            <endTime xsi:type="xsd:dateTime">?</endTime>
            <dataFrequency xsi:type="xsd:int">?</dataFrequency>
          </met:getBulkMetricData>
      </soapenv:Body>
    </soapenv:Envelope>
    


  • 9.  Re: Webservice calls make APM slow

    Posted 12-01-2014 10:47 PM

    I did not find getBulkMetricData listed in the API guide for 9.5. Can you upload the WSDL?



  • 10.  Re: Webservice calls make APM slow

    Posted 12-02-2014 05:56 AM
      |   view attached

    Yes, for sure!

    Follow file attached.

    Thanks!

    Attachment(s)



  • 11.  Re: Webservice calls make APM slow

    Posted 12-03-2014 01:00 AM

    Werle

     

    I haven't had a chance to startup one of my 9.5 VMs yet, but I do see from the bookshelf that this method is not documented.

    Here is what I see from the bookshelf.

     

    Metrics Data Web Service WSDL Definition

    WSDL URL

    http://<host>:<port>/introscope-web-services/services/MetricsDataService?wsdl

    Operations

    The following operations are used by this web service:

    WSDL URL

    http://<host>:<port>/introscope-web-services/services/MetricsDataService?wsdl

    Operations

    The following operations are used by this web service:

    Operation: getLiveMetricData
    Input

    getLiveMetricDataRequest

    agentRegex type string

    metricPrefix type string

    Output

    getLiveMetricDataResponse

    getLiveMetricDataReturn type array of type DTimeslicedResultSetMetricData

    • metricData - nullable; type ArrayOfMetricData - array of type MetricData agentName - nullable; type string

      metricName - nullable; type string

      metricType type int

      metricValue - nullable; type string

    • timesliceEndTime - nullable; type dateTime
    • timesliceStartTime - nullable; type dateTime
    Operation: getMetricData
    Input
    getMetricDataRequest

    agentRegex type string

    metricRegex type string

    startTime type dateTime

    endTime type dateTime

    dataFrequency type int

    Output

    getMetricDataResponse

    getMetricDataReturn type array of type TimesliceGroupedMetricData

    metricData - nullable; type ArrayOfMetricData - array of type MetricData

    • agentName - nullable; type string
    • metricName - nullable; type string
    • metricType type int
    • metricValue - nullable; type string
    timesliceEndTime - nullable; type dateTime

    timesliceStartTime - nullable; type dateTime

    Operation: getTopNMetricData
    Input

    getTopNMetricDataRequest

    agentRegex type string

    metricRegex type string

    startTime type dateTime

    endTime type dateTime

    dataFrequency type int

    topNCount type int

    decreasingOrder type boolean

    Output

    getTopNMetricDataResponse

    getTopNMetricDataReturn type array of type TimesliceGroupedMetricData

    • metricData - nullable; type ArrayOfMetricData - array of type MetricData

      agentName - nullable; type string

      metricName - nullable; type string

      metricType type int

      metricValue - nullable; type string

    • timesliceEndTime - nullable; type dateTime
    • timesliceStartTime - nullable; type dateTime
    Operation: getExtendedMetricData
    Input
    getExtendedMetricDataRequest
    • agentRegex type string
    • metricRegex type string
    • startTime type dateTime
    • endTime type dateTime
    • dataFrequency type int
    Output
    getExtendedMetricDataResponse
    • getExtendedMetricDataReturn array of type

    TimesliceGroupedExtendedMetricData

    • extendedMetricData - nullable; array of type
    • ExtendedMetricData
    Fault

    IntroscopeWebServicesException



  • 12.  Re: Webservice calls make APM slow

    Posted 12-03-2014 01:36 AM

    Hi,

     

    before we talk about the nitty-gritty details can you answer two questions first:

     

    • Why do you need to export metrics from CA APM every 30 seconds? What do you do with them? Why poll every 30 seconds?
    • How many metrics are you currently exporting every 30 seconds? How many would you like?

     

    Ciao,

    Guenter



  • 13.  Re: Webservice calls make APM slow

    Posted 12-03-2014 01:38 AM

    I also asked the same questions previously.



  • 14.  Re: Webservice calls make APM slow
    Best Answer

    Posted 12-03-2014 02:46 PM

    Hi,

    I know that it may sound weird, but we want to plot some metrics in real time (almost it). It is not a often practice, but this is an important task to us. I think that APM Webservice is not ready to this job because we tested the same thing with CLWorkstation client and it did the trick.

     

    The webservice calls was changed to:

     

    java -Xmx64M -Duser=admin -Dpassword=xxxxx -Dhost=mti-introscope.sicredi.net -Dport=5001 -jar /opt/ETLDashboards/lib/CLWorkstation.jar "get historical data from agents matching ($AGENT) and metrics matching ($METRIC) between \"$NOWL30\" and \"$NOW\" with frequency of $FREQ seconds"

     

    Thanks to everyone that helped me!!



  • 15.  Re: Webservice calls make APM slow

    Posted 12-03-2014 04:17 PM

    Be aware that the CLW does not have the same safeguards built in that the UI has; large historical queries from the CLW can negatively affect the overhead and performance of the EM.  If you're pulling data with great frequency, you might wish to explore the persistent collections in the EM; there's a few different ways to fork the data directly from SmartStor to a CSV or relational DB.