QoS data from logmon (multiple variables)

[Anon Anon]

Hi, I was wondering whether anyone has run into the following issue:

I'm parsing request timing data from logfiles that's presented in the format hh:mm:ss.ms with a regexp, the particular log shows the timings for a request in this format and occasionally some requests take more than one minute to process. Is there any way, as i extract the hours, minutes, seconds and milliseconds into separate variables, to combine these values arithmetically into a single millisecond count within nimbus?

Naturally I could do homebrew tricks to accomplish this on the server side, but I'd much prefer to be able to accomplish this without hacking outside Nimbus.

[Anon Anon]

Are you doing this from within the logmon probe?  You mentioned data comes from a log file, but I just want to be sure.  I would doubt that you could perform any mathematical operations like that within the logmon probe once you have the variables picked out.

If you do not mind sharing a bit more information, I would be curious to see a sample line from the log file and the section of the logmon config for that log file.  Perhaps someone here will be able to suggest another possible solution.

Probably a dumb question: I assume you have no control over what goes into the lines in the log file?

Regards,
Keith

[Anon Anon]

Yes, using the logmon probe, can't affect the format easily, and in any case, I'd expect i'm not/or will not be the only one who'd run into this. These particular entries look like the following:

2008-02-06 17:10:17,584  DEBUG XML-API-Performance  - XML API call 'LoadBooking_IN' took: 00:00:00.0937524

Which I'm matchin from the logmon probe with: /(.*),.* XML API call '(.*)' took: (+)+)+).(+)/

And assigning to variables on the variables tab and enabling as QoS data on the QoS tab.

Edit: added config
   <Integration_logs>
      active = yes
      interval = 1 min
      scanfile = C:\logs\SWWebServicesLogs\xml-api-log.txt
      scanmode = updates
      alarm = yes
      qos = yes
      message = no
      <excludes>
         <min1sec>
            active = no
            match = /.*00:00:0().(+)/
         </min1sec>
      </excludes>
      <watchers>
         <Performance>
            active = yes
            match = /(.*),.* XML API call '(.*)' took: (+)+)+).(+)/
            level = clear
            subsystemid =
            message = Integration XMLAPI slow response: $when $call took: $hour:$min:$sec
            restrict =
            expect = no
            abort = no
            sendclear = no
            count = yes
            separator =
            suppid =
            source =
            target = Integration_logs.Performance
            qos = DaftPunkSamiTest
            runcommandonmatch = no
            commandexecutable =
            commandarguments =
            <variables>
               <when>
                  definition = $1
                  operator =
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Stamp
               </when>
               <sec>
                  definition = $5
                  operator = lt
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Seconds
               </sec>
               <min>
                  definition = $4
                  operator =
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Minutes
               </min>
               <hour>
                  definition = $3
                  operator =
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Hours
               </hour>
               <call>
                  definition = $2
                  operator =
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Call
               </call>
               <msec>
                  definition = $6
                  operator =
                  threshold =
                  qosactive = yes
                  qosname = slow_xmlapi_response
                  qostarget = Integration_logs.Performance.Stamp.msec
               </msec>
            </variables>
         </Performance>
      </watchers>
   </Integration_logs>

[Anon Anon]

Great example!  Thanks for sharing.

Unfortunately, I do not see a way to combine those values within logmon.  Maybe someone else will have an idea.  Like you said, doing something external to the probe might be your only option.  If I were trying to do this myself, I would probably make a short script that accepts the hours, minutes, seconds, and milliseconds values as arguments, combines them, and then uses the nimqos command in the bin directory to send the data to the QoS database.

Regards,
Keith