ValueOps ConnectALL Product Community

 View Only

How does a TIM worker get its traffic ?

By Jörg Mertin posted Mar 24, 2017 12:29 PM

  

On all TIM's 9.7+,The captured network communication flow is handled by apmpacket (on the High performance TIM) or nqcapd (on the MTP). Its function is very simple. It will get data from a network device or capture-card, and distribute the network communications based on a load-balancing algorithm defined by the client or server IP address, to the available TIM Workers.

 

The main differences between apmpacket and nqcapd are as follow:

apmpacket:

  • apmpacket: similar functionality as nqcapd
  • gets traffic from regular network interfaces, or through a
    special setup (ask CA Support for details) from a Napatech capture
    board.

 

Note: that filter configuration for Napatech board needs to be written manually in a text file, and will be uploaded during start through the ntpcap.ini file.

Note 2: Any change done to the ntpcap.ini (Napatech card configuration) will require a restart of apmpacket! As long as any program is accessing the data-feed, the Napatech board cannot load the new configuration.

 

nqcapd:

  • nqcapd: Main capture program provided on the MTP
  • Receives traffic exclusively through the Napatech capture board
  • MTP UI provides UI to configure filters etc.
  • nqcapd needs to be restarted after each filter-change

 


When talking about "network communication", the TIM requires the communication flow that happens between a user's Web-browser, and the responses sent back by the Web-Server in its entirety. Any missing packet can invalidate the entire Monitoring of that communication, or cause a defect to show up. If that missing packet has been lost inside the SPAN infrastructure, this will be a false-positive (as the real traffic is not impacted) which is not wanted. (But that has been handled in a previous blog).

Every complete communication traffic from every user will then be dumped in pcap format in a TIM workers dedicated directory.

By default, there are as many workers as there are CPU Cores on the system. On a 16Core system, you'll have 16 timworkers actively monitoring their respective data directories for the data to be analyzed.

 

Example on a CA6000 series Multiport collector:

[root@ca-mtp /]# find /nqtmp/tim/0/w* -type d
/nqtmp/tim/0/w0
/nqtmp/tim/0/w1
/nqtmp/tim/0/w10
/nqtmp/tim/0/w11
/nqtmp/tim/0/w12
/nqtmp/tim/0/w13
/nqtmp/tim/0/w14
/nqtmp/tim/0/w15
/nqtmp/tim/0/w2
/nqtmp/tim/0/w3
/nqtmp/tim/0/w4
/nqtmp/tim/0/w5
/nqtmp/tim/0/w6
/nqtmp/tim/0/w7
/nqtmp/tim/0/w8
/nqtmp/tim/0/w9

 

Note - for performance reasons the nqtmp/tim directory is usually mounted as a RAM file-system with limited capacity. This is why it is imperative that there is enough space available, and the TIM workers fast enough to empty their respective queue. If the TIM's are not fast enough, out-of-space errors will be logged in the TIM logs.

 

As for the size of the temporary Ram-Filesystem - it does not really make sense to make it very large (4 to 6GB). If a TIM worker stops working, that queue will not be emptied anymore and it will fill up. Thing is that when this happens, usually something else is wrong, so increasing the size will not really help.

1 comment
1 view