This is the first time I have sent a post on the community in months. There is an internal document (was for MTP) that was created explaining this. I see no reason why it cannot be shared.
Tim protocol statistics files
This describes the format of the Tim packet statistics files stored in the /etc/wily/cem/tim/logs/protocolstats and /etc/wily/cem/tim/logs/protocolstats1 directories. These files are written by the tim program and read by the viewstats web page.
The files are in csv (comma-separated value) format. Each line contains one protocol statistics entry. There are no header or trailer lines.
Each line consists of a number of fields. Fields can be scalars or one-dimensional arrays. A scalar field consists of two comma-separated values: the name and the value. An array field consists of a number of sets of 3 comma-separated values; in each set, the first value is the field name, the second is the index (starting at 0), and the third is the value for that index. The currently defined arrays have names starting with "cpu-" and "w-". Numbers are in decimal.
A partial example is:
ver,1,fmt,m,cpu-use,0,70%,cpu-use,1,85%
The first field consists of the name "ver" (version) and the value "1". The second field consists of the name "fmt" (format) and the value "m" (multi-process). The third and fourth fields form the "cpu-use" array, where cpu-use[0] is 70% and cpu-use[1] is 85%.
Each line can have either of two formats: one for a single-process Tim and one for a multi-process Tim. The latter includes arrays that are indexed by the worker number. If Tim is restarted in a different mode, a single file can contain lines in both formats. The "fmt" field indicates which.
The following tables list the fields. A value in the "index" column indicates that the field is an array, in which case the index is a decimal number starting at 0.
The following fields are common to a single-process and multi-process Tim. The ones that mention MTP are present only when running on an MTP machine.
Name
|
Index
|
Value
|
ver
|
|
Version number of this line – always 1
|
fmt
|
|
m for multi-process or s for single-process
|
time
|
|
Date and time of this entry
|
pkts-capture
|
|
Number of packets captured
|
pkts-drop
|
|
Number of packets dropped
|
pkts-forward
|
|
Packets forwarded from MTP
|
pkts-nospace
|
|
Packets dropped by MTP because of no space to write them
|
pkts-short
|
|
Short packets received by MTP, possibly because user defined hardware filters incorrectly
|
pkts-tooold
|
|
Old packets not processed. Their timestamp is older than (now –TimMtpLimitPeriodInMinute * 60). Default value of TimMtpLimitPeriodInMinute is 15
|
pkts-analyze
|
|
Number of packets analyzed
|
bytes-analyze
|
|
Number of bytes analyzed
|
thruput
|
|
Throughput
|
stats
|
|
Number of statistics records open
|
cpu-use
|
CPU number
|
Time used by this CPU
|
The following fields are present for a multi-process Tim:
Name
|
Index
|
Value
|
hub-cpu
|
|
Percentage of CPU time used by the hub
|
hub-mem
|
|
Memory used by the hub
|
w-cpu
|
Worker number
|
Percentage of CPU time used by this worker
|
w-mem
|
Worker number
|
Memory used by this worker
|
w-conn
|
Worker number
|
TCP connections managed by this worker
|
w-ts
|
Worker number
|
Transets open for this worker
|
w-tu
|
Worker number
|
Transets open for this worker
|
w-tc
|
Worker number
|
Transets open for this worker
|
w-ssl
|
Worker number
|
SSL sessions open for this worker
|
w-lgn
|
Worker number
|
Logins managed by this worker
|
Records for a single-process Tim have the following fields:
Name
|
|
Value
|
tim-cpu
|
|
Percentage of CPU time used by Tim
|
tim-mem
|
|
Memory used by Tim
|
conn
|
|
TCP connections managed by Tim
|
ts
|
|
Transets open
|
tu
|
|
Transets open
|
tc
|
|
Transets open
|
ssl
|
|
SSL sessions open
|
lgn
|
|
Logins managed
|
Original Message:
Sent: 01-20-2020 04:03 AM
From: Joerg Mertin
Subject: Packets TIM
Hia,
there is no docs explaining exactly this, but from experience, here is what it means.
What you need to understand is that the TIM 9.6+ is using a third-party app to collect the data from the network sources (Network Capture board: Napatech, or network interface). The apm-packet copies the captured traffic to a dedicated directory (usually a tmpfs/Ram filesystem of 4GBytes of size) per TIM worker, and the TIM workers will in turn each look out for the pcaps assigned to it. Once the TIM has retrieved the pcap to work on, it will delete it from the file-queue on the tmpfs.
That - for a little understanding on the process.
Now - to the actual explanation of the content:
- Captured: Shows the number of packets the apm-packet process has captured from the network source
- Dropped: Shows the number of packets that have been dropped from the network source due to overload, missing space etc.. If a drop happens, something is wrong. A health-check should be done.
- Forwarded: The number of packets that have been forwarded from apm-packet to the TIM workers
- No-Space: means that the TIM workers (most common cause) can't process (actually analyze) the amount of data provided by apm-packet. There are various reasons this could happen. In that case, a health-check should be done.
- Too Short: Don't know that one, sorry. Maybe @Hallett German remembers what it was...
- Too Old: The data in the tmpfs filesystem is too old and is discarded by the TIM workers. Usually when the TIM process was stopped without stopping the apmpacket for a while. This also should not happen. Also causes the No Space counter to rise.
- Analyzed: The actual data packets that have been analyzed by the TIM workers (actually retrieved by the TIM workers).
- Bytes Analyzed: Number of bytes the TIM's have actually analyzed. note that the size of a packet can go from few bytes up top 9MBytes (for Jumbo Frames) per packet.
- Throughput in Kilo Bytes per second: Computed throughput of analyzed data by the TIM.
BONUS: If you want to have some more details, try adding a: "&unsupported=1" at the end of the URL of that page in the browser. You'll have the full per TIM Worker details displayed.
Note: Performing a health-check for the TIM can be done (accelerated) using the cem-healtcheck scripts : https://github.com/CA-APM/cem-healthcheck-scripts
Original Message:
Sent: 01-19-2020 12:13 AM
From: Richard Briceno
Subject: Packets TIM
Hi Teams,By chance there is some documentation that explains the meaning of the different Tim Packet StatisticsIt is not very clear, when forward packages, analyzed packages, captured packages are shown.I wish I could understand what each item of packages means.
Thanks
Richard B