How/where get traffic to be monitored

[Harold Martin]

Good day,
The CEM setup is running on a VM and we need first to show to security group the TIM is able to not capture/store sensible data.

At the moment, there is an issue, where the traffic is being seeing by the TIM but not TRX is  being captured: nor ATD nor Manual recorded.. no nothing!

I'm doing all by the book, literally speaking, with my old CEM book course by my side, just to be more than confident on every step.

Here is a network diagram of the setup (simplified)

------------------------------
APM Senior Consultant
Netready Solutions
------------------------------

[David Lewis]

Hi Harold,
If you say the TIM is seeing traffic, how do you know ? Is it based on packet statistics only ?

Can you enable logging in the TIM for HTTP and SSL information to see what actually comes out, also gather a pcap from the monitoring interface to see what is coming in. It might be a cipher that the TIM can't decrypt for example because of the nature of the cipher not being suitable for passive decryption. Assuming SSL is involved, check the SSL Server stats in the TIM page, is there a positive count for unsupported cipher suites there

It needs to be two-way traffic, so both the request and response, and it does need to have the HTTP data in there, they should be full packets, not just header information that other tools can work with.

You can use the scripts written by Joerg to do some diagnosis, in this case at least the TIM script and PCAP

https://github.com/CA-APM/cem-healthcheck-scripts

Many thanks,
David

[Jörg Mertin]

After receiving a packet capture from Harold, the issue is due to not seeing bi-directional traffic. See below traffic analysis (Note - I have replaced the public IP's by RFC1918 ones for privacy purposes). In a regular packet capture, we have the IP-couples change side all the time as every request sees an answer from the remote host. In this case, we only see one direction of the request. Not the answer.

=== Checking full communication. Check bidirectional traffic flow (Port 80, 8080 or 443) !  
   29 2020-06-03 05:39:29,490362 10.10.70.237 → 10.10.10.124 59522 80 0 0 128 TCP 59522 → 80 [SYN, ECN, CWR] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
   30 2020-06-03 05:39:29,506424 10.10.70.237 → 10.10.10.124 59523 80 0 0 128 TCP 59523 → 80 [SYN, ECN, CWR] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
   31 2020-06-03 05:39:29,632224 10.10.10.234 → 10.10.3.139  18938 80 0 0 44 TCP 18938 → 80 [SYN, ECN, CWR] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=602325070 TSecr=0 WS=128
   32 2020-06-03 05:39:29,632266 10.10.10.234 → 10.10.3.139  18938 80 0 0 44 TCP [TCP Out-Of-Order] 18938 → 80 [SYN, ECN, CWR] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=602325070 TSecr=0 WS=128
  329 2020-06-03 05:39:29,707716   10.10.3.29 → 10.10.3.139  2949 80 0 0 128 TCP 2949 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
  330 2020-06-03 05:39:29,707934   10.10.3.29 → 10.10.3.139  2949 80 0 0 128 TCP [TCP Out-Of-Order] 2949 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1
  331 2020-06-03 05:39:29,707976   10.10.3.29 → 10.10.3.139  2949 80 0 1 128 TCP 2949 → 80 [ACK] Seq=1 Ack=1 Win=64240 Len=0
  332 2020-06-03 05:39:29,707985   10.10.3.29 → 10.10.3.139  2949 80 0 1 128 TCP [TCP Dup ACK 331#1] 2949 → 80 [ACK] Seq=1 Ack=1 Win=64240 Len=0
  333 2020-06-03 05:39:29,707991   10.10.3.29 → 10.10.3.139  2949 80 0 1 127 TCP [TCP Dup ACK 331#2] 2949 → 80 [ACK] Seq=1 Ack=1 Win=64240 Len=0
  334 2020-06-03 05:39:29,707998   10.10.3.29 → 10.10.3.139  2949 80 0 1 127 TCP [TCP Dup ACK 331#3] 2949 → 80 [ACK] Seq=1 Ack=1 Win=64240 Len=0​

What needs to be done is provide the Monitored TIM interface bidirectional traffic. Make sure the vlan provided to the TIM actually has Tx and Rx mirrored.

------------------------------
Customer Solution Engineering
Broadcom Inc.
------------------------------

[Harold Martin]

Thanks for the answers!!!

The provided Traffic Mirroring was created by the VMWare administrator and I suspect he doesn't know how to set it properly, neither do I.
I got the following doc from vmware: https://docs.vmware.com/en/VMware-vSphere/6.0/com.vmware.vsphere.networking.doc/GUID-C3970893-9328-430C-A440-F22B7E26280A.html and on it there is the option to select Port Mirroring Session Type: 

• Distributed Port Mirroring
• Remote Mirroring Source
• Remote Mirroring Destination
• Encapsulated Remote Mirroring (L3) Source
• Distributed Port Mirroring (legacy)

Which one must be used?

Thanks a lot for the help @Jörg Mertin and @David Lewis​​​​

------------------------------
APM Senior Consultant
Netready Solutions
------------------------------

[Jörg Mertin]

Hi Harold,

I think you need to use:

Remote Mirroring Destination	Mirror packets from a number of VLANs to distributed ports.

where the distributed port is the one the TIM capture process is listening for traffic.
I assume this because of the drawing where you show the TIM to be linked to the VLAN 10.10.3.x.

FYI - the TIM does not care where its data (network data to be monitored) comes from. What it really cares about is clean data.
VLAN's are OK. Encapsulated data in any way is NOT OK.

------------------------------
Customer Solution Engineering
Broadcom Inc.
------------------------------