System Status - Harvester - Last Reaper Timestamp update time is 0

[Pedro Filipe Lourenco]

Hi,

 

My NFA console is showing this (NFA - Administration - Health - System Status):

 

Harvester - Last Reaper Timestamp update time is 0

Harvester - Last RealTimeTimestamp update time is 0

 

 

Anyone know what is this? What can I do to solve this?

This is a fresh installation-  CA Network Flow Analysis 9.5.0 (build101)

NFA Console - Windows server 2012r2

NFA Harvester - Linux 3.10.0-862.9.1.el7.x86_64 x86_64

I don't have any NetFlow traffic yet.

 

BR,

Pedro Lourenço

[Christopher Walsh]

It will show those two messages until you start collecting data.

Basically its saying the Reaper Service hasn't processed any data yet. 

[Pedro Filipe Lourenco]

Hi Christopher,

 

Now I have Netflow traffic, but still the same.

 

I have files on HarvesterArchive, but on the Reaper folders I don't have anything.

 

The service is working, and on log I can only see this:

 

[root@nfa_harvester Logs]# service nfa_reaper status
● nfa_reaper.service - SYSV: CA NFA Reaper Service. Consumes output from CA NFA Harvester.
Loaded: loaded (/etc/rc.d/init.d/nfa_reaper; bad; vendor preset: disabled)
Active: active (exited) since Fri 2018-12-21 09:47:24 WET; 1h 32min ago
Docs: man:systemd-sysv-generator(8)

Dec 21 09:47:24 nfa_harvester.lab.local systemd[1]: Starting SYSV: CA NFA Reaper Service. Consumes output from CA NFA Harvester....
Dec 21 09:47:24 nfa_harvester.lab.local systemd[1]: Started SYSV: CA NFA Reaper Service. Consumes output from CA NFA Harvester..
Dec 21 09:47:24 nfa_harvester.lab.local nfa_reaper[16881]: Starting CA NFA Reaper: [ OK ]
Dec 21 09:47:24 nfa_harvester.lab.local su[16885]: (to nfa) root on none
[root@nfa_harvester Logs]#

   

[root@nfa_harvester Logs]# cat RealTimeReaperErrors2018-12-21.log

 

**********
Start - GMT Offset=0 - Rollover at Sat Dec 22 00:00:00 2018
**********
09:46:29

**********
Start - GMT Offset=0 - Rollover at Sat Dec 22 00:00:00 2018
**********
09:47:24

**********
Start - GMT Offset=0 - Rollover at Sat Dec 22 00:00:00 2018
**********
[root@nfa_harvester Logs]# 

 

 

Can you help?

 

BR,

Pedro Lourenço

[Christopher Walsh]

Do you see data in the GUI for any devices from this harvester?

ReaperInput should be empty most of the time, ReaperWork should process files every 15 minutes.

ReaperArchive and ReaperArchive15 should have lots of sub directories in each. 

[Pedro Filipe Lourenco]

Hi,

 

On GUI I can't see any data.
On the harvester I did a tcp dump and I can see netflow packets from my routers reaching the harvester on port 9995.

 

BR,

Pedro Lourenço

[Christopher Walsh]

The netflow data may not be in the right format you can verify that by following the kb article below:

How can I determine if a NetFlow enabled device is - CA Knowledge 

 

When data is processing fine, you should first see devices/interfaces on the Admin->Enable interfaces page, then after some time, you will see interfaces get licensed and appear on the Admin->Physical and Virtual page, and then they should start showing data.

 

Do you see anything on the Admin->Enable interfaces page in NFA?

[Pedro Filipe Lourenco]

Hi,

 

I can't see anything on "Admin->Enable interfaces"
I will check if the netflow data is on the right format.

 

Thanks.
Pedro Lourenço

[Pedro Filipe Lourenco]

Hi, Christopher_Walsh 

This is the packet:

[Christopher Walsh]

Flow fields seem to be okay. Note that this is not a private forum so your IP addresses are exposed publicly by uploading that screenshot.  We would recommend opening a case and uploading the pcap there as it would be secure.

 

 

What version of RedHat is this harvester?

Are the local firewalls disabled on the server?

 

Are all CA services running?

 

Can you check for errors in the /opt/CA/NFA/Netflow/logs directory.

 

The first place you would see files on the harvester would be in the /opt/CA/NFA/Netflow/HarvesterArchive directory there should be files over 1kb.  If not, there is either a harvester service issue or the local firewall may be blocking the flows.

 

If you see files in the /opt/CA/NFA/Netflow/HarvesterArchive then check the /opt/CA/NFA/Netflow/ReaperArchive directory.

 

If nothign in theere, see if there are lots of files built up in the /opt/CA/NFA/Netflow/ReaperInput, /opt/CA/NFA/Netflow/ReaperWork, and /opt/CA/NFA/Netflow/HarvesterWork 

[Pedro Filipe Lourenco]

Hi,

 

I can see filer over 1kb on HarvesterArchive .

 

But no files on ReaperInput,ReaperWork or HarvesterWork.

 

All services ar up and running.

No firewalls(LAB)
CentOS Linux release 7.5.1804 (Core)

 

Br,

Pedro Lourenço

[Christopher Walsh]

What about ReaperArchive.

 

We do not support CentOS for Linux Harvesters.

The latest RedHat version we support is 7.4 with NFA 9.5.

[Pedro Filipe Lourenco]

Hi,

ReaperArchive it's clean as well.

 

Br,

Pedro Lourenço

[Christopher Walsh]

Anything in the log files I referenced above?

 

Check if the firewalld service is running

 

systemctl stop firewalld.service

 

Also this is an unsupported operating system, I would recommend installing on RedHat 7.4 or earlier.

[Pedro Filipe Lourenco]

Hi, firewall is off.

 

Only this on logs:

==> harvester-wrapper.log <==
INFO | jvm 1 | 2019/02/13 15:07:25 | java.rmi.RemoteException: Timed out while waiting for connection to com.ca.im.ra.poller.Poller because Connection refused to host: 10.150.178.3; nested exception is:
INFO | jvm 1 | 2019/02/13 15:07:25 | java.net.ConnectException: Connection refused (Connection refused)
INFO | jvm 1 | 2019/02/13 15:07:25 | at com.ca.im.ra.util.RMI.RmiClient.GetInstance(RmiClient.java:52)
INFO | jvm 1 | 2019/02/13 15:07:25 | at com.ca.im.ra.harvester.InterfaceMapper.work(InterfaceMapper.java:74)
INFO | jvm 1 | 2019/02/13 15:07:25 | at com.ca.im.ra.base.business.ServiceThread.run(ServiceThread.java:63)
INFO | jvm 1 | 2019/02/13 15:07:25 | at java.lang.Thread.run(Thread.java:748)
INFO | jvm 1 | 2019/02/13 15:07:25 | 3:07:25 PM - [SEVERE ] - Encountered RemoteException attempting to map interface with ifIndex=533 for router 10.150.17.155
INFO | jvm 1 | 2019/02/13 15:07:25 | java.rmi.RemoteException: Timed out while waiting for connection to com.ca.im.ra.poller.Poller because Connection refused to host: 10.150.178.3; nested exception is:
INFO | jvm 1 | 2019/02/13 15:07:25 | java.net.ConnectException: Connection refused (Connection refused)
INFO | jvm 1 | 2019/02/13 15:07:25 | com.ca.im.ra.util.RMI.RmiClient.GetInstance(RmiClient.java:52)
INFO | jvm 1 | 2019/02/13 15:07:25 | com.ca.im.ra.harvester.InterfaceMapper.work(InterfaceMapper.java:74)
INFO | jvm 1 | 2019/02/13 15:07:25 | com.ca.im.ra.base.business.ServiceThread.run(ServiceThread.java:63)
INFO | jvm 1 | 2019/02/13 15:07:25 | java.lang.Thread.run(Thread.java:748)
INFO | jvm 1 | 2019/02/13 15:07:25 |
^C

 

 

I don't know why NFA console is refusing...

I don't have firewall on NFA console.

 

Br,

Pedro Lourenço

[Steve Crowl]

Hi Pedro.

It appears to me that the poller service (nfa_poller) may not be running on the harvester: "Timed out while waiting for connection to com.ca.im.ra.poller.Poller because Connection refused to host"

Steve

[Pedro Filipe Lourenco]

Hi,

 

 

On poller-wrapper.log I can't see any error.

 

[root@nfa_harvester Logs]# service nfa_poller status
/home/nfa/.bashrc: line 12: PATH: command not found
/home/nfa/.bashrc: line 13: PATH: command not found
NFA Poller is running: PID:14192, Wrapper:STARTED, Java:STARTED
[root@nfa_harvester Logs]#

 

Thanks,

Pedro Lourenço