Netqos RA dont show data older than 2 hours.

[d.pereyra]

Hello  everybody ! - I have the Netqos RA version 9.0 (build 161) Update 1, with NPC 6.1.194.

 

I have this problem:  when I run a interface report, I see data only when the timeframe its only last 2 hours exacly, if I add a minute, no graph is available. Only in periods of 2 hours.

 

Also I see datta when the timeframe is weekly or older.

 

Examples: Weekly graph

 

The gap here if because the services stopped for a backup job, and services dont start automaticly

 

I start services manually at 12:00, but I see data only selecting 2 hours period

Last 2 hours and an extra minute.

But if I do exacly 2 hours, the graph works.

 

Why could be happening ? any error in DB ?

 

Thanks in advance

Diego MP

[Christopher Walsh]

First, 9.0.161 is a really old version and has been Out of Support for about 3 years now.  Upgrading would resolve this.

 

In the 9.0.161 architecture, the 15 minute resolution data, which is data longer than 2 hours in timeframe, is stored on the DSA.

 

The DSA gets its data files from a data share on the RA Console server.  If the user's credentials used to connect to that share's password expires or the account gets locked out, the DSA will not be able to pull files from the console server and you will not see 15 min resolution data.

 

The credentials for this data share are entered into the Admin->DSA->DSA Credentials page in RA.  You should update the user credentials to be a user that has full access to the share and folder on the D:\netqos\reporter\datashare\ directory on the console.

 

NFA 9.1.x and newer no longer uses a datashare, so you would not run into this problem in later versions.

 

Below is from an old RA 9.0 troubleshooting guide:

Confirm that the DSA credentials are configured properly. This is found in the RA web interface under ‘Administration’ -> ‘DSA’ -> ‘Credentials.’ The domain field should be the local hostname of the RA Console server (if a local account is in use) or the domain name (if a domain account is in use). The username and password specified must be an administrative account that has access to both the RA Console server and the DSA server. After making this change, check the size of the D:\netqos\reporter\datashare\data directory as described in step 4). If this does not resolve the issue, reset the Windows password for the specified user and then re-enter the password in the DSA ‘Credentials.’ If the account is local, this can be done by right-clicking ‘My Computer’, selecting ‘Manage,’ expanding ‘System Tools’ -> ‘Local Users and Groups,’ clicking ‘Users,’ right-clicking the user, and selecting ‘Set Password…’.

[d.pereyra]

Is an older version indeed, but still working. Im able to see the 15 min resolution data, Im not able to see resolution bigger than 2 hours in last 2 days. Could be because there is a gap, big gap in the middle ?

 

Diego

 

[Christopher Walsh]

Diego, RA Stores data for less then 2 hours on the Harvesters.  

This data is stored at 1 minute resolution, so each data point is 1 minute.

 

RA stores data greater then 2 hours on the DSA's, this data is 15 minute resolution, meaning each data point is 15 minutes.

 

When I say 15 minute resolution, that is different then 15 minute timeframe.  I understand 15 min timeframe works as the at less then 2 hours.  But all data less then a 2 hour timeframe will be in 1 minute resolution. 

 

So if data greater then 2 hours is not showing, it means there is a problem with the DSA data which is why I suggested verifying the DSA credentials above.

 

You can also check the \Reporter\logs\Pump*.log for any errors connecting to the datashare.

[d.pereyra]

Thanks, now I got it, and excuse me if i miss understood.

The credentials are OK., the test response ok.

 

The pump logs show this

17:19:27 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
17:19:43 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
17:19:58 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
17:20:13 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
17:20:15 4 - FilePumpManager: Fetching of data files from Harvesters will resume when the number of RPR files in input/staging directories (105) falls below ReaperFileThreshold (100).

 

Diego MP

[Christopher Walsh]

Are there over 100 files in the \Netwos\netflow\datafiles\NFMInput\ directory, if so move them out of that directory and see if the error in the pump log stops after a period of time and see if files begin moving in and out of the NFMInput directory and the \Netqos\Reporter\data\input and \Netqos\Reporter\data\staging directories.

[Christopher Walsh]

Also the "Test" button doesn't actually test the credentials it tests the connection to the dsa's database.

 

Best way to test the credentials, to ensure they have access to the share is to try to access the share remotely from a remote file explorer by using \\servername\datashare and use the DSA Credentials.

[d.pereyra]

Thanks Christopher, I move out files from NFMInput directory and start creating new files, same with ReaperWork and HarvesterWork directories, i move out olds files.

 

I reenter DSA credentials, test ok, restart services. and

NetworkSharePumpLog2016-08-26.log

12:01:40 -1 - NetworkSharePump: Pump started.
12:01:41 -1 - NetworkSharePump: Starting...
12:01:41 -1 - NetworkSharePump: Reading in local files
12:01:54 -1 - NetworkSharePump: Starting RemoteFileWatcher
12:01:54 3 - RemoteFileWatcher: Main thread started
12:01:54 -1 - NetworkSharePump: Starting 2 mover threads
12:01:54 -1 - NetworkSharePump: Started

12:04:01 3 - FileMover #1: Error copying from \\10.30.156.212\datashare\data\127.0.0.1\Host\1472051700-3960_h.csv to D:\NETQOS\reporter\datashare\staging\Host\3960_h.csv: Access to the path 'D:\NETQOS\reporter\datashare\staging\Host\3960_h.csv' is denied.
12:04:01 1 - FileMover #1 - :
Access to the path 'D:\NETQOS\reporter\datashare\staging\Host\3960_h.csv' is denied.

System.UnauthorizedAccessException: Access to the path 'D:\NETQOS\reporter\datashare\staging\Host\3960_h.csv' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
at System.IO.File.OpenFile(String path, FileAccess access, SafeFileHandle& handle)
at System.IO.File.SetLastWriteTimeUtc(String path, DateTime lastWriteTimeUtc)
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.MoveFiles() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 178
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.Work() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 61

But this error, doesn't repeat in last hour.

 

The PumpLog2016-08-26.log

12:51:51 4 - FilePumpManager: Fetching of data files from Harvesters will resume when the number of RPR files in input/staging directories (112) falls below ReaperFileThreshold (100).
12:52:01 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
12:52:16 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.

 

 

Diego MP

[Christopher Walsh]

Seems like there is still some type of permissions issue accessing that data share folder with the user specified in the DSA Credentials.

 

You have to make sure both the Permissions for the user on the "Sharing" tab and Folder Permissions on the "Security" tab allow access to this directory and share.

 

 

 

 

 

Also make sure you can login to the server with DSA user.

Make sure the username is set like domain\username or localhost\username if it is just a local user.

[d.pereyra]

Thanks once again !

The credential error dont show up anymore.

 

In system status there is this message "DSA - Last Load Timestamp 1.65 Hours before check."

 

The pumpLog still showing these kind of messages:

13:47:16 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.

 

Any log extra to checkt ?

 

Thanks

Diego MP

[Christopher Walsh]

The Pump and NetworkShare pump are the two main logs.

 

If the credentials issue is resolve, it just may need some time to catch up.  Hopefully you should start seeing that Last Load Timestamp reduce periodically which would mean files are now being processed.

 

You can watch the files in the input\staging folders to see if they start decreasing, or move them out temporarily to see if files start moving in and out, once there are less than 100 files in the directory.

[d.pereyra]

We wil see what happend in the weekend.

As input/staging folders you mean "D:\NETQOS\reporter\datafiles\input\staging" (from DSA/NPC server) ? there are .rpr file, not biggers.

The Last Load Timestamp is now 2.65 Hour, is bigger jeje. We will the the monday, thanks for all.

Diego

[d.pereyra]

Chistopher, good day, the samples biggers than 2 hours still not available.

In "D:\NETQOS\reporter\datafiles\input\staging" there are the same amount of files from last week.

 

Should I delete them and restart services ?

 

The pumplog

10:17:15 4 - FilePumpManager: Fetching of data files from Harvesters will resume when the number of RPR files in input/staging directories (126) falls below ReaperFileThreshold (120).
10:17:20 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
10:17:36 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.
10:17:51 4 - DataConverter: DataConverter pausing because 127.0.0.1 appears to be more than an hour behind.

 

networksharepumplog: At 3AM pumpservices restart and repeat two access denies to these files

 

3:03:30 1 - FileMover #2 - :
Access to the path 'D:\NETQOS\reporter\datashare\staging\Prot\1554_p.csv' is denied.
System.UnauthorizedAccessException: Access to the path 'D:\NETQOS\reporter\datashare\staging\Prot\1554_p.csv' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
at System.IO.File.OpenFile(String path, FileAccess access, SafeFileHandle& handle)
at System.IO.File.SetLastWriteTimeUtc(String path, DateTime lastWriteTimeUtc)
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.MoveFiles() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 178
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.Work() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 61
3:03:30 1 - FileMover #1 - :
Access to the path 'D:\NETQOS\reporter\datashare\staging\Prot2\1554_p2.csv' is denied.
System.UnauthorizedAccessException: Access to the path 'D:\NETQOS\reporter\datashare\staging\Prot2\1554_p2.csv' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
at System.IO.File.OpenFile(String path, FileAccess access, SafeFileHandle& handle)
at System.IO.File.SetLastWriteTimeUtc(String path, DateTime lastWriteTimeUtc)
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.MoveFiles() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 178
at NetQoS.ReporterAnalyzer.NetworkSharePump.NetworkSharePump.FileMover.Work() in d:\Builds\RA\Current\REPORTER\NetQoS.ReporterAnalyzer.NetworkSharePump\NetworkSharePump.cs:line 61

 

If have any procedure to run, or there is any path to update, without do a release-update.

 

Thanks

Diego MP

 

 

 

 

 

[Christopher Walsh]

There are still access denied messages indicating that the user specified in the DSA Credentials does not have access to that directory.  Try change the user to a local administrator account and give that account both full control for share and security permissions on the \reporter\datashare folder and make sure the permissions are the same on sub folders.

[d.pereyra]

Christopher, issue resolved.

 

• I stop all services in both servers, Harvester and RA/NPC,
• Delete all files, olders and news from Imput folders from both servers
• Re-enter DSA credential changing the hostname for 127.0.0.1 (is in the same server)
• Update permissions for netqos user in all Imput folders recursively.
• log-in as netqos and start services with netqos user, before I used Administrator account. (But I dont know if was wrong do this.)

 

And in pumplog all start to work, convert rpr files, moving files betweenservers..

 

This post teach me a lot of how netqos works. Thanks !

Diego MP

[Christopher Walsh]

Glad to hear its working now.

Good to hear you learned a lot as well.  

 

As I mentioned earlier however the architecture changes going forward, we no longer use a file share to move files to the DSA, instead its handled by Web Services, so there is no longer the issue with user credentials.