Dataretention for dx nfa

[Miller Echagarreta]

Hi everyone.

I understand that the data retention function with when the percentage of free disk space crosses the percentage threshold specified by the freeSpacePercentage attribute in the parameter_descriptions  table in the harvester database.


I understand that the prunePriorityLatestData  property determines which data to prune when the freeSpacePercentage  threshold is breached, but the documentation indicate that ny default that configuration is true, this means that it does not erase old data 

I understand that the prunePriorityLatestData  property determines which data to prune when the freeSpacePercentage threshold is breached, the documentation indicate that by default the configuration is true...


This means that it does not erase old data? for erase data need chenge this configuration to false ?



Any recommendation for manage the data retention in my NFA standalone system with 148 devices and 200 gb of storage? The client does not want to give me more space and I am afraid that the system may fail due to disk space.

[Jose Vicente Espinosa]

Hi Miller,

let me try to explain

Data Retention system runs 2 checks, one every 24 hours (or periodicPruneFrequency in settings table of data_retention database) and another every 1 minute (or thresholdCheckFrequency in settings table of data_retention database)

In the first one, it runs a Max level prune that deletes data to keep 24 hours of Raw data, 30 days of 1-min data, and 1 year of 15-min data. These temporal values can be changed.

In the second check, it firstly checks if free space threshold has been breached:

• If not, it stops here
• if yes, it runs a Max level prune that deletes data to keep 24 hours of Raw data, 30 days of 1-min data, and 1 year of 15-min data (values defined in maxThreshold column in datastores table of data_retention database). It checks if there is enoughFreeSpace:
  • if yes, it stops here
  • if not, it runs a Min level prune that deletes data to keep 1 hour of Raw data, 2 hours of 1-min data, and (more or less) 2 months of 15-min data (values defined in minThreshold column in datastores table of data_retention database). It checks if there is enoughFreeSpace again
    • if yes, it stops here
    • if not, prunePriorityLatestData defines the next step:
      • prunePriorityLatestData = true, it stops here to prioritize the existing historical data
      • prunePriorityLatestData = false, it deletes all data to prioritize the new incoming data

The procedure is a bit different in versions previous to 21.2.10


I hope this helps
Regards

[Miller Echagarreta]

Hi Jose Vicente, gusto en saludarte.

That means that with this procedure a standalone NFA should not run out of disk space since the data retention intelligence would not allow it.