CU8 QoS Retention Issue

[Sam Green]

Hey everyone,

I upgraded our preprod to CU8 yesterday, after the update I looked at a PRD report.  The QoS only went back to 00:55 that morning (our data_engine retention is 30 days).

Looking again this morning, our earliest QoS entry is 00:43 this morning.  Checked the DB and it confirms that our QoS data only goes back to 00:43 this morning.

Has anyone else experienced similar? 

[Vaclav Tomec]

Not in my test env.

[Joakim E]

We don't see that issue. Just upgraded our test env to CU8 but still got all metrics according to the retention plan.

[Sam Green]

Thanks Joakim,

I ran some tests by setting data_engine to run during the day so I could watch it.  Sure enough, it loses all QoS prior to the schedule I set.

[Jason Allen]

In the past I've seen similar behavior when there was an issue with the partition schema and tables were not propertly partitioned by date range.

The query in this KB should help you determine if this is the case - https://knowledge.broadcom.com/external/article/250815/how-to-determine-if-dataengine-partition.html

[Sam Green]

Thanks Jason,

When I query the DB directly I can see the oldest entry is 15:10 (yesterday, when I last rescheduled the data_engine).

Here are the results of the query suggested in the KB.

[Dhanunjaya Rao Lakhinana]

Hi Sam,

Could you please stop data maintenance by setting data_management_active=no in the Data Engine configuration file, and then deactivate and activate the probe.  This will disable the data maintenance process. Also, please raise a support case so we can investigate the issue further.

[Sam Green]

Thank you, I have set data_management_active to no, this has stopped the QoS deletion,

I've raised a support case with Arrow, support case 40683.

[Sam Green]

Here are the closure notes from the support case:

After a call with Broadcom SME and we checked your case. The default value for data_management_partition key should be set to no for Oracle DBs, but yes for Microsoft SQL DBs.

That's why it was set by default at your end to yes without changing it.

On CU4: If partitioning was set to yes but the database wasn't actually partitioned, older versions of the data_engine often silently failed the partition-drop command and fell back to standard row-by-row deletion. This allowed the system to appear "normal" even with an incorrect configuration.
On CU8: The maintenance logic is more advanced and strict, the probe specifically looks for partition boundaries to determine what data to keep. If the database is not partitioned, the script may fail to find these boundaries, interpret all existing data as expired, and purge it during the daily maintenance cycle.

Please keep the data_management_partition set to no on your CU8 environment to stop the 24-hour deletion cycle.

You may check the database partitioning status with your DBA team and align it with below Broadcom articles.

https://knowledge.broadcom.com/external/article/250815
https://knowledge.broadcom.com/external/article/247518/dataengine-best-practices-settings-perf.html