DX Infrastructure Manager

Expand all | Collapse all

Problem with data_engine queue when starting maintenance

  • 1.  Problem with data_engine queue when starting maintenance

    Posted 05-31-2018 11:39 PM

    Hi everyone.

     

    I have a problem when the maintenance of data_engine probe are active the subscriber is glued

     

    I don't know why this problem.

    Some suggestion?

     

    Best Regards

    M.E



  • 2.  Re: Problem with data_engine queue when starting maintenance

    Posted 05-31-2018 11:45 PM

    Hi 

    What  time is the data_engine maintenance activities scheduled.Default is 00:40 hrs every day .

    You can check this in the schedule tab in data_engine probe .

    Recommeded to schedule data_engine maintenance for non peak hours if not already done 



  • 3.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-01-2018 10:56 AM

    Franklin, I changed the schedule to 20:00 hours because to the 00:40 was glued too.

     

    I am checking the inserted data from data_engine to db server but the time in ms its very high, that indicated that the insert its very slow.

     

     

    This document indicated https://comm.support.ca.com/kb/my-dataengine-queue-is-backing-up-but-i-have-already-increased-my-dataengine-bulksize-as-well-as-made-my-dataengine-multi-threaded-what-do-i-do-now/kb000037377 that we should checking in DBSERVER---->

     

    1) CPU utilization. Best practice should be less than 6-% normally
    2) Memory utilization: Best practice should be less than 90% normally
    3) Disk queue: Should always be less than 1
    4) also check to make sure your database is in simple recovery mode. ( this is strictly for performance)
    5) check that the database is not set to grow in too small of chunks. Logs and data files size should be at least 500 MBs

     

    And only the point 4 we are not fulfilling


     

    I don't know if this issue is for I/O in storage although the response max is 18 ms to the LUN that support disk of the DB.

    Are times I/O is optimum for DBSERVER?

    What we should check to disk queue length ?

     

    Thanks for your support.

     



  • 4.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-03-2018 09:00 PM

    Hi 

     

    data_engine best practices

    https://comm.support.ca.com/kb/dataengine-best-practices/kb000033592

     

    Do you have high raw data retention in your environment ?
    Whats your sql server version and is it Enterprise version ?
    Are you using partitioning in your environment?

     

    CA UIM (Nimsoft) Database Best Practices for MS SQL Server

    https://comm.support.ca.com/kb/ca-uim-nimsoft-database-best-practices-for-ms-sql-server/kb000033680

     

    You can also use windows resource monitor to monitor storage performance

     

    Example reference
    https://www.techrepublic.com/blog/the-enterprise-cloud/use-resource-monitor-to-monitor-storage-performance/



  • 5.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-05-2018 11:31 AM

    Hi Franklin.

     

    The data_engine best practice it was applied a long time ago.

    I am checking the kb000033680, and we are following the recommendation.

     

    Do you have high raw data retention in your environment ?

    We have "Deleted raw data: 90 days" and "Deleted historic data: 365 days". I think the raw data are okay with this configuration.


    Whats your sql server version and is it Enterprise version ?

    DB Version: Microsoft SQL Server 2014 (SP2-GDR) (KB4057120) - 12.0.5214.6 (X64) Jan 9 2018 15:03:12 Copyright (c) Microsoft Corporation Standard Edition (64-bit) on Windows NT 6.3 <X64> (Build 9600: ) (Hypervisor)
    data_engine schema version: 8.50(0)


    Are you using partitioning in your environment?

    not because the standard version does not have



  • 6.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-04-2018 09:26 AM

    I am not aware of any specific recommendations for disk I/O or queue length.



  • 7.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-05-2018 02:59 PM

    MIller,

     

    A check to see if the data_engine is pruning the old data as it should is a good idea. If it isn't then the tables will grow excessively large. See this tech doc:

    Verifying that data_engine is deleting old raw and historic data properly from the database
    Document ID:  KB000034940
    https://comm.support.ca.com/kb/verifying-that-dataengine-is-deleting-old-raw-and-historic-data-properly-from-the-database/kb000034940

     

    The maintenance window can be increased via:

    raw configure > setup > data_management_timeout > 65534



  • 8.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-05-2018 06:33 PM

    Hi David. This is the output of the query. I understaind that I have a problem   

     

     :-(

     

    RN_QOS_DATA_0001 contains 3006936 samples which are older than 91 days old.
    RN_QOS_DATA_0002 contains 13588 samples which are older than 91 days old.
    RN_QOS_DATA_0003 contains 13588 samples which are older than 91 days old.
    RN_QOS_DATA_0004 contains 13588 samples which are older than 91 days old.
    RN_QOS_DATA_0007 contains 10973 samples which are older than 91 days old.
    RN_QOS_DATA_0008 contains 1358 samples which are older than 91 days old.
    RN_QOS_DATA_0009 contains 2992 samples which are older than 91 days old.
    RN_QOS_DATA_0010 contains 2240 samples which are older than 91 days old.
    RN_QOS_DATA_0011 contains 20384 samples which are older than 91 days old.
    RN_QOS_DATA_0012 contains 10872 samples which are older than 91 days old.
    RN_QOS_DATA_0013 contains 16518 samples which are older than 91 days old.
    RN_QOS_DATA_0014 contains 36680 samples which are older than 91 days old.
    RN_QOS_DATA_0015 contains 911 samples which are older than 91 days old.
    RN_QOS_DATA_0016 contains 10872 samples which are older than 91 days old.
    RN_QOS_DATA_0017 contains 9513 samples which are older than 91 days old.
    RN_QOS_DATA_0018 contains 17989 samples which are older than 91 days old.
    RN_QOS_DATA_0019 contains 8154 samples which are older than 91 days old.
    RN_QOS_DATA_0020 contains 16630 samples which are older than 91 days old.
    RN_QOS_DATA_0021 contains 8154 samples which are older than 91 days old.
    RN_QOS_DATA_0022 contains 8154 samples which are older than 91 days old.
    RN_QOS_DATA_0025 contains 894892 samples which are older than 91 days old.
    RN_QOS_DATA_0026 contains 2707583 samples which are older than 91 days old.
    RN_QOS_DATA_0027 contains 880448 samples which are older than 91 days old.
    RN_QOS_DATA_0029 contains 587924 samples which are older than 91 days old.
    RN_QOS_DATA_0030 contains 590722 samples which are older than 91 days old.
    RN_QOS_DATA_0032 contains 590796 samples which are older than 91 days old.
    RN_QOS_DATA_0034 contains 590770 samples which are older than 91 days old.
    RN_QOS_DATA_0037 contains 588042 samples which are older than 91 days old.
    RN_QOS_DATA_0041 contains 1082816 samples which are older than 91 days old.
    RN_QOS_DATA_0042 contains 849577 samples which are older than 91 days old.
    RN_QOS_DATA_0043 contains 842866 samples which are older than 91 days old.
    RN_QOS_DATA_0044 contains 852029 samples which are older than 91 days old.
    RN_QOS_DATA_0045 contains 20186061 samples which are older than 91 days old.
    RN_QOS_DATA_0046 contains 20186794 samples which are older than 91 days old.
    RN_QOS_DATA_0047 contains 20186099 samples which are older than 91 days old.
    RN_QOS_DATA_0051 contains 22541886 samples which are older than 91 days old.
    RN_QOS_DATA_0052 contains 20186755 samples which are older than 91 days old.
    RN_QOS_DATA_0053 contains 20212399 samples which are older than 91 days old.
    RN_QOS_DATA_0055 contains 20211711 samples which are older than 91 days old.
    RN_QOS_DATA_0056 contains 22530761 samples which are older than 91 days old.
    RN_QOS_DATA_0057 contains 20211697 samples which are older than 91 days old.
    RN_QOS_DATA_0058 contains 20212286 samples which are older than 91 days old.
    RN_QOS_DATA_0059 contains 22530356 samples which are older than 91 days old.
    RN_QOS_DATA_0060 contains 22528037 samples which are older than 91 days old.
    RN_QOS_DATA_0061 contains 20188232 samples which are older than 91 days old.
    RN_QOS_DATA_0062 contains 20181613 samples which are older than 91 days old.
    RN_QOS_DATA_0064 contains 14402 samples which are older than 91 days old.
    RN_QOS_DATA_0065 contains 5847721 samples which are older than 91 days old.
    RN_QOS_DATA_0066 contains 1719698 samples which are older than 91 days old.
    RN_QOS_DATA_0067 contains 684531 samples which are older than 91 days old.
    RN_QOS_DATA_0068 contains 2906361 samples which are older than 91 days old.
    RN_QOS_DATA_0069 contains 2171297 samples which are older than 91 days old.
    RN_QOS_DATA_0070 contains 1734074 samples which are older than 91 days old.
    RN_QOS_DATA_0071 contains 7500906 samples which are older than 91 days old.
    RN_QOS_DATA_0072 contains 7789208 samples which are older than 91 days old.
    RN_QOS_DATA_0073 contains 18563603 samples which are older than 91 days old.
    RN_QOS_DATA_0074 contains 22586933 samples which are older than 91 days old.
    RN_QOS_DATA_0075 contains 22588339 samples which are older than 91 days old.
    RN_QOS_DATA_0076 contains 622676 samples which are older than 91 days old.
    RN_QOS_DATA_0077 contains 622715 samples which are older than 91 days old.
    RN_QOS_DATA_0078 contains 622764 samples which are older than 91 days old.
    RN_QOS_DATA_0079 contains 588835 samples which are older than 91 days old.
    RN_QOS_DATA_0080 contains 85950 samples which are older than 91 days old.
    RN_QOS_DATA_0081 contains 85951 samples which are older than 91 days old.
    RN_QOS_DATA_0082 contains 24637 samples which are older than 91 days old.
    RN_QOS_DATA_0083 contains 24637 samples which are older than 91 days old.
    RN_QOS_DATA_0086 contains 9405 samples which are older than 91 days old.
    RN_QOS_DATA_0087 contains 8481 samples which are older than 91 days old.
    RN_QOS_DATA_0088 contains 8481 samples which are older than 91 days old.
    RN_QOS_DATA_0089 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0090 contains 1343 samples which are older than 91 days old.
    RN_QOS_DATA_0091 contains 1343 samples which are older than 91 days old.
    RN_QOS_DATA_0092 contains 1343 samples which are older than 91 days old.
    RN_QOS_DATA_0093 contains 10744 samples which are older than 91 days old.
    RN_QOS_DATA_0094 contains 1343 samples which are older than 91 days old.
    RN_QOS_DATA_0095 contains 10744 samples which are older than 91 days old.
    RN_QOS_DATA_0096 contains 18802 samples which are older than 91 days old.
    RN_QOS_DATA_0097 contains 18802 samples which are older than 91 days old.
    RN_QOS_DATA_0098 contains 18802 samples which are older than 91 days old.
    RN_QOS_DATA_0099 contains 1343 samples which are older than 91 days old.
    RN_QOS_DATA_0100 contains 40317 samples which are older than 91 days old.
    RN_QOS_DATA_0101 contains 48924 samples which are older than 91 days old.
    RN_QOS_DATA_0102 contains 290431 samples which are older than 91 days old.
    RN_QOS_DATA_0103 contains 423158 samples which are older than 91 days old.
    RN_QOS_DATA_0104 contains 423063 samples which are older than 91 days old.
    RN_QOS_DATA_0105 contains 423075 samples which are older than 91 days old.
    RN_QOS_DATA_0106 contains 916020 samples which are older than 91 days old.
    RN_QOS_DATA_0107 contains 423058 samples which are older than 91 days old.
    RN_QOS_DATA_0108 contains 225859 samples which are older than 91 days old.
    RN_QOS_DATA_0110 contains 1006670 samples which are older than 91 days old.
    RN_QOS_DATA_0111 contains 714106 samples which are older than 91 days old.
    RN_QOS_DATA_0112 contains 139141 samples which are older than 91 days old.
    RN_QOS_DATA_0113 contains 144849 samples which are older than 91 days old.
    RN_QOS_DATA_0114 contains 139167 samples which are older than 91 days old.
    RN_QOS_DATA_0115 contains 143896 samples which are older than 91 days old.
    RN_QOS_DATA_0116 contains 142350 samples which are older than 91 days old.
    RN_QOS_DATA_0117 contains 23670 samples which are older than 91 days old.
    RN_QOS_DATA_0118 contains 23715 samples which are older than 91 days old.
    RN_QOS_DATA_0119 contains 24393 samples which are older than 91 days old.
    RN_QOS_DATA_0120 contains 23715 samples which are older than 91 days old.
    RN_QOS_DATA_0121 contains 23649 samples which are older than 91 days old.
    RN_QOS_DATA_0122 contains 24110 samples which are older than 91 days old.
    RN_QOS_DATA_0123 contains 23758 samples which are older than 91 days old.
    RN_QOS_DATA_0124 contains 23630 samples which are older than 91 days old.
    RN_QOS_DATA_0125 contains 24253 samples which are older than 91 days old.
    RN_QOS_DATA_0126 contains 23728 samples which are older than 91 days old.
    RN_QOS_DATA_0127 contains 23721 samples which are older than 91 days old.
    RN_QOS_DATA_0128 contains 23685 samples which are older than 91 days old.
    RN_QOS_DATA_0129 contains 70734 samples which are older than 91 days old.
    RN_QOS_DATA_0130 contains 5440 samples which are older than 91 days old.
    RN_QOS_DATA_0132 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0133 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0134 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0135 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0136 contains 87072 samples which are older than 91 days old.
    RN_QOS_DATA_0137 contains 87072 samples which are older than 91 days old.
    RN_QOS_DATA_0138 contains 87072 samples which are older than 91 days old.
    RN_QOS_DATA_0140 contains 87072 samples which are older than 91 days old.
    RN_QOS_DATA_0145 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0146 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0147 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0148 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0149 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0150 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0151 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0152 contains 2071151 samples which are older than 91 days old.
    RN_QOS_DATA_0153 contains 87040 samples which are older than 91 days old.
    RN_QOS_DATA_0162 contains 97005 samples which are older than 91 days old.
    RN_QOS_DATA_0163 contains 97007 samples which are older than 91 days old.
    RN_QOS_DATA_0164 contains 97007 samples which are older than 91 days old.
    RN_QOS_DATA_0165 contains 24462 samples which are older than 91 days old.
    RN_QOS_DATA_0166 contains 9517 samples which are older than 91 days old.
    RN_QOS_DATA_0167 contains 9517 samples which are older than 91 days old.
    RN_QOS_DATA_0168 contains 19560313 samples which are older than 91 days old.
    RN_QOS_DATA_0169 contains 60476 samples which are older than 91 days old.
    RN_QOS_DATA_0170 contains 46454 samples which are older than 91 days old.
    RN_QOS_DATA_0171 contains 46454 samples which are older than 91 days old.
    RN_QOS_DATA_0172 contains 320449 samples which are older than 91 days old.
    RN_QOS_DATA_0176 contains 320446 samples which are older than 91 days old.
    RN_QOS_DATA_0177 contains 320443 samples which are older than 91 days old.
    RN_QOS_DATA_0181 contains 10750 samples which are older than 91 days old.
    RN_QOS_DATA_0182 contains 10746 samples which are older than 91 days old.
    RN_QOS_DATA_0183 contains 10768 samples which are older than 91 days old.
    RN_QOS_DATA_0204 contains 6800 samples which are older than 91 days old.
    RN_QOS_DATA_0205 contains 5440 samples which are older than 91 days old.
    RN_QOS_DATA_0206 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0207 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0208 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0209 contains 5440 samples which are older than 91 days old.
    RN_QOS_DATA_0210 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0211 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0212 contains 2720 samples which are older than 91 days old.
    RN_QOS_DATA_0213 contains 5440 samples which are older than 91 days old.
    RN_QOS_DATA_0214 contains 1402 samples which are older than 91 days old.
    RN_QOS_DATA_0215 contains 1360 samples which are older than 91 days old.
    RN_QOS_DATA_0216 contains 453 samples which are older than 91 days old.
    RN_QOS_DATA_0217 contains 2724 samples which are older than 91 days old.
    RN_QOS_DATA_0218 contains 454 samples which are older than 91 days old.
    RN_QOS_DATA_0219 contains 454 samples which are older than 91 days old.
    RN_QOS_DATA_0220 contains 82 samples which are older than 91 days old.
    RN_QOS_DATA_0221 contains 454 samples which are older than 91 days old.
    RN_QOS_DATA_0222 contains 454 samples which are older than 91 days old.
    RN_QOS_DATA_0223 contains 6356 samples which are older than 91 days old.
    RN_QOS_DATA_0224 contains 3632 samples which are older than 91 days old.
    RN_QOS_DATA_0225 contains 6356 samples which are older than 91 days old.
    RN_QOS_DATA_0226 contains 3632 samples which are older than 91 days old.
    RN_QOS_DATA_0236 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0237 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0238 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0239 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0240 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0241 contains 2718 samples which are older than 91 days old.
    RN_QOS_DATA_0242 contains 18919 samples which are older than 91 days old.
    RN_QOS_DATA_0243 contains 7792 samples which are older than 91 days old.
    RN_QOS_DATA_0244 contains 7792 samples which are older than 91 days old.
    RN_QOS_DATA_0245 contains 7792 samples which are older than 91 days old.
    RN_QOS_DATA_0246 contains 7792 samples which are older than 91 days old.
    RN_QOS_DATA_0247 contains 46109 samples which are older than 91 days old.
    RN_QOS_DATA_0248 contains 46109 samples which are older than 91 days old.
    RN_QOS_DATA_0249 contains 46108 samples which are older than 91 days old.
    RN_QOS_DATA_0250 contains 46108 samples which are older than 91 days old.
    RN_QOS_DATA_0251 contains 46108 samples which are older than 91 days old.
    RN_QOS_DATA_0252 contains 46108 samples which are older than 91 days old.
    RN_QOS_DATA_0253 contains 46109 samples which are older than 91 days old.
    RN_QOS_DATA_0257 contains 6800 samples which are older than 91 days old.
    RN_QOS_DATA_0258 contains 1162 samples which are older than 91 days old.
    RN_QOS_DATA_0259 contains 1162 samples which are older than 91 days old.
    HN_QOS_DATA_0001 contains 8584 samples which are older than 366 days old.
    HN_QOS_DATA_0002 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0007 contains 673 samples which are older than 366 days old.
    HN_QOS_DATA_0008 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0009 contains 263 samples which are older than 366 days old.
    HN_QOS_DATA_0011 contains 904 samples which are older than 366 days old.
    HN_QOS_DATA_0012 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0013 contains 3277 samples which are older than 366 days old.
    HN_QOS_DATA_0014 contains 1017 samples which are older than 366 days old.
    HN_QOS_DATA_0015 contains 228 samples which are older than 366 days old.
    HN_QOS_DATA_0016 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0017 contains 2825 samples which are older than 366 days old.
    HN_QOS_DATA_0018 contains 904 samples which are older than 366 days old.
    HN_QOS_DATA_0019 contains 791 samples which are older than 366 days old.
    HN_QOS_DATA_0020 contains 904 samples which are older than 366 days old.
    HN_QOS_DATA_0021 contains 791 samples which are older than 366 days old.
    HN_QOS_DATA_0022 contains 678 samples which are older than 366 days old.
    HN_QOS_DATA_0025 contains 4 samples which are older than 366 days old.
    HN_QOS_DATA_0026 contains 44820 samples which are older than 366 days old.
    HN_QOS_DATA_0027 contains 20046 samples which are older than 366 days old.
    HN_QOS_DATA_0029 contains 12327 samples which are older than 366 days old.
    HN_QOS_DATA_0030 contains 12327 samples which are older than 366 days old.
    HN_QOS_DATA_0032 contains 12327 samples which are older than 366 days old.
    HN_QOS_DATA_0034 contains 12327 samples which are older than 366 days old.
    HN_QOS_DATA_0037 contains 12327 samples which are older than 366 days old.
    HN_QOS_DATA_0041 contains 22073 samples which are older than 366 days old.
    HN_QOS_DATA_0042 contains 12230 samples which are older than 366 days old.
    HN_QOS_DATA_0043 contains 12230 samples which are older than 366 days old.
    HN_QOS_DATA_0044 contains 19458 samples which are older than 366 days old.
    HN_QOS_DATA_0045 contains 244255 samples which are older than 366 days old.
    HN_QOS_DATA_0046 contains 244255 samples which are older than 366 days old.
    HN_QOS_DATA_0047 contains 244255 samples which are older than 366 days old.
    HN_QOS_DATA_0051 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0052 contains 244255 samples which are older than 366 days old.
    HN_QOS_DATA_0053 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0055 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0056 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0057 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0058 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0059 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0060 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0061 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0062 contains 244256 samples which are older than 366 days old.
    HN_QOS_DATA_0065 contains 242685 samples which are older than 366 days old.
    HN_QOS_DATA_0066 contains 37006 samples which are older than 366 days old.
    HN_QOS_DATA_0067 contains 4552 samples which are older than 366 days old.
    HN_QOS_DATA_0068 contains 52159 samples which are older than 366 days old.
    HN_QOS_DATA_0069 contains 32929 samples which are older than 366 days old.
    HN_QOS_DATA_0070 contains 8376 samples which are older than 366 days old.
    HN_QOS_DATA_0071 contains 90369 samples which are older than 366 days old.
    HN_QOS_DATA_0072 contains 90369 samples which are older than 366 days old.
    HN_QOS_DATA_0073 contains 268508 samples which are older than 366 days old.
    HN_QOS_DATA_0074 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0075 contains 261794 samples which are older than 366 days old.
    HN_QOS_DATA_0076 contains 1356 samples which are older than 366 days old.
    HN_QOS_DATA_0077 contains 1356 samples which are older than 366 days old.
    HN_QOS_DATA_0078 contains 1356 samples which are older than 366 days old.
    HN_QOS_DATA_0079 contains 1243 samples which are older than 366 days old.
    HN_QOS_DATA_0080 contains 114 samples which are older than 366 days old.
    HN_QOS_DATA_0081 contains 114 samples which are older than 366 days old.
    HN_QOS_DATA_0082 contains 30687 samples which are older than 366 days old.
    HN_QOS_DATA_0083 contains 30687 samples which are older than 366 days old.
    HN_QOS_DATA_0087 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0088 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0089 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0090 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0091 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0092 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0093 contains 904 samples which are older than 366 days old.
    HN_QOS_DATA_0094 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0095 contains 904 samples which are older than 366 days old.
    HN_QOS_DATA_0096 contains 1582 samples which are older than 366 days old.
    HN_QOS_DATA_0097 contains 1582 samples which are older than 366 days old.
    HN_QOS_DATA_0098 contains 1582 samples which are older than 366 days old.
    HN_QOS_DATA_0099 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0100 contains 9081 samples which are older than 366 days old.
    HN_QOS_DATA_0101 contains 11074 samples which are older than 366 days old.
    HN_QOS_DATA_0102 contains 186 samples which are older than 366 days old.
    HN_QOS_DATA_0103 contains 14791 samples which are older than 366 days old.
    HN_QOS_DATA_0104 contains 14787 samples which are older than 366 days old.
    HN_QOS_DATA_0105 contains 14788 samples which are older than 366 days old.
    HN_QOS_DATA_0106 contains 14791 samples which are older than 366 days old.
    HN_QOS_DATA_0107 contains 14787 samples which are older than 366 days old.
    HN_QOS_DATA_0108 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0110 contains 12061 samples which are older than 366 days old.
    HN_QOS_DATA_0111 contains 40663 samples which are older than 366 days old.
    HN_QOS_DATA_0112 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0113 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0114 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0115 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0116 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0117 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0118 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0119 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0120 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0121 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0122 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0123 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0124 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0125 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0126 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0127 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0128 contains 676 samples which are older than 366 days old.
    HN_QOS_DATA_0129 contains 3164 samples which are older than 366 days old.
    HN_QOS_DATA_0130 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0131 contains 113 samples which are older than 366 days old.
    HN_QOS_DATA_0132 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0133 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0134 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0135 contains 226 samples which are older than 366 days old.
    HN_QOS_DATA_0136 contains 16452 samples which are older than 366 days old.
    HN_QOS_DATA_0137 contains 16452 samples which are older than 366 days old.
    HN_QOS_DATA_0138 contains 16452 samples which are older than 366 days old.
    HN_QOS_DATA_0139 contains 3649 samples which are older than 366 days old.
    HN_QOS_DATA_0140 contains 16452 samples which are older than 366 days old.
    HN_QOS_DATA_0141 contains 3649 samples which are older than 366 days old.
    HN_QOS_DATA_0142 contains 3649 samples which are older than 366 days old.
    HN_QOS_DATA_0143 contains 3649 samples which are older than 366 days old.
    HN_QOS_DATA_0144 contains 3649 samples which are older than 366 days old.
    HN_QOS_DATA_0145 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0146 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0147 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0148 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0149 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0150 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0151 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0152 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0153 contains 53844 samples which are older than 366 days old.
    HN_QOS_DATA_0154 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0155 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0156 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0157 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0158 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0159 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0160 contains 839 samples which are older than 366 days old.
    HN_QOS_DATA_0161 contains 839 samples which are older than 366 days old.
    No unmaintained data found in DN tables



  • 9.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-05-2018 07:52 PM

    Miller ,

    Since you are running standard edition of database so this does not also support online reindexing 

     

    You can explore below KB to delete older data

    How to trouble shoot data retention and Database g - CA Knowledge 



  • 10.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-08-2018 09:40 AM

    Hi Miller,

     

    I think disk queue could be the problem here. Please go through the below article as it contains valuable information on how to troubleshoot disk queue length issues. It also covers the fact that the average disk queue length counter doesn't always represent a problem, so you might want to rule that out first.

     

    High Avg Disk Queue Length and finding the Cause | IT Hacks 

     

    If you can confirm that the average disk queue length is high (in my opinion anything above 1 should be investigated) please check your disks/storage system to see where the bottleneck might be.

     

    Thanks,

    Alex



  • 11.  Re: Problem with data_engine queue when starting maintenance

    Posted 06-12-2018 12:23 PM

    Hi all.

     

    I resolved my problem of queue lentgh following:

     

    1- We change the backup schedule DB, of full daily to (Full Sunday and incremental Monday-Saturday)

    2- We changed the "Thread Count" to 24, from this document https://docops.ca.com/ca-unified-infrastructure-management-probes/ga/en/alphabetical-probe-articles/data_engine/data_engine-best-practices#data_engineBestPractices-ThreadCount

     

    3- We started the maintenance of data_engine and we don't see queue data_engine again.



  • 12.  Re: Problem with data_engine queue when starting maintenance

     
    Posted 06-13-2018 05:59 AM

    Hi Miller,

     

    Regards the Thread_count_insert we already had many discussions and test, the best number for you to set here is the number of the CPU you have at yours primary hub, open the task manager on the primary hub and check how many CPUś you have, if you have 8 you should set to 8, if you have 12 you should set to 12, do not set to 24 if you do not have 24 CPUś on the primary hub.

    This way the configuration will match the real threads that yous data engine can open.

     

    I hope this can help

     

    Rubens.