DX Application Performance Management

 View Only
  • 1.  Computing fork value for table appmap_states

    Posted May 30, 2016 10:05 AM

    I am upgrading my oldest Postgres DB from 10.1 to 10.2 (we recently converted from using Oracle as an APM DB Host for all our cluster but this one is the oldest one having been using Postgres for over a year).

     

    I am now in my 13th hour of upgrading the DB and in the ...\install\schematool.log file I see a slow progression of the upgrade:

     

    5/29/16 08:20:44.892 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_edges

    5/29/16 08:22:56.850 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_vertices

    5/29/16 08:24:58.578 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160307

    5/29/16 08:56:37.214 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160308

    5/29/16 09:40:23.884 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160309

    5/29/16 10:16:54.826 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160310

    5/29/16 10:52:45.266 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160311

    5/29/16 11:24:15.269 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160312

    5/29/16 11:42:25.426 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160313

    5/29/16 11:53:55.040 PM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160314

    5/30/16 12:27:01.394 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160315

    5/30/16 01:09:05.790 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160316

    5/30/16 01:52:59.478 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160317

    5/30/16 02:35:50.897 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160318

    5/30/16 03:23:45.325 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160319

    5/30/16 03:46:13.947 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160320

    5/30/16 03:57:19.265 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160321

    5/30/16 04:22:28.802 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160322

    5/30/16 04:51:12.558 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160323

    5/30/16 05:17:30.824 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160324

    5/30/16 05:41:44.546 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160325

    5/30/16 06:06:08.347 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160326

    5/30/16 06:20:42.597 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160327

    5/30/16 06:28:20.007 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160328

    5/30/16 06:55:13.886 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160329

    5/30/16 07:24:40.443 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160330

    5/30/16 07:52:32.860 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160331

    5/30/16 08:17:08.430 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160401

    5/30/16 09:03:27.524 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160402

    5/30/16 09:19:56.430 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160403

    5/30/16 09:32:12.630 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160404

    5/30/16 09:52:56.944 AM EDT [INFO] [main] [root] [Upgrade10_1_0_0] - Computing fork value for table appmap_states_20160405

     

    I did not do a separate DB Upgrade since I was upgrading the EM that runs on this machine, I chose to upgrade the Postgres.

     

    This is a production cluster so I currently have all the MoM and collectors down so as to not muck with the APM DB...however this is hurting. I am on the phone right now waiting for CA Support....any ideas?



  • 2.  Re: Computing fork value for table appmap_states

    Broadcom Employee
    Posted May 30, 2016 10:47 AM

    hello Steve,

     

    I got some similiar behavior, if you do not care for CEM historical data, you can export your configs and recreate the database, will be much more faster.

    About this currenlty behavior, did you set to upgrade the database and it is taking too much to complete the action? or it is trowed some error?

     

    Gustavo.



  • 3.  Re: Computing fork value for table appmap_states

    Posted May 30, 2016 11:04 AM

    Thanx, I was trying to avoid that since I have siteminder and SSL set up on my TIM and would have to go back and find that...

    however, it looks like it may be the only option.

     

    I have an open case  00420093:
    Computing fork value for table appmap_states

     

    So I will see my options in terminating this and what I need to do to get it back working....

     

    This will be kind of un-sustainable for future upgrades....

     

    I set the database to upgrade and it is just taking too long. I have not received any error but is it taking over 30 minute/day...

     

    Thanx again for your reply on a holiday!!!

     

    Steve



  • 4.  Re: Computing fork value for table appmap_states
    Best Answer

    Posted May 31, 2016 06:20 AM

    In the end, Support said there was something wrong with the postgres tables appmap_states and the alternatives were to restart the upgrade and lose the APM DB or wait it out. At this point, I was into the middle of April and decided to wait it out (it was the holiday and I did not have anyone screaming).

    I was able to wait and then the upgrade went fine. Now I will probably have to deal with this on the next upgrade in a couple of months and I will probably restart the APM DB at that time...but since I had already invested 15 or so hours (most of the time NOT at the desk) I decided to wait it out and in the end it took about 21 hours.

     

    Asi es la vida...



  • 5.  Re: Computing fork value for table appmap_states

    Broadcom Employee
    Posted May 31, 2016 09:27 AM

    Hi Steve:

    Marking this as answered. Although you still have some work to do

    Thanks

    Hal German