Test Data Manager

Expand all | Collapse all

CA TDM Portal Slow in Cloning

  • 1.  CA TDM Portal Slow in Cloning

    Posted 08-30-2018 02:58 PM

    Hello,

     

    I has a data generator with a subset and all the rules of the generador are sql. When I execute the generator trought Gt Datamaker, it finish the publish in 2 minutes, but when a execute the same generator but by CA TDM Portal it complete the publish in 28 minutes.

     

    TDM Portal

     

    GT DataMaker  

     

    I tried modify the wrapper.java.initmemory and wrapper.java.maxmemory according to:

    TDM Portal Performance 

    Increasing CA TDM Portal JVM memory - CA Knowledge 

     

    It is only one iteracction but I tried modify the tdmweb.publish.iterationsBeforeCommit option to but que slow persist.

     

    The subset has 44 tables, and it delay in inserting each record because when more rows has a table more slower is.

     

    Nota: When I executed the same generator by GT Datamaker it is faster and take 2 minutes maximum.

     



  • 2.  Re: CA TDM Portal Slow in Cloning

    Posted 08-30-2018 04:00 PM

    John,

     

    What version of TDM Datamaker and Portal do you have installed?

     

    Where is the repo located? Is it local to the TDM server or remote? SQL or Oracle?

     

    Regards,

    Sean



  • 3.  Re: CA TDM Portal Slow in Cloning

    Posted 08-30-2018 04:19 PM

    Hello Sean,

     

    I has the versión 4.4.24.0 of the TDM Portal (This versión is a specific versión that solve a issue with data painter)

    https://comm.support.ca.com/kb/error-when-attempting-to-publish-tdmserviceexception-randlov-john-not-a-number/kb000009265

     

    And the versión 4.4.0.23 of GT Datamaker

     

    The repo is located in a remote server, both the GT Datamaker and TDM portal are in the same server and it connet to a repository in other server in a SQL database.



  • 4.  Re: CA TDM Portal Slow in Cloning

    Broadcom Employee
    Posted 09-03-2018 02:49 AM

    Hi,

     

    Are you using the same connection profiles in both Datamaker and TDM Portal?

     

    Regards,

    Peter



  • 5.  Re: CA TDM Portal Slow in Cloning

    Posted 09-03-2018 08:43 AM

    Hello,

     

    I'm using a connection to the same database but the profile was created in both Datamaker and TDM Portal in each one. But it has the same conection configurations.

     

    Thanks.



  • 6.  Re: CA TDM Portal Slow in Cloning

    Broadcom Employee
    Posted 09-03-2018 08:56 AM

    Hi johnviot

     

    I am struck by your observation that:

    The subset has 44 tables, and it delay in inserting each record because when more rows has a table more slower is.'

     

    I have sometimes seen situations where fresh target databases are extremely sensitive to bulk inserts when they are first being populated.  Until they reach a certain size, the database statistics have been refreshed and the optimizer can build a stable execution plan, insert behaviour can be very unpredictable.

     

    You may well have already explored this but it leads me to a couple more questions for clarification please:

    • What is the target database (Oracle, SQL Server, etc)?
    • Where is it in relation to the TDM server?
    • What is the size of the target DB compared to the data being generated (before and after)?
      • Are DB stats refreshed before the next bulk insert of newly generated data?
    • Is this slow down repeatable?
      • I mean, if you continue to grow the target DB by adding another set of generated data via DataMaker, then Portal, then DataMaker, then Portal, is DataMaker always fast (2min) and Portal always slow (28min)?
      • If you start with an empty target schema, and load data from Portal first instead, does the performance change?

     

    Regards,

                  Adrian



  • 7.  Re: CA TDM Portal Slow in Cloning

    Posted 09-03-2018 09:51 AM

    Hello AdrianJohnsonUK

     

    • What is the target database (Oracle, SQL Server, etc)?
      • A// Oracle
    • Where is it in relation to the TDM server?
      • A//The database is in a diferen server, it is a AIX in a lab vlan.
    • What is the size of the target DB compared to the data being generated (before and after)?
      • Are DB stats refreshed before the next bulk insert of newly generated data?
      • A// Although it's a subset of 44 tables, in this case it only has a few data, the table that has more rows has 116 and normally each one table has 1 to 10 rows and many tables don't has rows(0). The size of the target DB compared to the data before and after is minimum.
    • Is this slow down repeatable?
      • I mean, if you continue to grow the target DB by adding another set of generated data via DataMaker, then Portal, then DataMaker, then Portal, is DataMaker always fast (2min) and Portal always slow (28min)?
      • A// Yes it always the same time, it is indifferent, if I do a request via DataMaker 2 or 3 times consecutive always take 2 minutes or 2 or 3 times with TDM portal always takes 28 minutes, and Gt Datamaker-Portal-Datamaker-Portal always the same times.
      • If you start with an empty target schema, and load data from Portal first instead, does the performance change?
      • A//The database are empty, I usually clean the tables, but I tried with update without clean the table and it takes the same times.

    Thanks.