Since, I am performing data generation locally on docker containers:
Do I have to enable batch mode (
application.properties) as follows:
- tdmweb.publish.batchCommit=true
Or
Do I have to edit docker-compose.yml file as follows:
- 'APPLICATION_PROP="tdmweb.publish.batchCommit=true"'
Original Message:
Sent: 10-18-2019 05:11 AM
From: Abderrahmane Zahrir
Subject: Data Generation velocity for publishing to SQL Server (using Docker)
If you are inserting new data and do not expect issues with duplicates then you can enable batch mode (update application.properties) so that generated data is not pushed one row at a time to the database.
You should also make sure that network bandwidth is high enough between your portal and database
Regards, Dahman
Original Message:
Sent: 10-17-2019 01:46 PM
From: Amanpreet Kaur Aulakh
Subject: Data Generation velocity for publishing to SQL Server (using Docker)
Hi Team
To generate 1 million rows of data (for 7 columns using docker containers) and publish to:
- database( SQL server) took around 6 hours.
- CSV/ SQL file took around 1 hour
Is there any way to speed up the publish (generated data) to SQL server?
Thanks
Amanpreet Aulakh