Option to optimize the performance of the Database Masking

[Ronald Dsouza]

Hi there,
We are trying to obfuscate data in a MSSQL database that consist around 3 million records we are using a combination of SQL function and Hash List of Value as we need to have referential integrity between the data that is obfuscated between tables.  However we noticed the speed is quite slow it process it takes average of 3 hours to mask 10000 records.

We have use parallel connection and largesplit table (did not work as we use SQL functions) gave it init memory of 24 GB and Max memory of 26 GB on a 32 GB system. 

Is there anything else we can do to expedite this process as the time take is to long and we are doing a POC.

Any other requirement that we need to keep in mind when masking large tables.

[Scott Schmitz]

@Ronald Dsouza

Has the Resource Usage on the SQL Server been monitored during this time?   Sounds more like a potential bottleneck during the SQLFunction / queries / writes, unless you can confirm that the database server's memory/CPU/network usage is within norms during the process.​

[Ronald Dsouza]

Hi Scott,

Checking the performance on the SQL server is something I need to get confirmed by the DBA . will check and update.

[Ronald Dsouza]

Hi Scott,

MSSQL Server seems to be one of the issue as the POC box was under spec with only 32 GB of RAM of which 26 GB was allocated to the MSSQL server. So we did find one culprit. 

Are there any guides with regards to Best practice guide that we can follow in terms of ensuring proper masking on both places on the Target and on the TDM Server or even any standard performance guide we can distribute.

Thanks,
Ronald

[Scott Schmitz]

See this library entry for a Best Practices guide.