I am trying to perform in-place masking using Fast data masker. The data base that i am trying to mask is in SQL and the lookup table is in oracle. When i run the masking script , it is trying to load all the data in the lookup table first which is taking a lot of time.For masking 5000 rows that too only column it is taking 1 hour.
What are you trying to achieve? It is a known limitation with FDM that if you want to maintain consistency with a 100% guarantee and hence are using XREF method, it will be time consuming because of the additional table update and lookup involved.
However, if you 100% is not important then you can use other functions that can server the purpose. Once we know what is it that you are actually trying to achieve, we can guide you further.
Thanks for the information. We are trying achieve consistent masking across databases with one single lookup table as a source of truth
for masking and yes we need 100% guarantee. Our data are in IMS, DB2 , SQL, ORACLE.
Our main concern is that , in the start of the masking uisng FDM , the time to load the lookup table is consuming a lot of time.
Our lookup table has 2 million records and it was taking 1 hour aprox to load the data and start the masking.
Is there a way to minimise this time?
Please ensure the following activities are performed before masking.
You may also need to give FDM more memory. The following link will be helpful.
You can also set the Max Memory (MB) in the FDM GUI.( you can find it in the bottom of the page)The number you enter in that depends on how many rows you have. It should always be in line or more.
For huge amounts of data, you should split the masking using where clauses into 3 or 4 and re-try. Pick a row that has unique values in them and put the where clauses using that.
Fire multiple jobs one after the other. I do not think you can run that in parallel.