Test Data Manager

Expand all | Collapse all

Consistent Masking across applications

  • 1.  Consistent Masking across applications

    Posted 04-12-2016 09:23 PM

    Hi,

     

    I have a scenario where one of my application is on IMS and other on SQL Server. Both applications have customer demographic details and I have to privatize these details.

     

    To ensure "Mark" is "Peter" for customer ID 12345 (for example) in both applications after masking, that is, to have consistent masking what are the best practices in FDM?

     

    I know we can use hashlov function but what should be the parameters to this function?

     

    For masking IMS, I am using FileMaster to generate copybook and CA TDm to generated transformation maps and for SQL Server I am using FDM directly.

     

    Also we have million records in a table to be masked using hash function, how the masking speed can be improved as my previous run took 30 hrs for masking firstname for 0.3 million records that too using 30 threads.



  • 2.  Re: Consistent Masking across applications

    Posted 04-13-2016 11:26 AM

    Hi Swapnil, we are currently researching your questions and will get back to you shortly.



  • 3.  Re: Consistent Masking across applications

    Posted 04-14-2016 08:18 AM

    Hi Swapnil,

     

    Here is some more information about understanding masking functions and parameters in FDM, and some FDM best practices:

    Understand Masking Functions and Parameters - CA Test Data Manager - 3.5 - CA Technologies Documentation 

    Fast Data Masker Best Practices - CA Test Data Manager - 3.5 - CA Technologies Documentation

     

    If FDM connects to the same DB2 table on the mainframe to get its seed data, AND if option SEEDTABLEINDEXCOLUMN=RL_RN is set, they should both mask the same.

     

    Required mainframe options are :

    TRIMMEDHASHLOV=Y

    HASHTYPE=JAVA

    CASEINSENSITIVEHASHLOV=Y

     

    Please let us know if this helps or if you have any follow up questions.



  • 4.  Re: Consistent Masking across applications

    Posted 04-14-2016 08:18 PM

    Thanks Taylor.

     

    I tried running the masking job in 10 threads however got the error. I have attached option and masking file here.

     

    Also, I could not find SEEDTABLEINDEXCOLUMN in FDM Is it available on mainframe only?



  • 5.  Re: Consistent Masking across applications

    Posted 04-14-2016 08:19 PM

    Masking File:

     

    Table,Column,Function,Parm1,Parm2,Parm3,Parm4,KeepNulls,DateFormat,Cross Reference,Override SQL,Unique Columns,Xpath Element,Substr start,Substr length,Notes,Preformat,update,Use Masked Values,restart column

    esit_gcm_name,,Where,ciskey like'1%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'2%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'3%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'4%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'5%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'6%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'7%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'8%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'9%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,

    esit_gcm_name,,Where,ciskey like'0%',,,,N,,,,,,,,,,,,

    esit_gcm_name,first_name,HASHLOV,firstnames.txt,,ciskey,,Y,,,,ciskey,,,,,,,,



  • 6.  Re: Consistent Masking across applications

    Posted 04-14-2016 08:20 PM

    Option File:

    AUDIT=SAMPLE10

    AUDITDIR=G:\Westpac TDM Team\Output Files\Rahul\Workspace\ISSMARIN\logs

    AUDITFILE=myaudit.csv

    AUDITPASSWORD=Rahul123

    AUDITZIP=winzip

    BACKUPDIR=G:\Westpac TDM Team\Output Files\Rahul\Workspace\ISSMARIN\\backup

    DIAGLEVEL=1

    LOGDIR=G:\Westpac TDM Team\Output Files\Rahul\Workspace\ISSMARIN\logs

    PARALLEL=10



  • 7.  Re: Consistent Masking across applications

    Posted 04-14-2016 08:51 PM

    Error Log:

    ############################################################
    #   Default Logging Configuration File
    #
    # You can use a different file by specifying a filename
    # with the java.util.logging.config.file system property. 
    # For example java -Djava.util.logging.config.file=myfile
    ############################################################

    ############################################################
    #   Global properties
    ############################################################

    # "handlers" specifies a comma separated list of log Handler
    # classes.  These handlers will be installed during VM startup.
    # Note that these classes must be on the system classpath.
    # By default we only configure a ConsoleHandler, which will only
    # show messages at the INFO and above levels.
    handlers= java.util.logging.ConsoleHandler

    # To also add the FileHandler, use the following line instead.
    #handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler

    # Default global logging level.
    # This specifies which kinds of events are logged across
    # all loggers.  For any given facility this global level
    # can be overriden by a facility specific level
    # Note that the ConsoleHandler also has a separate level
    # setting to limit messages printed to the console.
    .level= SEVERE

    ############################################################
    # Handler specific properties.
    # Describes specific configuration info for Handlers.
    ############################################################

    # default file output is in user's home directory.
    java.util.logging.FileHandler.pattern = %h/java%u.log
    java.util.logging.FileHandler.limit = 500000
    java.util.logging.FileHandler.count = 1
    java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter

    # Limit the message that are printed on the console to INFO and above.

    java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter

    # Example to customize the SimpleFormatter output format
    # to print one-line log message like this:
    #     <level>: <log message> [<date/time>]
    #
    # java.util.logging.SimpleFormatter.format=%4$s: %5$s [%1$tc]%n

    ############################################################
    # Facility specific properties.
    # Provides extra control for each logger.
    ############################################################

    # For example, set the com.xyz.foo logger to only log SEVERE
    # messages:
    #com.wibu.xpm.Runtime.level = FINEST
    com.wibu.xpm.Runtime.level = SEVERE



  • 8.  Re: Consistent Masking across applications

    Posted 04-26-2016 08:24 AM

    Hi Swapnil, If you are still facing masking issues, please open a support case so that we can investigate this issue further. I want to make sure we are not posting too much of your system and database information.

    If you are no longer facing this issue, we would appreciate it if you marked your question as answered and posted the way you resolved the issue.



  • 9.  Re: Consistent Masking across applications

    Posted 04-29-2016 04:03 AM

    Hi Taylor,

     

    I am not able to find SEEDTABLEINDEXCOLUMN in FDM option tab. FDM Version: 4.2.200.50