I receive a file with n number of records for masking. File 1st column: HH, HDR,DTL,HDRDTL etc.. these are like a file and having definition for their columns. (Please refer below eg for files- this is conditional file with definition)
these columns names are getting occurred inconsistently, please suggest how to mask such kind of files data using FDM or TDM?
HH^~^DDE^~^436^~^EOD^~^20150325^~^20150326HDR^~^101334^~^KUIYO^~^Salary -HDRDTL^~^101334^~^50110717^~^01DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^HDR^~^101334^~^KUIYO^~^Salary -HDRDTL^~^101334^~^50110717^~^01DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^HDRDTL^~^101334^~^50110717^~^01DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^DTL^~^01^~^00002^~^50110717^~^
you can split file into multiple files using VB script or manually. Mask individually and combine them later. if you need ordering in the masked file add one column to original file with sequential numbering.
Please let me know if you have any questions, am happy to help.
Files are having more than 1Lakh records in it, do you have any utility to extract data into different file based on format and again combine all the files data based on row value?
I wonder if this is a viable approach for you to use FDM for this problem:
1. In your input file, globally substitute ',' for '^~^' as the column delimiter and add a header record - e.g. 'RECTYPE,FLD1,FLD2,FLD3,FLD4,FLD5,FLD6'. Then label the input file as CSV, and use the column labels in the where clause and in masking.
2. In the Options settings in FDM, change option 'WHEREASSUBSET' to 'N', which changes the behaviour of the 'Where' clause. According to the option description: "Y (default), flat files are scrambled AND subsetted according to where clause, N use WHERE as criteria for masking and output all records" - so, instead of the 'where' clause causing only records that match the 'where' to be output, FDM will apply the mask to the selected records but output all records, matched and unmatched.
3. If you need to apply different masking to different record types, you can create a different mask for each record type with a 'where' clause based on 'RECTYPE', and apply the masks sequentially, each mask taking as input the file output by the previous mask.
4. Finally, delete the header record from the output file and globally replace ',' with '^~^', to reproduce the original input format.
Hope that helps!