Test Data Manager

 View Only
Expand all | Collapse all

Unable to run .sh file on FDM

  • 1.  Unable to run .sh file on FDM

    Posted Apr 27, 2020 11:07 AM
    Hi,

    We have installed only FDM console mode on Linux and not whole package of CA TDM. The contents of .bat file are copied to .sh file and other files such as connection file, .csv file and seedfile are copied as well on Linux. There are few extra contents in batch file which are not present in FDM Linux files and it is throwing error while executing the shell file.

    Can you please assist here. Much appreciated.
    Sharing below the contents of batch and shell file along with the installed FDM files screen.
    .bat file:
    java -Dfile.encoding=UTF-8 -Djava.util.logging.config.file="C:/Program Files/Grid-Tools/FastDataMasker/logging.properties" -Xms100M -Xmx1000M -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path="%PATH%;C:/Program Files/Grid-Tools/FastDataMasker/SQLSERVER_DLLs/x64" -cp ".;C:/Program Files/Grid-Tools/FastDataMasker/Fastdatamasker.jar;C:/Program Files/Grid-Tools/FastDataMasker/lib/*;C:/Program Files/Grid-Tools/FastDataMasker/custom/*;" com.grid_tools.products.datamasker.Datamasker "C:/Users/zashish.jaiswal/Desktop/connect_test.txt" "C:\Users\zashish.jaiswal\Desktop\test.csv" "C:\Users\zashish.jaiswal\Desktop\test_options.txt"

    .sh file:

    java -Dfile.encoding=UTF-8 -Djava.util.logging.config.file="/opt/FDM/logging.properties" -Xms100M -Xmx1000M  -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path="%PATH%" -cp ".;/opt/FDM/Fastdatamasker.jar;/opt/FDM/lib/*;/opt/FDM/custom_config/*;" com.grid_tools.products.datamasker.Datamasker "/home/Premium/connect_test.txt" "/home/Premium/test.csv" "/home/Premium/test_options.txt"

    Installed FDM files:

    [zashish.jaiswal@tdc1app020 ~]$ cd /opt/FDM
    [zashish.jaiswal@tdc1app020 FDM]$ ls -ltr
    total 103628
    drwxrwxr-x 5 1000 1000 121 Jun 18 2019 jre
    -rwxrwxr-x 1 root root 603578 Jul 22 2019 FDM.jar
    -rwxrwxr-x 1 root root 2462 Jul 22 2019 logging.properties
    -rwxrwxr-x 1 root root 814 Jul 22 2019 JavaSupportedEncodeMapping.propert ies
    -rwxrwxr-x 1 root root 548 Jul 22 2019 custom_config.xml
    -rwxrwxr-x 1 root root 85769 Jul 22 2019 config.xml
    -rwxrwxr-x 1 root root 78336 Jul 22 2019 BuildMap.xls
    -rwxrwxr-x 1 root root 355 Jul 22 2019 bigfdm.sh
    -rwxrwxr-x 1 root root 731486 Jul 22 2019 Fastdatamasker.jar
    -rwxrwxr-x 1 root root 728071 Jul 22 2019 Fastdatamasker_flat.jar
    -rwxrwxr-x 1 root root 728071 Jul 22 2019 Fastdatamasker_simulate.jar
    -rw------- 1 root root 102971351 Mar 30 14:06 fast_data_masker.tar.gz
    drwxr-xr-x 3 root root 44 Mar 30 14:19 FDM_Installer_Linux
    drwxrwxr-x 2 root root 31 Mar 30 15:06 GTFiles
    drwxrwxr-x 2 root root 4096 Mar 30 15:06 lib
    drwxrwxr-x 2 root root 4096 Mar 30 15:06 seedtables
    drwxrwxr-x 2 root root 45 Mar 30 15:06 training
    -rw-rw-r-- 1 root root 54889 Mar 30 15:06 lax.jar
    -rwxr-xr-x 1 root root 83377 Mar 30 15:06 FastDataMasker
    -rwxrwxr-x 1 root root 3043 Mar 30 15:06 FastDataMasker.lax
    -rw-rw-r-- 1 root root 124 Mar 30 15:06 FDM.config
    drwxrwxr-x 3 root root 230 Mar 30 15:06 _FastDataMasker_installation




    ------------------------------
    Thanks & Regards,
    Ashish Jaiswal
    ------------------------------


  • 2.  RE: Unable to run .sh file on FDM

    Posted Apr 28, 2020 04:04 AM
    Hi, 
    I got one of the thread response for my query but while running it is throwiing below error.

    $ java -Dfile.encoding=UTF-8 -Djava.util.logging.config.file="/opt/FDM/logging.properties" -Xms100M -Xmx1000M -XX:+HeapDumpOnOutOfMemoryError -cp ".;/opt/FDM/Fastdatamasker.jar;/opt/FDM/lib/*;/opt/FDM/custom/*;" -jar "/opt/FDM/Fastdatamasker.jar" "/tmp/connect_test.txt" "/tmp/test.csv"
    2020-04-28 08:52:15.865 Fastdatamasker version: 4.8.100.0
    2020-04-28 08:52:15.872 Fastdatamasker build date - July 22 2019
    2020-04-28 08:52:15.874 PID:67990@test
    2020-04-28 08:52:15.875 Java version 1.8.0_242
    2020-04-28 08:52:15.876 OPERATING SYSTEM USER: tmp
    2020-04-28 08:52:15.887 Skipping cloakware configuration
    2020-04-28 08:52:15.891 java.lang.NoClassDefFoundError: com/ca/tdm/eventserviceclient/exception/EventPostException
    at com.grid_tools.products.datamasker.Datamasker.main(Datamasker.java:2720)
    Caused by: java.lang.ClassNotFoundException: com.ca.tdm.eventserviceclient.exception.EventPostException
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
    ... 1 more


    Can you please assist here. Thanks in advance.
    Ashish Jaiswal


  • 3.  RE: Unable to run .sh file on FDM

    Broadcom Employee
    Posted Apr 28, 2020 05:40 AM
    Ashish could you please explain how you installed FDM on Linux?  Did you used the standalone FDM install image?


  • 4.  RE: Unable to run .sh file on FDM

    Posted Apr 28, 2020 06:24 AM
    Hi Billy,

    We installed only FDM in console mode and not whole CA TDM package.


  • 5.  RE: Unable to run .sh file on FDM

    Broadcom Employee
    Posted Apr 28, 2020 08:56 AM
    Ashish, here's a running version as a reference.   Note that it is used for masking flat files so I'm referencing FastDatamasker_flat.

    Two differences I see:

    (1) I keep the reference to com.grid_tools.products.datamasker.Datamasker in place
    (2) The jar paths after the -cp are separated by colons (:) rather than semi-colons (;) as you have

    <x-row>java -Dfile.encoding=UTF-8 -Djava.util.logging.config.file="/home/scott_schmitz/fdm/logging.properties" -Xms1000M -</x-row><x-row>Xmx4000M -XX:+HeapDumpOnOutOfMemoryError -cp ".:/home/scott_schmitz/fdm/Fastdatamasker_flat.jar:/home/scott_schmit</x-row><x-row>z/fdm/lib/*:" com.grid_tools.products.datamasker.Datamasker "/home/scott_schmitz/fdm/ccc.txt" "/home/scott_schmitz/</x-row><x-row>fdm/custmask.csv"</x-row>


  • 6.  RE: Unable to run .sh file on FDM

    Posted Apr 29, 2020 01:17 AM
    Hi Scott,
    Thanks for your input. I tried with the above script and now getting below error with regards to portal.

    2020-04-29 06:14:18.828 Fastdatamasker version: 4.8.100.0
    2020-04-29 06:14:18.835 Fastdatamasker build date - July 22 2019
    2020-04-29 06:14:18.838 PID:13650@test
    2020-04-29 06:14:18.838 Java version 1.8.0_242
    2020-04-29 06:14:18.839 OPERATING SYSTEM USER: zashish.jaiswal
    2020-04-29 06:14:18.850 Skipping cloakware configuration
    2020-04-29 06:14:18.856 The connection to Test Data Manager telemetry service is currently disabled.
    You can reenable the connection to the service by ensuring that environment variable TDM_PORTAL_URL is set and points to the
    hostname where TDM Portal is installed (e.g. 'https://hostname:8443' or 'http://hostname:8080').
    For details, please refer to the Test Data Manager documentation at https://docops.ca.com/display/TDM/.Configure+Telemetry+v4.8
    2020-04-29 06:14:18.862 Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
    javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
    at com.sun.crypto.provider.CipherCore.unpad(CipherCore.java:975)
    at com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1056)
    at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:853)
    at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:432)
    at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316)
    at javax.crypto.Cipher.doFinal(Cipher.java:2164)
    at com.grid_tools.products.datamasker.DesEncrypter.decrypt(DesEncrypter.java:83)
    at com.grid_tools.products.datamasker.Datamasker.main(Datamasker.java:2753)

    2020-04-29 06:14:18.868 Error decrypting epassword
    2020-04-29 06:14:18.870 exit value = 1



  • 7.  RE: Unable to run .sh file on FDM

    Posted Apr 30, 2020 11:10 AM
    Hi Team,

    Any update on the above error message which we have got while executing few Linux commands from Linux console.


  • 8.  RE: Unable to run .sh file on FDM

    Broadcom Employee
    Posted May 01, 2020 06:37 AM
    The portal message is the standard message you get when you dont have a connection set up to it.  This is not the problem.   What are you masking?  When  I run on linux i see the message sequence of:

    2020-05-01 11:33:49.387 The connection to Test Data Manager telemetry service is currently disabled.
    You can reenable the connection to the service by ensuring that environment variable TDM_PORTAL_URL is set and points to the
    hostname where TDM Portal is installed (e.g. 'https://hostname:8443' or 'http://hostname:8080').
    For details, please refer to the Test Data Manager documentation at https://techdocs.broadcom.com/bin/gethidpage?space=TDM49&hid=HID_telemetry&language=&format=rendered
    2020-05-01 11:33:49.395 attempting to connect
    .....



  • 9.  RE: Unable to run .sh file on FDM

    Posted May 03, 2020 02:00 AM
    Hi Billy,

    Yes, It is a standard message.I copied the database connection file from windows to linux and connection file has epassword into it.
     It is unable to decrypt epassword and throwing error decrypting epassword error. I am able to connect FDM to database using the same connection on windows. Is there any other way to define epassword on linux?

    Please assist.

    Thanks,
    Ashish Jaiswal


  • 10.  RE: Unable to run .sh file on FDM
    Best Answer

    Broadcom Employee
    Posted May 04, 2020 03:48 AM
    Make sure that you are using the same FDM version on linux as the one on windows that generated the config files. 
    we have improved security so that encryped passwords use AES 256 and it is possible that the version on linux can not decrypt the password

    Regards, Dahman





  • 11.  RE: Unable to run .sh file on FDM

    Posted May 05, 2020 07:43 AM
    Thanks for your input Dahman.

    We have asked the respected team to install the latest version on Linux. Then we can run the command again on Linux.

    Thanks,
    Ashish Jaiswal