Symantec Identity Management

Expand all | Collapse all

Deploy a vApp Sandbox with Production TDM Data

  • 1.  Deploy a vApp Sandbox with Production TDM Data

    Posted 12-18-2017 09:23 PM

    Team,

     

    I am refining a method to assist with design sessions & provide customers, consultants, partners an offline vApp sandbox with current production TDM data (non-sensitive information).   Provisioning Data will be pulled in as well.  

    TDM (Test Data Management) with CA Identity Suite/vApp 

     

    While I document the entire process, I thought I would share the overall methodology & draft sample scripts.

     

    The benefit of this process will assist with documentation, review of the business logic in PX/Identity Policies, and building of automation test scripts.

     

     

     

    Sandbox or Prod: Migration Data flow

    Speed versus Quality

    Using the Identity vAPP for design sessions or migrations with Customers TDM Data (IMCD/IMPD/IME)

    Review methodology & execution

     

     

     

     

     

    Thoughts?

     

     

    -A



  • 2.  Re: Deploy a vApp Sandbox with Production TDM Data

    Posted 12-21-2017 07:21 AM

    Hi Allan

     

    Here is what I think the current build of the migration tools can assist in:

    1. User store migration
      1. Attribute mapping
        1. will generate an update XML file for the new environment with the additional attributes required
        2. Schema updates - schema elements for the imUserAux schema
        3. List of attributes for the data migration
      2. Data migration
        1. rename from source names to target name
        2. change OC and RND
        3. change references to the environment name (e.g. Identity Polices)
        4. change DN for manager and group members
        5. Create Manager ID from Manager DN
        6. Add Login ID
    2. Provisioning Store Migration
      1. Handle differences in domain name
      2. update etaadmin password
      3. Move all data
    3. Enviroment
      1. Filter relevant elements (roles, tasks, etc)
      2. Handle dependencies (e.g. add tasks pased PX, admin roles and screen based on task)
      3. Rename fields according to user store mapping
      4. Add Login ID to create screens
      5. Odd Organization for OrgLess enviroment

     

    The unavoidable disclaimer: This is work in progress. I will update builds as they come out.



  • 3.  Re: Deploy a vApp Sandbox with Production TDM Data

    Posted 12-21-2017 04:01 PM

    Thanks Gil,

     

    I will be happy to work with any version.

     

    FYI - Below are current timings for the load of data with minimal data / attribute mapping.

     -  Note: timing are dependent on disk I/O & fragmentation of data on the disc

     

    ### ###

     

    Validating process with 300 K identities on Vmware Workstation Image with 4vCPU, 16 GB RAM 

     

     

    Time for IMCD Pre-processing:  15 min for 300 K

       1)  Create temp DSA to convert customer imcd ldif data to "no-wrap" at column 78

             a) Load loading customer ldif data (with customer schema)

             b) Use dxsearch & perl to extract with "no-wrap"

     

        2) Convert customer domain to vApp domain & create two (2) input files:   [5 min for each]

          a) collapse all ou=people types to a single ou=people

          b) keep hierarchic ou=people types

     

    ***I created two (2) input files to validate behavior with the vApp and the default IME / replacement IME Roles & Tasks.   One with the collapsed ou structure (some customers separate service accounts from standard users) 

     

     

     

    Populate IMCD with above data   20 min for 300 K

       1)  Update im_user_aux.dxc  with customer schema

              a) String replace to match vApp nomenclature

       2)  Update incoming ldif with vApp objectClass:  imUser for each user object

              a) ldifsort data to remove duplicates or bad ldif data

       3) Load updated customer LDIF file to vApp IMCD dsa

       4) Overwrite/Append vApp IMCD dsa

            a) dxmodify to ensure default OU structure & etaadmin account's password hash is updated 

     

    Validate able to view data & search on objectClass: imUser for all customer identities

       1) Use 3rd party tool:  Jxplorer or SoftTerra LDAPBrowser or Apache Dir Studio

     

     

     

    Populate IMPD (not required to convert):    25 min for 300 K GU identities   & millions entries for correlated endpoint accounts/inclusions

     

     

     

     

    Cheers,

     

    A.



  • 4.  Re: Deploy a vApp Sandbox with Production TDM Data

     
    Posted 12-22-2017 11:56 PM

    Alan,

     

    Your have it for OOTB deployment - which address a large user population. In case there are custom components (class files) and custom Connector Xpress connector, then the associated .con and .jar files will have to be brought in also.  Also if there are mainframe custom attributes, ADS schema extension etc etc . those may also have to be factored in.

     

    -Bala



  • 5.  Re: Deploy a vApp Sandbox with Production TDM Data

    Posted 12-23-2017 12:13 AM

    Hi Bala,

     

    You are correct.   I did not have any issues with pulling in the CX Con files (since they are stored as XML data within the IMPD DSA dynamic endpoints).  I could pull this data and create a new project.    However, after I found this info, I was able to identify the jar files, and have those provided to me as well.

     

    The most challenging part was the Identity Portal import, as a r12.6.8 to r14.1, there is some slight changes to objects.

    Using the debug feature of log4j, I was able to identify the deltas, to allow an import.

     

    I also updated my methodology for the import to adjust for the changes.

     

    Example of log4j debug loggers for IP:

     

     

     

    Example:  IP Import JSON files per object type:

     

     

     

    Example of methodology for import for IP:

     

     

     

    I now have an "alpha" version of a "sandbox" for design sessions with the customer.   

    I will vet the use-cases within the sandbox in Jan/Feb.

     

     

     

     

    Cheers,

     

    A.



  • 6.  Re: Deploy a vApp Sandbox with Production TDM Data

    Posted 12-24-2017 03:11 AM

    After I have shared this image, I needed a way to manage the required fixed IP address.

     

    Here is the process I have used:

     

    Update Vmware Image's Dynamic IP Address to Static IP (outside of the image) 

     

     

    Cheers,

     

    A.



  • 7.  Re: Deploy a vApp Sandbox with Production TDM Data

    Posted 01-24-2018 06:51 PM

    Follow up for IP (Sigma) Migration Process using the vApp r12.6.8 and vApp r14.1

     

    Customer provided an exported full JSON file for the Identity Portal r12.6.6.     

     

    Steps to upgrade to IP r14.1 using the vAPP.

     

    1) Deploy the vApp r12.6.8 in a sandbox configuration with IP & IM & IMCD (minimal) & embedded Oracle XE DB.

    2) Login to the vApp r12.6.8 IP Management Console & export the current OOTB JSON file for IP

    3)  Delete all vApp r12.6.8 IP objects with the exception of General Configuration & IM Connector.

    4)  Rename the IM Connector to the naming convention used by customer IP data.

       -  Review the IM objects/tasks used by the IM Connector, to ensure they align with standard OOTB tasks

    5)  Edit the customer JSON file, using either Notepad++ (with JSONviewer plugin) or JSONBuddy (3rd party tool)

       - Remove the General Configuration & IM Connector objects

       - Save this updated file.

    6)  Import the newly updated customer data;  ensure no error on import.

    7)  Restart the IP service and ensure no errors in the logs; as config userID:   stop_ip   /  start_ip

    8)  Download both the IP upgrade packages and copy them to the /home/config folder

        - IdentityPortal_14.0_PSF 71809.zip

        - IdentityPortal_14.1_PSF 72340.zip

      9)  Refer to wiki for installation processes and confirm next steps

    Upgrading CA Identity Portal - CA Identity Suite - 14.0 - CA Technologies Documentation 

    Upgrading CA Identity Portal - CA Identity Suite - 14.1 - CA Technologies Documentation 

    10)  Steps to upgrade vApp r12.6.8 IP to IP 14.0

            a) Shutdown IP.
            b) Execute before.sql script     [Connect to the vApp r12.6.8 Oracle XE Database]

                      -  JDBC THIN URL:   jdbc:oracle:thin:@//192.168.242.151:1521/XE

                      -  LOGINID:  IDENTITYPORTAL    PWD:  CAIMAG1!   (per vApp Help Guide)

                      -  Example of before.sql

                               ALTER TABLE SIGMAMODULE MODIFY (TEMPLATE varchar2(255) null);
                               commit;

            c) Start up IP

            d)  Copy/Deploy r14.0 sigma.war file

                      -  Extract the sigma.war from the IP upgrade package.

                      -  Create a local wildfly/jboss admin user to deploy the IP update package

                          sudo /opt/CA/wildfly-portal/bin/add-user.sh             [ add a new admin userID: admin1 ]

                      -  Create a jboss-cli.sh  script or copy the following three (3) lines

     

    /opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="deployment-info"

    /opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="undeploy sigma.war"

    /opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="deploy /home/config/upgrade/r14-0_UpgradePackage/Sigma\ Files/sigma.war"

     

                      -  Execute each of the three (3) lines, to confirm current deployment, remove current deployment, deploy the latest sigma.war file.

                      -  Monitor for any error messages in the IP logs.

     

     

             e)   Execute the after.sql script

                      alter table PROFILE drop column REQUESTSCOPE_ID;
                      alter table TASKRULE drop column TARGETPERMISSION_ID;
                      commit;

     

           f) Restart the IP services:   stop_ip  / start_ip

           g)  Monitor for any error messages in the IP logs.

               -  ####  One error noted was conflict with JBOSS JMS Queue Port availablity; updated the IP reference XML files.

                    sed -i 's|port_range="0"|port_range="20"|g' sigma-hibernate-jgroups-unicast.xml
                    sed -i 's|port_range="0"|port_range="20"|g' sigma-portal-jgroups-unicast.xml

     

    11) Export the latest updated IP JSON file of the configuration using the customer data.

     

    12)   May repeat the above process for the IP r14.1 upgrade package or may directly import IP r14.0 into the vApp r14.1 image.

          -  ### Note:  Two (2) minor issues with the IP objects; repaired with Notepad++   / Errors identified with IP logs. 

                           tag:  MODULE_LANDING_PAGE  was missing  objectType: "NONE"

                           tag:  MODULE_MYREQUESTS was missing objectType: "USER"

     

     

     

    This process allows for the resources to review design changes between the current state architecture of r12.6.x to r14.1 using the exact customer data.

     

     

     

    Cheers,

     

    A.