I am refining a method to assist with design sessions & provide customers, consultants, partners an offline vApp sandbox with current production TDM data (non-sensitive information). Provisioning Data will be pulled in as well.
TDM (Test Data Management) with CA Identity Suite/vApp
While I document the entire process, I thought I would share the overall methodology & draft sample scripts.
The benefit of this process will assist with documentation, review of the business logic in PX/Identity Policies, and building of automation test scripts.
Sandbox or Prod: Migration Data flow
Speed versus Quality
Using the Identity vAPP for design sessions or migrations with Customers TDM Data (IMCD/IMPD/IME)
Review methodology & execution
Here is what I think the current build of the migration tools can assist in:
The unavoidable disclaimer: This is work in progress. I will update builds as they come out.
I will be happy to work with any version.
FYI - Below are current timings for the load of data with minimal data / attribute mapping.
- Note: timing are dependent on disk I/O & fragmentation of data on the disc
Validating process with 300 K identities on Vmware Workstation Image with 4vCPU, 16 GB RAM
Time for IMCD Pre-processing: 15 min for 300 K
1) Create temp DSA to convert customer imcd ldif data to "no-wrap" at column 78
a) Load loading customer ldif data (with customer schema)
b) Use dxsearch & perl to extract with "no-wrap"
2) Convert customer domain to vApp domain & create two (2) input files: [5 min for each]
a) collapse all ou=people types to a single ou=people
b) keep hierarchic ou=people types
***I created two (2) input files to validate behavior with the vApp and the default IME / replacement IME Roles & Tasks. One with the collapsed ou structure (some customers separate service accounts from standard users)
Populate IMCD with above data 20 min for 300 K
1) Update im_user_aux.dxc with customer schema
a) String replace to match vApp nomenclature
2) Update incoming ldif with vApp objectClass: imUser for each user object
a) ldifsort data to remove duplicates or bad ldif data
3) Load updated customer LDIF file to vApp IMCD dsa
4) Overwrite/Append vApp IMCD dsa
a) dxmodify to ensure default OU structure & etaadmin account's password hash is updated
Validate able to view data & search on objectClass: imUser for all customer identities
1) Use 3rd party tool: Jxplorer or SoftTerra LDAPBrowser or Apache Dir Studio
Populate IMPD (not required to convert): 25 min for 300 K GU identities & millions entries for correlated endpoint accounts/inclusions
Your have it for OOTB deployment - which address a large user population. In case there are custom components (class files) and custom Connector Xpress connector, then the associated .con and .jar files will have to be brought in also. Also if there are mainframe custom attributes, ADS schema extension etc etc . those may also have to be factored in.
You are correct. I did not have any issues with pulling in the CX Con files (since they are stored as XML data within the IMPD DSA dynamic endpoints). I could pull this data and create a new project. However, after I found this info, I was able to identify the jar files, and have those provided to me as well.
The most challenging part was the Identity Portal import, as a r12.6.8 to r14.1, there is some slight changes to objects.
Using the debug feature of log4j, I was able to identify the deltas, to allow an import.
I also updated my methodology for the import to adjust for the changes.
Example of log4j debug loggers for IP:
Example: IP Import JSON files per object type:
Example of methodology for import for IP:
I now have an "alpha" version of a "sandbox" for design sessions with the customer.
I will vet the use-cases within the sandbox in Jan/Feb.
After I have shared this image, I needed a way to manage the required fixed IP address.
Here is the process I have used:
Update Vmware Image's Dynamic IP Address to Static IP (outside of the image)
Follow up for IP (Sigma) Migration Process using the vApp r12.6.8 and vApp r14.1
Customer provided an exported full JSON file for the Identity Portal r12.6.6.
Steps to upgrade to IP r14.1 using the vAPP.
1) Deploy the vApp r12.6.8 in a sandbox configuration with IP & IM & IMCD (minimal) & embedded Oracle XE DB.
2) Login to the vApp r12.6.8 IP Management Console & export the current OOTB JSON file for IP
3) Delete all vApp r12.6.8 IP objects with the exception of General Configuration & IM Connector.
4) Rename the IM Connector to the naming convention used by customer IP data.
- Review the IM objects/tasks used by the IM Connector, to ensure they align with standard OOTB tasks
5) Edit the customer JSON file, using either Notepad++ (with JSONviewer plugin) or JSONBuddy (3rd party tool)
- Remove the General Configuration & IM Connector objects
- Save this updated file.
6) Import the newly updated customer data; ensure no error on import.
7) Restart the IP service and ensure no errors in the logs; as config userID: stop_ip / start_ip
8) Download both the IP upgrade packages and copy them to the /home/config folder
- IdentityPortal_14.0_PSF 71809.zip
- IdentityPortal_14.1_PSF 72340.zip
9) Refer to wiki for installation processes and confirm next steps
- Upgrading CA Identity Portal - CA Identity Suite - 14.0 - CA Technologies Documentation
- Upgrading CA Identity Portal - CA Identity Suite - 14.1 - CA Technologies Documentation
10) Steps to upgrade vApp r12.6.8 IP to IP 14.0
a) Shutdown IP. b) Execute before.sql script [Connect to the vApp r12.6.8 Oracle XE Database]
- JDBC THIN URL: jdbc:oracle:thin:@//192.168.242.151:1521/XE
- LOGINID: IDENTITYPORTAL PWD: CAIMAG1! (per vApp Help Guide)
- Example of before.sql
ALTER TABLE SIGMAMODULE MODIFY (TEMPLATE varchar2(255) null); commit;
c) Start up IP
d) Copy/Deploy r14.0 sigma.war file
- Extract the sigma.war from the IP upgrade package.
- Create a local wildfly/jboss admin user to deploy the IP update package
sudo /opt/CA/wildfly-portal/bin/add-user.sh [ add a new admin userID: admin1 ]
- Create a jboss-cli.sh script or copy the following three (3) lines
/opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="deployment-info"
/opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="undeploy sigma.war"
/opt/CA/wildfly-portal/bin/jboss-cli.sh -c controller=localhost:9991 --user=admin1 --password=Password01 --command="deploy /home/config/upgrade/r14-0_UpgradePackage/Sigma\ Files/sigma.war"
- Execute each of the three (3) lines, to confirm current deployment, remove current deployment, deploy the latest sigma.war file.
- Monitor for any error messages in the IP logs.
e) Execute the after.sql script
alter table PROFILE drop column REQUESTSCOPE_ID; alter table TASKRULE drop column TARGETPERMISSION_ID; commit;
f) Restart the IP services: stop_ip / start_ip
g) Monitor for any error messages in the IP logs.
- #### One error noted was conflict with JBOSS JMS Queue Port availablity; updated the IP reference XML files.
sed -i 's|port_range="0"|port_range="20"|g' sigma-hibernate-jgroups-unicast.xml sed -i 's|port_range="0"|port_range="20"|g' sigma-portal-jgroups-unicast.xml
11) Export the latest updated IP JSON file of the configuration using the customer data.
12) May repeat the above process for the IP r14.1 upgrade package or may directly import IP r14.0 into the vApp r14.1 image.
- ### Note: Two (2) minor issues with the IP objects; repaired with Notepad++ / Errors identified with IP logs.
tag: MODULE_LANDING_PAGE was missing objectType: "NONE"
tag: MODULE_MYREQUESTS was missing objectType: "USER"
This process allows for the resources to review design changes between the current state architecture of r12.6.x to r14.1 using the exact customer data.