Team,
Existing monitoring tools may be used to monitor when services are running, and some may do a basic query operation to ensure that basic operational functionality is being returned.
For the IM C++ Connector Service, there is a need to build a monitor process that will confirm that the full data path from the IMPS (Provisioning Server), via the IAMCS (JCS Connector Server), then to the embedded CCS Service, is able to query/update an endpoint. This need is driven by scale performance testing with the Identity Suite connector tier and the impact if a CCS service is available, but not responding in a timely manner.
The below design, leverage existing configurations and functionality to allow CA services, partners, customers to enhance their existing monitoring processes with an in-depth query approach. If this query fails, then the JCS/CCS services should be restarted (bounced). If on a remote MS Windows server, this may be as simple as:
net stop im_jcs & net stop im_ccs followed by net start im_jcs & net start im_ccs [Note: On later version of IM solution, the IAMCS(jcs) service is dependency tied to the CCS service, to allow JCS to auto start/stop the CCS service]
CCS Monitoring
Expand upon and leverage the full feature high available functionality at the IM Connector tier via the IAMCS (JCS) to the embedded CCS Service.
Goal(s):
1) Review an in-depth monitoring process for N CCS servers that use round-robin IAMCS process
2) Address a possible gap where the round-robin feature set would provide a possible false positive response if one of the N CCS Services is fine, but the others are non responsive.
3) Challenge Note: Unable to build a in-depth query process directly to the im_ccs service (TCP 20410/20411) as credentials for managed endpoints are stored within the IMPS service/store; and are only passed down to the im_ccs service once upon first bind connection.
Example below has three (3) stacks of the IM Provisioning/Connector Tier to manage endpoint(s).
Step 1: Create new endpoint service ID with minimal access.
Step 2: Create new IMPS endpoint definitions that will be used to monitor each data flow from the IMPS server to the endpoint. In the example below, we have the primary definition for Active Directory, followed by three definitions to be used to monitor the three (3) stacks of the provisioning/connector tier.
Step 3: Use the IM Connector Xpress tool to view the IAMCS(JCS) routing rules.
Step 4: Update the routing rules for the three (3) new ADS definitions.
Step 5: Testing. Use either the IMPS Provisioning GUI to perform a 1st Explore operation; monitor the CCS ADS logs to confirm that ONLY the correct CCS service is being used as defined by the above routing rules. These scripts would be used by the monitor solution, to perform an indepth query. Use these same queries to pull a single attribute, then confirm the value of that attribute returned. The example below query the service account that was created on active directory for each of the three (3) data flows. The only difference is the "name" of the endpoint definition in the scripts (see # marked in RED).
ldapsearch -LLL -H ldaps://192.168.242.158:20390 -D "eTGlobalUserName=etaadmin,eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im,dc=eta" -w CAdemo123 -b "eTADSContainerName=Users,eTADSDirectoryName=ccs1monitor,eTNamespaceName=ActiveDirectory,dc=im,dc=eta" -s base "(&(objectClass=eTADSContainer)(eTADSContainerName=Users))" eTExploreUpdateEtrust
ldapsearch -LLL -H ldaps://192.168.242.158:20390 -D "eTGlobalUserName=etaadmin,eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im,dc=eta" -w CAdemo123 -b "eTADSAccountName=ccs monitor,eTADSContainerName=Users,eTADSDirectoryName=ccs1monitor,eTNamespaceName=ActiveDirectory,dc=im,dc=eta" -s base "(&(objectClass=eTADSAccount)(eTADSAccountName=ccs monitor))" eTExploreUpdateEtrust
Note: If using CLI scripts, use the CCS or IMPS binary of ldapsearch; as this binary has been updated with three new action verbs, e.g. eTExploreUpdateEtrust.
OPTIONAL:
Step 6: If you experience any error message, see below, ensure the following has been addressed.
Cheers,
Alan
Edit: 10/02/2018
A view of the network data path using MS Sysinternal Process Explorer to view the network traffic.