Idea Details

CA SSO : Policy Server trying to SEARCH over a CLOSE_WAIT Connection.

Last activity 05-31-2019 03:51 PM
HubertDennis's profile image
04-08-2016 04:29 PM

In continuation of the following Tech note : CA SSO : Policy Server VS 3rd party components closing Idle Connection. & If needed I could share the Support Case number internally (same questions have been posted on the support case).

 

 

We opted OPTION-2 i.e. LDAP Closing the Connection before Firewall and LDAP sending a FIN to Policy Server.

 

 

What we identified was the following.

 

 

1) LDAP notifies Policy Server it is closing the connection. LDAP sent a packet (actually a formal LDAP response) to the client (Policy Server) indicating it will be terminating the connection.

 

 

2) Policy Server then ACK's this message. LDAP sends FIN, and Policy Server returns a ACK... We would have expected Policy Server to send a FIN and ACK. The concern here is that that the socket remains in a CLOSE_WAIT on the client (Policy Server) side - likely a direct result of client not sending a FIN/ACK (the policy server is still hanging onto the connection and hasn't given the OS the ok to tear it down).

 

 

3) When Policy Server initiates the next SEARCH request (e.g. for a IsAuth call) it tries to reuse the same CLOSE_WAIT connection first. Policy Server see's that connection is in CLOSE_WAIT, hence issues a CLOSE PENDING on its side (believe thats what tears down the connection from Policy Server Side). Then tries a rebind using a new connection.

 

 

[SmDsLdapConnMgr.cpp:1190][LogMessage:ERROR:[sm-Ldap-02230] Error# '81' during search: 'error: Can't contact LDAP server' Search Query = '(cn=AAAAAA)']

[SmDsLdapConnMgr.cpp:1201][CSmDsLdapConn::SearchExts][LDAP search of (cn=AAAAAA) took 0 seconds and 8237 microseconds]

[SmDsLdapFunctionImpl.cpp:3155][CSmDsLdapProvider::SearchExts][Ldap Search failed, ErrorMsg is Can't contact LDAP server]

 

 

The Enhancement Request is to look at the design and see if there are opportunities for improving this.

 

 

As we know Policy Server maintains 3 connections DIR, USR and PING. We assume that the above action of a Firewall OR LDAP sending a RST, puts the DIR and/or USR in CLOSE_WAIT. However PING connection is actively returning success. Could we do something from the Policy Server side (within DIR and/or USR connection, similar to introducing a PING within DIR and/or USR) that allows Policy Server to send a FIN/ACK and close down the connection immediately, rather than Policy Server send only an ACK and keep the connection on CLOSE WAIT. This would eliminate the additional call over a CLOSE_WAIT connection during a actual SEARCH request and Policy Server starts off with a fresh connection.

 

 

 

 

 

 

Log Snippets :

 

 

[SmDsDir.cpp:66][CSmDsDir::CSmDsDir][Start of call InitDir.][About to initialize directory, Oid='0e-000231d3-2718-16f4-83ec-693a0a0a909d', Name='LDAP-DIR'][

[SmDsLdapProvider.cpp:1424][CSmDsLdapProvider::InitDir][Using LDAP server bank #1][ldapserver1.ca.com][1389]

[SmDsLdapFunctionImpl.cpp:1952][ImproveLDAPConnection][Enter ImproveLDAPConnection]

[ImproveLDAPConnection][Exit ImproveLDAPConnection]

[SmDsDir.cpp:81][CSmDsDir::CSmDsDir][Return from call InitDir.]

[SmDsObj.cpp:94][CSmDsObj::IsValid][Start of call IsValid.]

[SmDsObj.cpp:96][CSmDsObj::IsValid][Return from call IsValid.][true]

[SmDsDir.cpp:1080][CSmDsDir::GetDirectoryVersionInfo][Enter function CSmDsDir::GetDirectoryVersionInfo]

[SmDsDir.cpp:1082][CSmDsDir::GetDirectoryVersionInfo][Leave function CSmDsDir::GetDirectoryVersionInfo][18][00:00:00.000023]

[SmObjCache.cpp:824][CSmObjCache::Fetch][Retrieve an object from the object cache.]

[SmObjStore.cpp:3363][IsADEnhanced][Global Preferences:]

[SmDsDir.cpp:194][CSmDsDir::GetConnectionObject][Start of call GetDirConnectionObject.][Get dir connection object.]

[SmDsLdapFunctionImpl.cpp:1952][ImproveLDAPConnection][Enter ImproveLDAPConnection]

[ImproveLDAPConnection][Exit ImproveLDAPConnection]

[SmDsLdapFunctionImpl.cpp:1740][GetConHandle][Enter GetConHandle]

[SmDsLdapFunctionImpl.cpp:1741][GetConHandle][host=ldapserver1.ca.com, port=1389, secure=0, automatic=1, search=1]

[GetConHandle][Exit GetConHandle]

[SmDsDir.cpp:196][CSmDsDir::GetConnectionObject][Return from call GetDirConnectionObject.][Ok]

[SmDsDir.cpp:202][CSmDsDir::GetRawHandle][Start of call GetDirRawHandle.][Get dir raw handle]

[SmDsDir.cpp:204][CSmDsDir::GetRawHandle][Return from call GetDirRawHandle.][Ok]

[SmAuthUser.cpp:5117][CSmAuthUser::Authenticate][Enter function CSmAuthUser::Authenticate]

[SmAuthHtml.cpp:279][SmAuthenticate][Enter function SmAuthenticate]

[SmAuthHtml.cpp:284][SmAuthenticate][Leave function SmAuthenticate][6][00:00:00.000181]

[SmAuthUser.cpp:1695][CSmAuthUser::SavePasswordState][Enter function CSmAuthUser::SavePasswordState]

[SmAuthUser.cpp:1697][CSmAuthUser::SavePasswordState][Leave function CSmAuthUser::SavePasswordState][false][00:00:00.000038]

[SmAuthUser.cpp:5385][CSmAuthUser::Authenticate][Leave function CSmAuthUser::Authenticate][6][00:00:00.002154]

[SmDsDir.cpp:272][CSmDsDir::IsValidUsername][Start of call IsValidUsername.][User ='AAAAAA']

[SmDsDir.cpp:274][CSmDsDir::IsValidUsername][Return from call IsValidUsername.][true]

[SmDsDir.cpp:425][CSmDsDir::Search][Start of call Search.][Advanced search, Root='dc=ca,dc=com',Filter='(cn=AAAAAA)']

[SmDsAliases.cpp:328][CSmDsAliases::GetSmDsAliases][Enter function CSmDsAliases::GetSmDsAliases]

[SmDsAliases.cpp:377][CSmDsAliases::GetSmDsAliases][Leave function CSmDsAliases::GetSmDsAliases][true][00:00:00.000757]

[SmDsAliases.cpp:428][CSmDsAliases::GetAttributeMapping][Enter function CSmDsAliases::GetAttributeMapping]

[SmDsAliases.cpp:435][CSmDsAliases::GetAttributeMapping][Leave function CSmDsAliases::GetAttributeMapping][false][00:00:00.000169]

[SmDsLdapFunctionImpl.cpp:1952][ImproveLDAPConnection][Enter ImproveLDAPConnection]

[ImproveLDAPConnection][Exit ImproveLDAPConnection]

[SmDsLdapProvider.cpp:1783][CSmDsLdapProvider::SearchImpl][search filter is : (cn=AAAAAA)]

[SmDsLdapFunctionImpl.cpp:3127][SearchExts][Enter SearchExts]

[SmDsLdapConnMgr.cpp:1190][LogMessage:ERROR:[sm-Ldap-02230] Error# '81' during search: 'error: Can't contact LDAP server' Search Query = '(cn=AAAAAA)']

[SmDsLdapConnMgr.cpp:1201][CSmDsLdapConn::SearchExts][LDAP search of (cn=AAAAAA) took 0 seconds and 8237 microseconds]

[SmDsLdapFunctionImpl.cpp:3155][CSmDsLdapProvider::SearchExts][Ldap Search failed, ErrorMsg is Can't contact LDAP server]

[SmDsLdapFunctionImpl.cpp:2013][RebindServer][Enter RebindServer]

[SmDsLdapFunctionImpl.cpp:2044][RebindServer][server OK]

[SmDsLdapConnMgr.cpp:501][CSmDsLdapConnMgr::AddDeadHandleList][Marked dir connection (seq: 3) ldapserver1.ca.com:1389 as Close Pending]

[SmDsLdapConnMgr.cpp:501][CSmDsLdapConnMgr::AddDeadHandleList][Marked dir connection (seq: 1) ldapserver1.ca.com:1389 as Close Pending]

[SmDsLdapConnMgr.cpp:501][CSmDsLdapConnMgr::AddDeadHandleList][Marked user connection (seq: 2) ldapserver1.ca.com:1389 as Close Pending]

[SmDsLdapFunctionImpl.cpp:3002][LdapBind][Enter LdapBind]

[SmDsLdapFunctionImpl.cpp:3053][LdapBind][err=ErrCode: 0]

[LdapBind][Exit LdapBind]

[SmDsLdapConnMgr.cpp:895][IsAvailable][Successful V3 Bind server][ldapserver1.ca.com][1389]

[SmDsLdapConnMgr.cpp:628][PingServer][LDAP Server Ping Successful][ldapserver1.ca.com][1389]

[SmDsLdapFunctionImpl.cpp:2110][CSmDsLdapProvider::RebindServer][Reconnect to server 'ldapserver1.ca.com:1389' as it's previous connections are closed and it is available for connecting now]

[SmDsLdapFunctionImpl.cpp:2132][RebindServer][GetData(pDs)->m_szServer=ldapserver1.ca.com:1389, nRebindTimestamp=1459965750, szBestServer=, nBestServerTimestamp=0][

[SmDsLdapFunctionImpl.cpp:2169][RebindServer][ to rebind to same server as current connection is closed by server]

[SmDsLdapFunctionImpl.cpp:2170][RebindServer][szBestServer=ldapserver1.ca.com:1389]

[SmDsLdapFunctionImpl.cpp:2203][CSmDsLdapProvider::RebindServer][Rebind attempt on 'dir' connection to best LDAP server 'ldapserver1.ca.com:1389']

[SmDsLdapFunctionImpl.cpp:2408][BindServer][Enter BindServer]

[SmDsLdapFunctionImpl.cpp:2409][BindServer][szServer=ldapserver1.ca.com:1389, szBindDN=cn=adminr1252,ou=admin,dc=ca,dc=com, nSearchResults=0, nSearchTimeout=90]

[SmDsLdapFunctionImpl.cpp:2410][BindServer][bRequireCredentials=1, bSSL=0, bAutomatic=0]

[SmDsLdapFunctionImpl.cpp:2564][BindServer][(Bind) For this handle LDAP automatic referrals are disabled.]

[SmDsLdapFunctionImpl.cpp:3002][LdapBind][Enter LdapBind]

[SmDsLdapFunctionImpl.cpp:3053][LdapBind][err=ErrCode: 0]

[LdapBind][Exit LdapBind]

[SmDsLdapFunctionImpl.cpp:2733][BindServer][szBindDN << nSearchResults << nSearchTimeout (cn=adminr1252,ou=admin,dc=ca,dc=com, 0, 90)]

[BindServer][Exit BindServer]

[SmDsLdapFunctionImpl.cpp:2256][RebindServer][szBestServer=ldapserver1.ca.com:1389]

[SmDsLdapFunctionImpl.cpp:2294][RebindServer][pDsLdap->m_nCurr+1=1, szBestServer=ldapserver1.ca.com:1389]

[SmDsLdapFunctionImpl.cpp:2408][BindServer][Enter BindServer]

[SmDsLdapFunctionImpl.cpp:2409][BindServer][szServer=ldapserver1.ca.com:1389, szBindDN=cn=adminr1252,ou=admin,dc=ca,dc=com, nSearchResults=0, nSearchTimeout=90]

[SmDsLdapFunctionImpl.cpp:2410][BindServer][bRequireCredentials=1, bSSL=0, bAutomatic=0]

[SmDsLdapFunctionImpl.cpp:2564][BindServer][(Bind) For this handle LDAP automatic referrals are disabled.]

[SmDsLdapFunctionImpl.cpp:3002][LdapBind][Enter LdapBind]

[SmDsLdapFunctionImpl.cpp:3053][LdapBind][err=ErrCode: 0]

[LdapBind][Exit LdapBind]

[SmDsLdapFunctionImpl.cpp:2733][BindServer][szBindDN << nSearchResults << nSearchTimeout (cn=adminr1252,ou=admin,dc=ca,dc=com, 0, 90)]

[BindServer][Exit BindServer]

[SmDsLdapFunctionImpl.cpp:2322][RebindServer][szBestServer=ldapserver1.ca.com:1389]

[RebindServer][Exit RebindServer]

[SmDsLdapConnMgr.cpp:1201][CSmDsLdapConn::SearchExts][LDAP search of (cn=AAAAAA) took 0 seconds and 2981 microseconds]

[SearchExts][Exit SearchExts]

[SmDsLdapProvider.cpp:2311][CSmDsLdapProvider::Search][Ldap Search callout succeeds.][(Search) Base: 'dc=ca,dc=com', Filter: '(cn=AAAAAA)'. Status: 1 entries]

 

 

 

Regards

 

Hubert


Comments

07-29-2016 03:45 PM

Thank you for your contribution of an enhancement idea to the CA Community. CA is continually working to improve its software and services to best meet the needs of its customers. Your input is vital to that effort. The CA Single Sign-On Product Management team is reviewing your enhancement suggestion. The Community will continue to be able to vote on this enhancement idea.

04-19-2016 12:21 PM

Sounds good, in that case to tag only CA SSO. Would do so.

04-19-2016 12:18 PM

Hubert, CA SSO and CA Secure Cloud are two separate products. CA Secure Cloud reuses the source code from CA SSO, but it does not bundle CA SSO binaries.

 

In any case, each idea submitted here is ultimately triaged by the corresponding product team. By tagging the idea for two products, you are asking two product teams to evaluate and decide on the next steps. If these two teams decide to proceed differently, there is no way to track different dispositions per product for the same idea. I therefore suggest you only tag your ideas for just one product, and if you think they also apply to other products, then open same ideas for those products as well.

 

It would be helpful for the product team evaluating your idea submissions to fully understand the scope of applicability and any specific use cases that apply to the corresponding product.

 

Is this idea for CA SSO or CA Secure Cloud?

 

Thank you

04-19-2016 12:01 PM

The reason I tagged this for CA Secure Cloud, is because we ship CA SSO as a bundled component within CA Secure Cloud.

04-15-2016 03:35 PM

I see this idea is tagged for both CA SSO and CA Secure Cloud. Is there anything unique in the use case that makes it applicable to CA Secure Cloud, or is it essentially the same for both products?