Layer7 Access Management

Expand all | Collapse all

SPS Webservice issues

Jump to Best Answer
  • 1.  SPS Webservice issues

    Posted 10-07-2015 11:18 AM

    Good morning all,

     

    I am trying to configure SPS and doing just a basic test and i see below error it is logged in SPS logs (server.log)., i followed the Hubert suggestion on the other links (https://communities.ca.com/message/241818769#241818769) and did everything as said in the other discussions, no luck, please can anybody let me know facing the same issue.

     

     

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,663 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header username: mracamad

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header password: data not shown

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Headeraction: GET

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Headerappid: app2

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Headerresource: /spswebservice/hello

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Printing Headers

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header content-type: application/soap+xml;charset=UTF-8;action="urn:login"

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header connection: Keep-Alive

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header host: stsccppviamwb2.va.neustar.com:8090

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_TRANSACTIONID: 1ac10618-930d0d52-7d75c784-4e10cb05-8739e31c-a9

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header resource: /spswebservice/hello

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header appid: app2

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header user-agent: Apache-HttpClient/4.1.1 (java 1.5)

    [07/Oct/2015:14:57:58-664] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_SDOMAIN: .va.neustar.com

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,664 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header password: data not shown

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_USER:

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_AUTHTYPE: Not Protected

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header username: mracamad

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_UNIVERSALID:

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header content-length: 572

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header action: GET

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header Accept-Encoding: gzip,deflate

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Header SM_USERDN:

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Done Printing Headers

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Exit getFilterCtxData()

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 ERROR [com.ca.soa.services.authaz.webservice.ServiceLogicBackend] - MissingResourceException (Can't find resource for bundle java.util.PropertyResourceBundle, key SM_WSZ_00032_INVALID_VIRUTAL_HOST). Cannot find resource with key [SM_WSZ_00032_INVALID_VIRUTAL_HOST]

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Entered getResponseAttribute

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 DEBUG [com.ca.soa.services.authaz.webservice.ServiceLogic] - Leaving getResponseAttribute



  • 2.  Re: SPS Webservice issues

    Posted 10-08-2015 09:39 PM

    Hi,

     

    The error message complains it cannot find the resource bundle. I check in my testing environment (SPS_INSTALL_PATH\secure-proxy\Tomcat\webapps\CA_AuthAZ\WEB-INF\classes\com\ca\soa\services\authaz\webservice\messages\authazws*) but couldn't find "SM_WSZ_00032_INVALID_VIRUTAL_HOST". That's why the above error thrown. I believe this missing resource need to be fix on the product.

     

    However I have not much idea on why it complains invalid virtual host.

     

    Since it complains virtual host, did you configure any additional virtual host in server.conf? If you are using default virtual host, I expect the hostnames defined and are you using the FQDN to access?

     

    Regards,

    Kar Meng



  • 3.  Re: SPS Webservice issues

    Posted 10-09-2015 11:54 AM

    Hello Karmeng,

     

    Thank you so much for looking into this issue, i checked the resource bundle in my installation path, please find the below path file content, i did notice the error code SM_WSZ_00032 has mentioned in the properties file, but not sure why it complains for missing resource bundle, trust me i tried all possible ways as mentioned below and also my webservice VH is valid host-name and it is able to DNS resolve.

     

    Combination-1:

    DefaultAgentName: webagent1

    AgentName: webagent2,app2

     

    Combination-2:

    DefaultAgentName: webagent1

    AgentName: webagent2,app2

    AgentName: webagent1,webservicevhname

     

    In my case webservicevhname = stsccppvsmdr1.va.neustar.com

     

    Combination-3:

    DefaultAgentName: webagent1

    AgentName: webagent2,app2

    AgentName: webagent2,spsserverhostname

    AgentName: webagent1,spsserverhostname

     

    In my case spsserverhostname = stsccppviamwb2.va.neustar.com:8090

     

    Combination-4:

    DefaultAgentName: webagent1

    AgentName: webagent2,app2

    AgentName: webagent2,spsserverhostname

    AgentName: webagent1,webservicevhname

    AgentName: webagent2,webservicevhname

    AgentName: webagent1,spsserverhostname

     

    vi /opt/app/sps/secure-proxy/Tomcat/webapps/CA_AuthAZ/WEB-INF/classes/com/ca/soa/services/authaz/webservice/messages/authazws_en.properties

     

    #AuthAzJaxWSService.java

    SM_WSZ_00001_PRGM_ERR_WS_CTX_NULL=Program error. Web Services Context is null.

    SM_WSZ_00032_MISSING_SMSESSION_COOKIE=SMSESSION cookie not found in the request

    SM_WSZ_00033_REQUIRE_AGENT_ENFORCEMENT_FAIL=The service is not protected by an Agent as required by the RequireAgentEnforcement setting


    , still don't know what i am missing here, is this a bug or still i asm missing anything here .. ?



  • 4.  Re: SPS Webservice issues

    Posted 10-09-2015 12:11 AM

    Hi Sreekanth

     

    The error messsage is misleading, yes the property cannot be found, but the root cause turned out to be that there was no mapping to a webagent for the webserver you are hitting (stsccppviamwb2.va.neustar.com:8090) or for the application.  (app2)

     

    The easiest way is in the ACO for the webservices is to add a defaultAgentName :

     

        DefaultAgentName=webagent1

       

    You can also enter mapping for both theser entries eg :

     

        AgentName=webagent1,stsccppviamwb2.va.neustar.com:8090

        AgentName=webagent2,app2


    Or possibly :

     

        DefaultAgentName=webagent1

     

        AgentName=webagent2,app2


    You should also check the following ACO settings as well - just in case :

        enableAz=YES

        enableAuth=YES

        RequireAgentEnforcement=No ( but will need to change to Yes for deployment)

    Just a note that these changes need to be made the the ACO for the webservices entry, not the ACO for the normal webagent.


    Cheers - Mark

    PS: I will add the note to the entry on the Communities site as well.




  • 5.  Re: SPS Webservice issues

    Posted 10-09-2015 12:00 PM

    Hello Mark,

     

    Thank you so much for looking into this issue, i got your email as well (thanks to manjari for referring you), i did applied your below mentioned changes and its still the same issue , and also please can you check my comments above replied to Karmeng, i mentioned about the resource bundle path as well.

     

    ---------------------------------------------------------------------------------------------------------------------------------------------

    The easiest way is in the ACO for the webservices is to add a defaultAgentName :

     

        DefaultAgentName=webagent1

       

    You can also enter mapping for both theser entries eg :

     

        AgentName=webagent1,stsccppviamwb2.va.neustar.com:8090

        AgentName=webagent2,app2


    Or possibly :

     

        DefaultAgentName=webagent1

     

        AgentName=webagent2,app2


    You should also check the following ACO settings as well - just in case :

        enableAz=YES

        enableAuth=YES

        RequireAgentEnforcement=No ( but will need to change to Yes for deployment)

    ---------------------------------------------------------------------------------------------------------------------------------------------

    is there anything i am still missing here ?



  • 6.  Re: SPS Webservice issues
    Best Answer

    Posted 10-11-2015 10:12 PM

    Hi Sreekanth

     

    I will send a bit more via the internal case, but the response here does allow me to add some screenshots, so I will post most of the information here :

     

    For this problem I have not had much problem with this setting, other than the need to add that defaultAgentName entry. 


    I'll cover a few things, hopefully at the end it leads to a resolution of the issue :



    1) Firstly from review, it would be good to check this stmt in the last reply :


      Just a note that these changes need to be made the the ACO for the webservices entry, not the ACO for the normal webagent.

     

    There are two ACO's at play here, the main one rests with the defaultagent, and when you activate webservicesagent it has it's own agent (and ACO) :

     

    In the following screenshot's I'll show where it is located, and how to turn logging on for this 2nd agent :

     

    The location for the setup of the 2nd webagent (the webserviceagent one) is here :

    Capture1.PNG

     

    The WebAgent.conf file in that directory (should) specify a different ACO :

     

    Capture2.PNG

     

    Which in my case is ACO: aco.authazws :

    Capture3.PNG

     

    If we look at the policy server, we can see the aco.sps-01 which is the original ACO as specified in the defaultagent directory, and the webservices aco :

     

    Capture4.PNG

     

    The directory mappings need to go into the webserviceagent ACO: aco.authazws

     

    In the following ACO you can see I have a mapping for "AppID" mobile, and a default agent as well  via;

     

    DefaultAgentName=a.spsWSDefault

    AgentName=a.spsWS,mobile

     

    Capture5.PNG

     

    That means on my WS call ,the AppId of "mobile" in this example will map to a.spsWS, and if not then it will return the default agent name of : a.spsWSdefault.

     

    The agent: a.spsWSdefault can then be used to apply protection to determine who can access the REST/SOAP webservices.

     

    And the agent: a.spsWS can be used to determine which users & URL's are protected by the app named : mobile.

     

    Hopefully the answer is as simple as that, but if not follow the 2nd section.

     

     

    2) Enable trace logging on the webservicesagent : 

     

    In the ACO for the servervice : aco.authazws we can also enable normal webagent and trace webagent logging : eg :

     

    Enable wa logging for authazws:

    Capture6.PNG

    Enable watrace logging for authazws

    Capture7.PNG

     

    So, as before these are as well as the logging performed by the defaultagent :  And so in the logs directory we will see two sets of files, the wa/watrace logs produced by the normal (defualt) webagent and these new ones for the authazws eg:  


    In the following you can see the original default sps_wa.log, sps_watrace.log, and the new log files authazws-wa.log and authazws-watrace.log. 

    Capture8.PNG

     

    Now if we look into the new authazws-watrace.log file we will see the various mapping that occurs (and that results in the missing virtual host message) :

     

    Here we can see the Auth/Az for the initial request from the REST page : - as I have host was ws.secure.lab, and since it didnt have an entry in AgentName mapping, it used the defaultagentname of a.spsWSdefault.  (that agent is set with /authazws/... as unprotected, so it allows the access to proceed) - 

     

    Capture9.PNG


    Having determined that the request is authorized, we move onto the next step, where it extracts the REST request and process that - this also is logged as a normal Siteminder request  in the authazws-watrace.log


    And here we can see the subsequent check, where appId: mobile is extracted, and that is mapped via AgentName to the a.spsWS, the resourse is the tail end of the prior URL just /api.


    And in this case the resource /api with agent a.spsWS is configured as protected, so the proceeds with the authentication.

    Capture10.PNG

    So in your case must be happening in either of those two mappings, where either the host name you have or the appId you are using do not map to a valid agent name. 

    And so, if the right ACO is being set, then turning on this detailed trace logging should tell us which one of those mappings is not working correctly (perhaps it is something as simple as a uppercase/lowercase issue).

     

    I do hope that resolves your problem, otherwise as per the support case, we will organize a follow up webex.

     

    Cheers - Mark



  • 7.  Re: SPS Webservice issues

    Posted 10-12-2015 01:16 AM

    Thanks Mark for the detailed explanation. It clearly list out the action plans and hope we can achieve the positive result, if not will need to have a trace log to review.



  • 8.  Re: SPS Webservice issues

    Posted 10-12-2015 02:31 PM

    Hello Mark,

     

    That was detailed info, trust me from beginning of the issue, i have already tried all the options you mentioned it, but still the same issue, no change, find below what i am doing from very beginning.

     

    - two separate aco's i have created,one for webservices (ACO: spswsaco) and other is for regular agent (ACO: spsaco).

     

    All the below relevant options for the webservices aco has been set, find below.

                       - agent and trace log enabled

                        - enableauth=yes and enableaz=yes

                        - requireagentenforcement=no

     

    The below is the snippet in the server.conf for the webservices entry

     

                 <VirtualHost name="WebServicesAgentVirtualHost">

                  hostnames="stsccppvsmdr2.va.neustar.com"

                 <WebAgent>

                 sminitfile="/opt/app/sps/secure-proxy/proxy-engine/conf/webservicesagent/WebAgent.conf"

                 </WebAgent>

                 </VirtualHost>

     

    its still the same issue, no change



  • 9.  Re: SPS Webservice issues

    Posted 10-12-2015 06:29 PM

    Hi Sreekanth

     

    If you have :

    - agent and trace log enabled

     

    then you should have two sets of webagent.log and webagenttrace.log files - one from the normal agent and one from the spswsaco.   For the logs uploaded to the support case, I could only see one set of those logs, that looked like the normal webagent logs - (the spswsaco trace logs, should have the details about checking the host name etc).

     

    Do you mind checking you've got two sets of agent logs, then running the test, then zipping and uploading proxy-engine/conf  and proxy-engine/logs  and upload them to the internal support case - we should in the 2nd trace log find some evidence of processing the REST request, and hopefully a bit more info on why it failed.

     

    Cheers - Mark

    PS: Kar Meng is also available for a webex, if you want a hand - Unfortunately I have limited access but Kar Meng can reach me by phone if needed.



  • 10.  Re: SPS Webservice issues

    Posted 10-12-2015 07:23 PM

    Please be careful, in one of the version (probably 12.52sp1) I was able to get only one of the logging (either normal agent or sps web service ) to work at anytime..

    so if you are not able to get both the logs ..better disable the logging for the component for which you don't need log..



  • 11.  Re: SPS Webservice issues

    Posted 10-26-2015 10:34 AM

    Thanks Ujwal,

     

    In my case there are no logging issues identified. i am using the latest version of SPS.

     

    Thanks

    Sreekanth



  • 12.  Re: SPS Webservice issues

    Posted 10-16-2015 03:48 PM

    Hello Mark / Kar,

     

    Thanks for the help, finally the issue was resolved.

     

    Root Cause is:

     

    Everything was correct in place, except we need to spoof the hosts file for the defined webservice VH in the local machine where the SOAP call is initiated as part of testing.

     

    Thanks

    Sreekanth



  • 13.  Re: SPS Webservice issues

    Posted 10-25-2015 10:04 PM

    Just to elaborate a bit on how this case was solved:

     

    For SPS it is expecting to be handling a number of virtual hosts.  The default virtual host is then setup to process requests via proxy-rules.xml and forward them to the backend :

    <VirtualHost name="default">

                  hostnames="stsccppvsmdr2.va.neustar.com"

                <WebAgent>

                sminitfile="/opt/app/sps/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf"

                </WebAgent>

                </VirtualHost>

    </VirtualHost>


    When adding a new virtual host to handle webservices, it needs a different virtual host name  eg:

     

    <VirtualHost name="WebServicesAgentVirtualHost">

                  hostnames="webservices.va.neustar.com"

                <WebAgent>

                sminitfile="/opt/app/sps/secure-proxy/proxy-engine/conf/webservicesagent/WebAgent.conf"

                </WebAgent>

    </VirtualHost>

     

    For this case both virtual host settings had the same virtual host name.   Fixing them so the WebServices had a different hostname to the default then resolved the issue.

     

    Cheers - Mark



  • 14.  Re: SPS Webservice issues

    Posted 10-26-2015 10:09 AM

    Mark,

     

    its absolutely wrong what you have mentioned the resolution above, please find the exact resolution below, the reason i am correcting the resolution here for  (This is to give proper info for Audience whomsoever face the similar issue).

     

    Initial Prerequisites were already applied to SPS while having/seeing this issue are below:

    - SPS is properly configured

    - SPS default config hostname is different then webservices hostname (this is when webservices service is enabled in SPS).

    - default hostname name was given in my case "stsccppviamwb2.va.neustar.com"

    - WebServicesAgent hostname was given "stscvdvvsmmi02.va.neustar.com" when enabled the services.

    - Checked the logs, all the settings are perfectly configured from SPS side and indeed we see the Auth/AZ service is was successfully started, see my above logs in my previous posts in this thread.

    - And i could see the webservice endpoint URL is successfully initialized and able to see from the browser. URL in my case: http://stsccppviamwb2.va.neustar.com:8090/authazws/auth?wsdl

    FROM HERE (BELOW STEPS) IS THE ISSUE WE NOTICED AND IDENTIFIED.

    - With this endpoint http://stsccppviamwb2.va.neustar.com:8090/authazws/auth i tried to do SOAP call from SOAPUI, with the below simple login snippet:

                        <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:aut="http://www.ca.com/siteminder/authaz/2010/04/15/authaz.xsd">

                          <soap:Header/>

                          <soap:Body>

                                  <aut:login>

                                      <identityContext>

                                              <userName>mracamad</userName>

                                              <password>480676753905</password>

                                              <binaryCreds></binaryCreds>

                                                </identityContext>

                                                <appId>app1</appId>

                                                <resource>/spswebservice/hello</resource>

                                            <action>GET</action>

                                    </aut:login>

                              </soap:Body>

                              </soap:Envelope>

    - After hitting the above SOAP call, we have got Authentication Failed SOAP Response and i have Checked the logs to see what was the root cause, i get the below error from the server.log.

    [07/Oct/2015:14:57:58-665] [INFO] - 2015-10-07 14:57:58,665 ERROR [com.ca.soa.services.authaz.webservice.ServiceLogicBackend] - MissingResourceException (Can't find resource for bundle java.util.PropertyResourceBundle, key SM_WSZ_00032_INVALID_VIRUTAL_HOST). Cannot find resource with key [SM_WSZ_00032_INVALID_VIRUTAL_HOST]

     

    - The we tried to investigate with many changes and combinations on AgentName of the both webservices ACO and default agent ACO, none worked, after a long troubleshoot with CA, we identified the actual and ridiculously simple fix for this issue is:


    ACTUAL RESOLUTION:


    - We need to spoof our local hosts file with "default VH = WebServicesAgent VH" from the computer wherever we are executing the SOAP call and also we need to change the SOAP EndpOint URL also, find the below in my case:

                 

                         10.91.34.250    stscvdvvsmmi02.va.neustar.com

       

     

    10.91.34.250                                -->  This is default VH of SPS "stsccppviamwb2.va.neustar.com"

    stscvdvvsmmi02.va.neustar.com  --> This is WebServicesAgent VH of SPS "stscvdvvsmmi02.va.neustar.com"


    - Need to change the SOAP Endpoint URL from http://stsccppviamwb2.va.neustar.com:8090/authazws/auth

      to: http://stscvdvvsmmi02.va.neustar.com:8090/authazws/auth


    I Hope this clears the issue perfectly with no doubts !!!.


    Adding Extra Note to CA: Please fix the SPS documentation to avoid this issue for the customers with the proper info, otherwise whomsoever face this issue, this thread will temporarily helps them.


    Thanks

    Sreekanth Rachamadugu





  • 15.  Re: SPS Webservice issues

    Posted 10-27-2015 12:29 AM

    Hi Sreekanth

     

    Yes, it is good to make that point fairly clear, when accessing the webservices you need to access the virtual host as named in the WebServicesAgentVirtualHost :

     

    <VirtualHost name="WebServicesAgentVirtualHost">

                  hostnames="webservices.va.neustar.com"

                <WebAgent>

                sminitfile="/opt/app/sps/secure-proxy/proxy-engine/conf/webservicesagent/WebAgent.conf"

                </WebAgent>

    </VirtualHost>


    Giving a working URL for the above as :

        http://webserices.va.neustar.com:8090/authazws/auth?wsd

     

    Using any other virtual host name, including the default host name for the SPS, will be processed through normal SPS proxy-rules.xml file and the request passed onto a backend server.

     

    Usually the setup to add additional virtual hosts will be done via your DNS server, but for testing purposes adding entries into the local hosts file does serve the same purpose.

     

    But glad that it is now resolved !

     

    Cheers - Mark



  • 16.  Re: SPS Webservice issues

    Posted 10-27-2015 11:10 AM

    Thank you so much Mark,for fixing this issue... in the troubleshoot session.