Layer 7 API Management

Expand all | Collapse all

Unable to do protocol transformation between sftp and http

Jump to Best Answer
  • 1.  Unable to do protocol transformation between sftp and http

    Posted 01-18-2018 10:16 AM
      |   view attached

    Hi al

     

    We're trying to do a protocol transformation between sftp and http on the API gateway (version 9.1). The goal is to receive an inbound attachment via SFTP and forward the message to a web service. 

     

    I've found some documentation on how to use the SFTP  within the API Gateway but did not found an answer yet. 

    Working with SCP/SFTP Messages - CA API Gateway - 9.1 - CA Technologies Documentation 

    Accepting requests via SSH2 on the Layer 7 Gateway 

     

    In the attached screenshot you'll see I'm able to connect to the API Gateway using a sftp client (configured via SSH2 Inbound listen port). When sending a document ('test.json' in my example), the API Gateway service is correctly invoked (see service debugger) but I'm not able to extract the attachment from it and route it to the web service.

     

    Anyone has idee how this should be done?

    Big thanks in advance!
    Samuel Vandecasteele



  • 2.  Re: Unable to do protocol transformation between sftp and http

    Posted 01-18-2018 02:08 PM

    Hi Samuel is this 9.1?

    I have done similar and seen similar in 9.2 

    I am uploading a sample of a policy that I have made in house that I know works. Which is a get not a put. 

     

    And I have seen the reverse done (your usecase)  with a message-received policy where it sftp was routed to a http back-end call.

     

    If you use a non json file is it working? Are you Auditing the request.mainpart? I see the policy debugger but it might be of value to audit the response.mainpart.


    Do you see multiple executions? I know at connection it tends to run through a message received policy multiple times. 

    But I would wonder 

    1) if its specific to the file type?

    2) Is it specific to the listen port settings for sftp listen port is it streaming etc..


    If you want to open a support case we can likely help you as well. 



  • 3.  Re: Unable to do protocol transformation between sftp and http

    Posted 01-19-2018 04:16 AM

    Hi Charles

     

    Thanks for your response! yes we're running 9.1 and i'm auditing the request.mainpart (added now in separate audit) but it's still empty (also in audit logs).

     

    I see one execution for each sftp command. All 3 scenarios (A, B and C) below invoke the service debugger. See screenshot below:

    A: From client execute an 'ls' which should list the remote directory. This fails as there is no remote sftp exposed directory.

    B & C: Trying to upload a file to the Gateway. Both fail with same error 'Couldn't get handle: Failure'.

     

    My conclusion would be that this is not possible as the exposed policy on the gateway does not speak the sftp protocol. And the gateway can only be used to proxy sftp traffic, so there is always a separate sftp server needed. 

    I hope you can tell me that my conclusion is wrong  

     

    In your sample policy, you connect to an external sftp server and the gateway is playing the role of an sftp client.



  • 4.  Re: Unable to do protocol transformation between sftp and http
    Best Answer

    Posted 01-19-2018 02:45 PM

    Hi Samuel,

     

    I just ran a quick test and this  if working fine for me I can upload the file via ftp and I can see the contents of the file.

     

     try adding request.mainpart.size to your service debugger output.  If it has a size but the content are not showing it might be a issue with the ContentType you specified in the listener port causing this. you can also try changing the contentType in the listen port to be text/plain and see if you see the mainpart then. 

     

    Barry



  • 5.  Re: Unable to do protocol transformation between sftp and http

    Posted 02-21-2018 07:31 AM

    Hi all


    Thanks for the feedback. We finally managed to have a working setup for this use case. Also a big thanks for the input by CA Support!

     

    The API Gateway has an sftp server on OS level to which we'll route the incoming sftp commands. When it's a PUT command we'll put the file in the /tmp directory of the gateway and retrieve it back within the same thread. 
    See image below for the assertion:

    Example connecting to this service via SFTP:

    Notes:

    • You'll see 3 entries in the audit log. One to change to "/tmp" directory on the gateway. And 2 for uploading the file itself. 
    • The route via SFTP assertion are configured to use the 'ssgconfig' user when connecting to localhost port 22.
    • The credentials provided by the client (user admin in my example) is validated in the first section of my assertion
    • An SSH Listen port should be configured

     

     

    Kind regards
    Samuel