Layer7 API Management

Expand all | Collapse all

SFTP LIST/STAT command implementation

  • 1.  SFTP LIST/STAT command implementation

    Posted 08-27-2018 09:32 PM

    We have an external party connecting via SFTP to our Gateway, with SSH certificate credentials. When they connect to the gateway, they use the CD command to change directory to the policy. This works fine, and they are able to PUT a file, and the policy processes it without any issues.

     

    However. In doing so, a number of client side errors are raised: specifically, the SFTP client is trying to use the LIST or STAT command presumably to check the file exists first, or to validate that the file is uploaded successfully – this seems to be a standard function of the “open source” command line “sftp” included with MacOSX / Linux.

     

    Has anyone seen this at all, and do you have to implement anything within your policies to handle the LIST/STAT commands?



  • 2.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 01:09 AM

    Hello,

     

    Have you checked "LIST" and "STAT" check boxes in the "Manage Listen Ports" task - "Listen Port Properties" - "Other Settings" tab?

    Without checking them, these FTP commands are rejected even if they're handled properly in your policy.

     

     

    Cheers,

    Seiji



  • 3.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 01:13 AM

    This is definitely already enabled

     

    Below is a screenshot of the gateway log showing the error.

     

    How do I generate a response for LIST or STAT command? The Return Template Response to Requestor assertion seems to wrap everything with HTTP headers which is useless.



  • 4.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 02:02 AM

    I thought you're trying to get the response from your backend ftp server. In such case, we don't need to care how the response is look like.
    I looked into the response on the API Gateway (checked the response.mainpart context variable) and found it was XML-formatted. In my case, the following Template Response works well:

     

    With the sftp command on my Linux box, I can get the following results:

     

    sftp> ls
    Hello World!
    sftp> ls -l
    -rw-r--r-- 1 -------- -------- 0 Dec 31 16:00 Hello World!
    sftp>

     

    Is it what you'd like to do?

     

    Cheers,

    Seiji



  • 5.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 02:15 AM
      |   view attached

    Can you please direct me to the documentation on how to format this?

    The below is what the policy now looks like.

     

     

    For a “LIST” this produces the following error to the SFTP client calling the policy:

    20180828 16:08:41.572

     

    WARNING

     

    9434

     

    SSH routing error: SSH2 Route Assertion error: Error getting directory listing: 4: Error parsing file info for: /path/of/policy

     

     

    Does the file list need to contain entries for “.” and “..” ?



  • 6.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 02:30 AM

    Would you like to resolve the "SSH routing error"? How your Route via SSH2 assertion is configured?

    I don't think you need to use "Return Template Response to Requester" assertion if you can get the response from your backend sftp server properly.



  • 7.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 05:32 AM

    Hi Ashk,

     

    I have a simple setup where the sftp client calls the gateway which then calls a backend server over route via ssh2.

     

     

    And I am able to run sftp commands without any issue

     



  • 8.  Re: SFTP LIST/STAT command implementation

    Posted 08-28-2018 04:49 PM

    Let me outline the scenario a little better. We're trying to do a protocol translation as follows:

     

    1. User connects via SFTP to a new listener on the API Gateway (With SFTP PUT/LIST/STAT enabled)

    2. User CDs to the relevant directory of the policy

    3. User PUTs a new file to upload

    4. Uploaded file is routed via HTTP to a backend system

     

    We are seeing failures on the client side at step 3 due to the SFTP client trying to issue the LIST / STAT commands. The client error was noted above (note using the SSH2 route assertion as the client for testing only, the SFTP client from command line gets the same issue) - "SSH routing error: SSH2 Route Assertion error: Error getting directory listing: 4: Error parsing file info for: /path/of/policy"

     

    How do I resolve the gateway not returning a valid LIST/STAT response when the gateway is acting as an SFTP server, with a HTTP backend?

     

    It is worth noting that message processing is successful for the PUT command. Only the LIST/STAT commands fail, which results in the "Error parsing file info" type message



  • 9.  Re: SFTP LIST/STAT command implementation

    Posted 08-29-2018 05:01 AM

    I see your scenario clearer but it seems difficult to make the API Gateway thoroughly pretend as a native SFTP server.
    As far as I tried with the sftp command on Linux, the SFTP client was "spoofed" by disabling "STAT" command in the SSH2(SFTP) listener.


    The SFTP client uses "LIST" instead of "STAT" when the SFTP doesn't accept "STAT". Let me try explaining how user operations can be handled along with your scenario.

     

    1. User connects via SFTP to a new listener on the API Gateway (With SFTP PUT and LIST enabled)
    2. User CDs to the relevant directory of the policy
      When the user typed "cd /path/of/policy" in the SFTP client, a LIST command is sent and the expected response is like below:

      <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
      <files>
      <file file="false" lastModified="0" name="policy" permissions="755" size="0"/>
      </files>

      This response means "policy" is a directory (file="false") and the user can change directory to it (permissions="755").
    3. User PUTs a new file to upload
      When the user typed "put sftptest.dat", another LIST command is sent first and the response is same as above. The user has write-permisson to the directory.
      A PUT command follows and the contents of the file are sent to the API Gateway and stored in the request context variable.
    4. Uploaded file is routed via HTTP to a backend system

     

    Users can issue any FTP commands from their clients. I'm not sure if this kind of tricks works for your purpose.

     

    Best regards,
    Seiji