check if a file exists in a ftp server before dowloading it

[Legacy User]

I have a SFTP remote server which I have to inspect every now and then for a particular file to appear before attempt to download it (File Watch funcionality)

 

I can not install a WA Agent on this server, and the only access I have is using an FTP user + password provided.

 

What are your suggestions to solve this?

[Jonathan Calloway]

We had a similar situation.  However, in our case, the developer had to resort to doing this almost exclusively in scripts.  The result is a set of Unix jobs and tasks.  The first job logs in and checks for the existence of the file.  If the file exists, it exits with a return code of 0.  So it completes, and the next set of jobs runs.  They connect to the server and download the file.  If the file does not exist, the initial job exits with a 99, and releases a task that forces the other jobs complete.  We have this process set up to repeat hourly throughout the day using a template.  We're doing this in ESP.  I can add sample code if you like.

[Legacy User]

Hi, jonathan_calloway,

 

I was afraid we'll forced to do so. Could you be so kind to share the sample code you offered?

[Jonathan Calloway]

Hi dennis.naranjo,

 

Here is CLANG from our proc.  I don't have any of the code from the scripts involved:

 

TEMPLATE DSNP270_HOURLY (1,OFFSET)                     
                                                       
  UNIX_JOB ZN470X02.%OFFSET                            
    EXITCODE 0 SUCCESS                                 
    EXITCODE 99 SUCCESS                                
    RUN WORKDAYS                                       
    NORUN TNSTATEHOLIDAY                               
    DELAYSUB 06:00 PLUS %OFFSET HOURS                  
    AGENT <agent>
    USER <user>
    SCRIPTNAME <script>
    ENVAR JOBNAME=%ESPAPJOB                            
    RELEASE ADD(BYPASS_DSNP270) COND(RC(99))           
    RELEASE ADD(ZN470Y01.%OFFSET) COND(RC(0))          
  ENDJOB                                               

 

  UNIX_JOB ZN470Y01.%OFFSET                                      
    RUN WORKDAYS                                                 
    NORUN TNSTATEHOLIDAY                                         
    AGENT <agent>
    USER  <user>
   SCRIPTNAME <user>        
   ARGS %USER1                                                   
   ENVAR JOBNAME=%ESPAPJOB                                       
  ENDJOB                                                         
                                                                 
  JOB BYPASS_DSNP270 LINK PROCESS                                
    RUN WORKDAYS                                                 
    NORUN TNSTATEHOLIDAY                                         
    ESPNOMSG AJ ZN470Y01.%OFFSET BYPASS APPL(%ESPAPPL..%ESPAPGEN)
  ENDJOB                                                         
                                                                 
ENDTEMPL                                                         
 DSNP270_HOURLY 0                                                
 DSNP270_HOURLY 1                                                
 DSNP270_HOURLY 2                                                
 DSNP270_HOURLY 3                                                
 DSNP270_HOURLY 4       

DSNP270_HOURLY 5 
DSNP270_HOURLY 6 
DSNP270_HOURLY 7 
DSNP270_HOURLY 8 
DSNP270_HOURLY 9 
DSNP270_HOURLY 10
DSNP270_HOURLY 11
DSNP270_HOURLY 12
DSNP270_HOURLY 13
DSNP270_HOURLY 14                                          

 

Explanation: If ZN470X02 ends with a return code of 99, then it releases a task (BYPASS_DSNP270) which bypasses ZN470Y01.  This process uses a template that ensure this occurs every hour for 15 hours.  You can make this more complex by using AFTER ADD statements to add other jobs to the stream.

 

Hope this helps!

[Legacy User]

Thank you, Jonathan.

[Chandru.Venkataraman]

cURL could be used.

 

$ curl -u ftpuser:ftppass -O ftp://ftp_server/public/

 

cURL will list all the files and directories under the given ftp URL. One could use the -n or --netrc option to cause cURL to fetch the FTP user name and password from the ~/.netrc file

 

Man page for cURL is here

curl - How To Use 

 

I hope this helps!

 

Cheers,

Chandru

[Rick Romanowski]

Can the SFTP remote server PUT the file on your server with an Agent?

[Nitin Pande]

Hi,

The CA Agent can only act as an SFTP client.  The Agent can download/upload the file from your remote SFTP server.  If you want remote server to login to the host where agent is then you will need SSH/SFTP service.    

The CA Agent can however act as an FTP client and server.  You can start FTP server and use FTPS or FTP over SSL for better security than plain FTP.

 

Thank you,

Nitin Pande

CA Technologies

[Lenn Thompson]

Dennis, 

 

It looks like you got a few suggestions here -- did they help you? Or do you still need help with this?

[Legacy User]

Lenn,

 

'Till now, the CURL option sound better than the restm but I'm currently flooded with work so I'm not able to test it right now...

[Lenn Thompson]

Once you have a chance to test it, please come back and let us know!