Question about SFTP

[KATHY Mullin]

We have never used the SFTP option in ESP and are wanting to test it. I have some questions and also would appreciate ideas, advise, tricks and trips, etc. that anyone may have.  Thanks in advance!

 

1) Does the module just need to commented out of the init parm and loaded to use or is a plug-in needed at an additional cost?

2) Does using SFTP hog system resources? Increase MIPS, Network band, etc.?

3) Are there any additional costs that would be incurred and an eta of what/costs would be needed?

4) What do you feel it does well/not well?

5) Is it simply a setup such as?

                       FTP_JOB NTFTP                              
                           RUN MON                                          
                           AGENT WINDOWSERVER                            
                           USER TEST\sa-esp                             
                           SERVERADDR 1xx.xx.xx                               
                           SERVERPORT 6665                                  
                           TRANSFERDIRECTION UPLOAD                         
                           FTPFORMAT A                                      
                           REMOTENAME 'ESP.P000.JOB(+1)'                
                           LOCALNAME 'D:ESP\ESP\new.txt' 
                           AFTER ADD(ESPRCSR2)                              
                   ENDJOB              

         6) Any ideas, advise, tricks and trips, etc. that anyone may have?             

[powdo03]

Hi, 

A couple of notes. Let me know if you have questions. 

There is no additional license/cost for SFTP/FTP

 

You mentioned SFTP but the job definition is an FTP job. 

When the agent is being installed choose to add the FTP plugin. Depending on what options are picked it will add line(s) to the agentparm.txt file. 

For the base ftp/sftp process it adds the line below. 

plugins.start_internal_6=ftp

 

Below is my example for an SFTP job. Let me know if this isn't what you are looking for. 

SFTP_JOB SFTPUNIX
   RUN ANY
   AGENT AGENTU_SFTP
   USER espadmin
   SERVERADDR powdo03-U107996
   TRANSFERDIRECTION U
   LOCALNAME /opt/data/a.txt
   REMOTEDIR /opt/data
   REMOTENAME a2.txt
ENDJOB

 

I can give you a call if you would like to discuss this more. 

[KATHY Mullin]

thank you so much for your response...I just sent an example to an FTP that I had found and didn't really pay any attention to FTP/SFTP but I appreciate the reminder. I may not have noticed that.  

 

I will seeing about configuring this and testing. If I have any more questions maybe we can connect. Thanks!

[Jonathan Calloway]

Rick,

Would you or anyone else in the Community that runs FTP jobs be willing to provide any output (spool file) for an one of these jobs?  I would like to see if there is any output, and if so, what it looks like.

[Rick Romanowski]

ESP PROC

FTP_JOB FTPJOB                                                   
  AGENT <Agent Name>                                                  
  USER <ID>                                                   
  SERVERADDR <Server>                                 
  SERVERPORT 21                                                  
  TRANSFERDIRECTION U                                            
  TRANSFERCODETYPE A                                             
  REMOTEFILENAME '''<Remote File>'''          
  LOCALFILENAME <Local File>                        
  RUN      ANYDAY                                                
ENDJOB                                                           

 

WSF CSF Extension to Display Job Output

Attempting to connect to <Server>:21                               
Connection established to <Server>:21                              
USER <ID>                                                                  
331 Send password please.                                                     
PASS (hidden)                                                                 
230 <ID> is logged on.  Working directory is "<HLQ>.".                       
SYST                                                                          
215 MVS is the operating system of this server. FTP Server is running on z/OS.
Uploading ASCII <Local File>--> '<Remote File>'
PORT 153,2,252,252,138,99                                                     
200 Port request OK.                                                          
TYPE A                                                                        
200 Representation type is Ascii NonPrint                                     
STOR '<Remote File>'                                       
125 Storing data set <Remote File>                         
Uploaded 1430 bytes in 0.18 seconds, 77.58 Kbytes/sec.                        
250:Transfer completed successfully.                                          
Upload successful                                                             
QUIT                                                                          
221 Quit command received. Goodbye.                                           
Logged out                                                                    

[Jonathan Calloway]

Did this output come from a CA supplied CSF extension or one that you wrote?

[Rick Romanowski]

From the CA supplied CSF extension for an FTP_JOB

Note that the FTP_JOB is more verbose than the SFTP_JOB 

[powdo03]

My output looks like this

----------------------------------------------------------------
Output of messages for workload object SFTPUNIX/DPTST022.5/MAIN
Start date Wed Mar 07 14:21:54 CST 2018
----------------------------------------------------------------
Uploaded 12 bytes in 0.6 seconds, 1.95 Kbytes/sec.

[powdo03]

Mine was for a SFTP_JOB type.  I think Rick's is for a FTP_JOB type. 

[Rick Romanowski]

Yes, that is correct

The FTP_JOB output is much more verbose, is there any way to get that with an SFTP_JOB?

Might help during troubleshooting when there is an issue.

[powdo03]

There is no method to make the SFTP_JOB verbose. I looked briefly to see if the sshd_config could be modified on the server but I was not successful in making it verbose. 

[Jonathan Calloway]

Would anyone be willing to send an agentparm.txt where the agent is configured to use FTP?  You can obfuscate any names / other info to protect the innocent!

 

Thanks!

 

Jonathan

[powdo03]

Hi, 

The only line that needs to be added is the one below. The _6 should be changed to the next available number for the plugins. This depends on what plugins are installed. Add this line after the last plugin. 

plugins.start_internal_6=ftp

[Jonathan Calloway]

We've done that, and set up SSL following the directions on DocOps.  However, when we attempt to run an FTP job, we're getting "Plugin Not Available".

[Steve Carrobis]

If you do an autorep -p -m machdef

What comes back ?

You should see jar files in jars and also in ext

Sounds like you may need a ticket with CA.

 

Good luck

 

 

Steve C.

 

 

Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.

 

Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.

[Denise Cronin]

Hi Don,

I found this thread because I am looking for guidance on setting up an SFTP job that will download a file from the mainframe where ESPm is running to a Windows server. The agent I will be using is running on the Windows server. Can you share a simple MF --> Windows SFTP job?

Thanks,
Denise Cronin​

[Lucy Zhang]

Hi Dennis,

SFTP_JOB can be used to for mainframe uss files, and you can use FTP_JOB for other type of mainframe files.

Hope this helps,

Lucy

[Michael Bieganski]

Hi All,
just to be clear, assuming the ftp plugin was not added during the original installation,
simply adding the line: plugins.start_internal_6=ftp
and bouncing the agent will allow one to do both ftp and sftp?
Assuming the USER  used in the SFTP_JOB
has its password stored with ESPM on the mainframe, nothing else  would be required
like keys or certificates?
Is there a module/wobdef one has to enable or load in one's initparms or via a pagemode command?

thanks!

[Don Powell]

Hi, 
You are correct. Adding the additional line will enable FTP and SFTP in the agent.  It is important to get the number of the plugin correct (_6 in the example). I needs to be the next higher number for the plugins. 

The modules below will need to be added to the WOBDEF list in the ESP init parms or added dynamically.
CYBESOFT  - FTP 
CYBESOSF - SFTP 

Let us know if thee are any questions. 
Don
2¢

[Michael Bieganski]

cool...thanks so much Don

[Don Powell]

Hi Denise, 
Below is an example that downloads a file from the mainframe to the windows server.  I used SSL but that is not critical. 
Let me know if you have any questions.... 

FTP_JOB FTPSSLDN.DWNLOAD
AGENT DPW14_FTP
USER MFUSER 
SERVERADDR MF_address.com 
SERVERPORT 21
TRANSFERCODETYPE A SSL(YES)
TRANSFERDIRECTION DOWNLOAD
FTPFORMAT A
REMOTENAME '''MF.FILE.HERE'''
LOCALNAME C:\TEMP\SFTP99.txt
RUN DAILY
ENDJOB