FTPConnection object giving error when using Remote Command

[Derrick Williams]

We are using Automic v9 and the FTPConnection object, and using the Remote Command

mv /x/y/*.csv /x

and we are getting the following error

String index out of range: -1
com.uc4.ftpjob.DataTransferException: String index out of range: -1

Any help would be appreciated, as to why this error is occurring

[Anon Anon]

are you sure that the FTP server supports the mv commnd?

does it work manually via console with teh same user from CONN object?

[Derrick Williams]

Hi Wolfgang, thank you for replying.

Yes, the command works manually via the MS-DOS window when I connect to the site using the same logon credentials

[Anon Anon]

whats the complete err message in the job report?

...and the complete remote command script...

[Derrick Williams]

Hi

I hope this is what you mean

The Remote Command is

 

mv /tobfm/InProgress/*.csv /tobfm

The complete Job Report is below

2016-10-11 10:19:46            host name = LON0248, remote user identification  = williad
2016-10-11 10:19:46            connecting on port : 22
2016-10-11 10:19:46            Using hosts file = C:\Users\z_automic_agent_dev/.ssh/known_hosts
2016-10-11 10:19:46            EFT Server 7.0.3.29
2016-10-11 10:19:46            connection completed
2016-10-11 10:19:46            Skip remote directory check false
2016-10-11 10:19:46            sftp channel opened
2016-10-11 10:19:46           
2016-10-11 10:19:46            Copying from Local File System to MID.CONN.FTAGENT.FTPCONNECTION.SFTP.BRS
2016-10-11 10:19:46            copy D:\schroders\dv1\eai\export\brs\*.csv to /tobfm/InProgress
2016-10-11 10:19:46            host name = LON0248, remote user identification  = williad
2016-10-11 10:19:46            connecting on port : 22
2016-10-11 10:19:46            Using hosts file = C:\Users\z_automic_agent_dev/.ssh/known_hosts
2016-10-11 10:19:46            connection completed
2016-10-11 10:19:46            Skip remote directory check false
2016-10-11 10:19:46            sftp channel opened
2016-10-11 10:19:46            copied: D:\schroders\dv1\eai\export\brs\nav_asia1_20140501.csv  - 9 bytes sent in 0.00 Seconds 3.0 Kbytes/sec.
2016-10-11 10:19:46            1 files transferred - 9 bytes sent in 0.08 Kbytes/sec
2016-10-11 10:19:46            Executing command: mv /tobfm/InProgress/*.csv /tobfm
2016-10-11 10:19:46            on MID.CONN.FTAGENT.FTPCONNECTION.SFTP.BRS
String index out of range: -1
com.uc4.ftpjob.DataTransferException: String index out of range: -1
 at com.uc4.ftpjob.commands.impl.RemoteCommand.execute(RemoteCommand.java:50)
 at com.uc4.ftpjob.commands.CommandProcessor.execCommand(CommandProcessor.java:257)
 at com.uc4.ftpjob.commands.CommandProcessor.processCommands(CommandProcessor.java:162)
 at com.uc4.ftpjob.commands.CommandProcessor$processCommands.call(Unknown Source)
 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
 at FtpJob.runJob(FtpAgent.groovy:304)
 at com.uc4.extensibility.types.CITAgent.dispatchJob(CITAgent.java:258)
 at com.uc4.ex.cit.CITJob.execute(CITJob.java:176)
 at com.uc4.ex.Job.run(Job.java:464)
 at com.uc4.ex.ThreadPool$RunnableDecorator.run(ThreadPool.java:53)
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
 at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -1
 at java.lang.String.checkBounds(Unknown Source)
 at java.lang.String.<init>(Unknown Source)
 at com.uc4.ftpjob.commands.SSHClient.getServerResponse(SSHClient.java:290)
 at com.uc4.ftpjob.commands.SSHClient.sendShell(SSHClient.java:236)
 at com.uc4.ftpjob.commands.impl.RemoteCommand.execute(RemoteCommand.java:33)
 ... 17 more

[Anon Anon]

and the remote command script in the job please...

[Derrick Williams]

https://us.v-cdn.net/5019921/uploads/editor/r9/pfw5dwzzcut5.png" width="1237">there is no remote command script. The command

mv /tobfm/InProgress/*.csv /tobfm

is entered in the JOBS.FTPAGENT.FTPJOB object. There is a list of commands (Copy Files/Make Directory/Delete/.../Echo/Remote Command) that you drag into the Command Sequence window. I have dragged the Remote Command into the Command Sequence window, and in the Remote Command dialog box I have entered the above command in the Command text box.

[Derrick Williams]

I have created used a Free Form Command with  the command

   mv /tobfm/InProgress/&FILENAME# /tobfm/&FILENAME#

Where i am now using the FTP Object in a FOREACH JOBP reading the files one at a time, and this has seemed to work.

Thank you all for your inputs

[KesavanElangovan]

We are receiving the exact error when trying to rename multiple files once they are copied to the external vendor's FTP server. We are on the latest version of Automic and the RA FTPConnection object, and using the Remote Command. 

Is this a limitation of the product? SFTP renaming multiple files? 

[Carsten Schmitz]

Is this a limitation of the product? SFTP renaming multiple files? 

Depends on what exactly you try to do, on what platform and with which client and server for sftp, but yes, this is a likely restriction.

On UNIX you can't do stuff like "mv *.txt *.dat" in most shells if that's what you're trying, and sftp essentially wraps the shell:

$ for i in $( seq 1 10 ) ; do touch $i.txt ; done
$ mv *.txt *.dat
mv: target ‘*.dat’ is not a directory

So yes, you probably can't rename multiple files with wildcards at once with sftp. You may possibly need to construct a loop over your file list and generate individual rename commands.

[Anon Anon]

What about renaming your files befor you transfer them?

[KesavanElangovan]

FrankMuffke the vendor we are transferring the files to requires to drop the files with a tmp extension and after the transfer, rename the files. This is to avoid their job getting triggered during mid-transfer. We do have a workaround to transfer the renamed files to a staging directory and copying into the root folder for their job to pick it up.  

Carsten_Schmitz_7883
"So yes, you probably can't rename multiple files with wildcards at once with sftp. You may possibly need to construct a loop over your file list and generate individual rename commands."
 -- We tried the below commands. 

rename *.txt *.tmp

( echo cd /folder/; echo get \*.tmp ) ls -1 | sed -ne '1icd /folder/' -e 's/^.*\.tmp$/rename & &.txt/p' 

( echo cd /folder/; ls -1 *.tmp | awk '{print "rename", $0, $0 ".txt";' } )

[Anon Anon]

Another option could be creating a list of files to transfer via script (e.g. prep_process_filename) and start one FTP job per file.
you could do this per activate_uc_object or use an Foreach Workflow.

[KesavanElangovan]

FrankMuffke  - We briefly discussed that idea, and the idea of using PREP_PROCESS_REPORT to get the transferred file names and passing them to a JOBS to rename. 

But the files generated for a single transfer could be anywhere between 1 and, well no upper limit but very well could go to 2000-3000. So additional job would not be a best solution, in this scenario. 

[Anon Anon]

Oh yeah - in this case absolutely correct :-)

[Carsten Schmitz]

One of our upcoming "fun" tasks will be rewriting a job that fetches a dozen files from one of 800 servers each, so a few thousand files in total, and spawns one Automic job for each one.

So yes, I strongly confirm that for huge numbers of files, making one Automic job for each file is NOT a good idea. Avoid it whenever you can and use OS methods instead, is my advice, too.