only 2 files processed in file transfer

[MateuszKowalski613123]

Hello,

I'm have an issue during transferring files, job run fine but transfer only 2 files in one run, error message is:

Failure

com.uc4.ftpjob.DataTransferException: Failure

     at com.uc4.ftpjob.commands.SFtpCommandImpl.runCmd(SFtpCommandImpl.java:105)

     at com.uc4.ftpjob.commands.SFtpCommandImpl.delete(SFtpCommandImpl.java:125)

     at com.uc4.ftpjob.commands.FtpFacadeCommandImpl.delete(FtpFacadeCommandImpl.java:348)

     at com.uc4.ftpjob.commands.impl.CopyCommand.execute(CopyCommand.java:668)

     at com.uc4.ftpjob.commands.ThreadPoolJobExecutor$ThreadRunner.run(ThreadPoolJobExecutor.java:86)

     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

     at java.util.concurrent.FutureTask.run(FutureTask.java:266)

     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

     at java.lang.Thread.run(Thread.java:745)

Caused by: 2: Failure

     at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2579)

     at com.jcraft.jsch.ChannelSftp.rm(ChannelSftp.java:1780)

     at com.uc4.ftpjob.commands.SFTPHelperDelegate$30.run(SFTPHelperDelegate.java:421)

     at com.uc4.ftpjob.commands.SFTPHelperDelegate$1.run(SFTPHelperDelegate.java:64)

     ... 1 more

2018-01-23 11:18:42             2 files transferred, 75 not processed - 113734749 bytes sent in 1549.89 Kbytes/sec

I hav ealso another FT which transfer files between same hosts and same path, only file name is different, file name in flow that run fine is x/***/xxxx/yy_*match and from FT which transfer only 2 files is x/***/xxxx/yy_*nomatch.

Any idea what can be wrong ?

[Carsten Schmitz]

I notice the third line in the exception is something about "delete".

Did you, by any chance, tell the transfer to delete source files, but it can't (due to permissions or somesuch)?

Just an idea ...

[MateuszKowalski613123]

unmarked "erase source file" and all files transfered fine but they were also deleted from source host ???

[Anon Anon]

Does your RA FTP job run in a Workflow?

[MateuszKowalski613123]

yes, this is transfer from host A to B then from B to C, one more difference is that amount of all files for one flow was
- 131256279 bytes inflow which transfer all file in one run
- 38723806498 - flow witch this issue

[Anon Anon]

does the ftp job run fine, if you start it manually (without running in a workflow) ?

[MateuszKowalski613123]

yes, it went fine.

[Anon Anon]

This could be an old issue with RA.FTP jobs and Task properties (just search in community)

pls remove the jobs from that workflow - save - and insert them again + save

then it should work again...

[MateuszKowalski613123]

ok, thx will test as new files will be in place

[shravani_vedantham_6826]

 

I think it is because of a Job setting on the FTP tab. I have here screenshots of both AWI and Java Client.
In  AWI,. under FTP tab, you have job settings and there are three settings that can be changed.

The "number of transfer threads" Is by default 2. Once you change that , you should be able to transfer more files.

https://us.v-cdn.net/5019921/uploads/editor/xh/xywcznll6v2u.jpg" width="802">https://us.v-cdn.net/5019921/uploads/editor/6o/1gztrzgulv4z.jpg" width="937">

Above is java Client

[Carsten Schmitz]

The number of transfer threads is, in my humble understanding, the number of simultaneously running OS threads used to transfer files.

Two threads should in no way limit the number of files that can be transfered, but the degree of parallelization.

[shravani_vedantham_6826]

@Carsten Schmitz thankyou. For some reason I thought it was for controlling the number of file transfers.

[Carsten Schmitz]

shravani_vedantham_6826 you're welcome, and thank you kindly for the "insightful".

I was just imagining someone who wanted to transfer 10000 files cranking that thread count up to 10000, which would have been an exciting rapid automation experiment ;)

[shravani_vedantham_6826]

:#   Still new to AE V12.

[shravani_vedantham_6826]

:D @Carsten Schmitz no wonder that would have been me, if I had the same issue and didn't come across this thread.

[Carsten Schmitz]

No worries. I would think Automic has put a limit in there ... I hope.

When I get my RA agent to work (you're way ahead of me on that front ;) ) then I might give it a try and see what happens :)