Automic Workload Automation

Expand all | Collapse all

only 2 files processed in file transfer

Jump to Best Answer

Anon Anon01-23-2018 07:48 AM

MateuszKowalski61312301-23-2018 10:06 AM

MateuszKowalski61312301-23-2018 10:12 AM

shravani_vedantham_682601-23-2018 12:03 PM

  • 1.  only 2 files processed in file transfer

    Posted 01-23-2018 05:32 AM
    Hello,

    I'm have an issue during transferring files, job run fine but transfer only 2 files in one run, error message is:

    Failure
    com.uc4.ftpjob.DataTransferException: Failure
         at com.uc4.ftpjob.commands.SFtpCommandImpl.runCmd(SFtpCommandImpl.java:105)
         at com.uc4.ftpjob.commands.SFtpCommandImpl.delete(SFtpCommandImpl.java:125)
         at com.uc4.ftpjob.commands.FtpFacadeCommandImpl.delete(FtpFacadeCommandImpl.java:348)
         at com.uc4.ftpjob.commands.impl.CopyCommand.execute(CopyCommand.java:668)
         at com.uc4.ftpjob.commands.ThreadPoolJobExecutor$ThreadRunner.run(ThreadPoolJobExecutor.java:86)
         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         at java.lang.Thread.run(Thread.java:745)
    Caused by: 2: Failure
         at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2579)
         at com.jcraft.jsch.ChannelSftp.rm(ChannelSftp.java:1780)
         at com.uc4.ftpjob.commands.SFTPHelperDelegate$30.run(SFTPHelperDelegate.java:421)
         at com.uc4.ftpjob.commands.SFTPHelperDelegate$1.run(SFTPHelperDelegate.java:64)
         ... 1 more
    2018-01-23 11:18:42             2 files transferred, 75 not processed - 113734749 bytes sent in 1549.89 Kbytes/sec

    I hav ealso another FT which transfer files between same hosts and same path, only file name is different, file name in flow that run fine is x/***/xxxx/yy_*match and from FT which transfer only 2 files is x/***/xxxx/yy_*nomatch.

    Any idea what can be wrong ?


  • 2.  only 2 files processed in file transfer

    Posted 01-23-2018 06:16 AM
    I notice the third line in the exception is something about "delete".

    Did you, by any chance, tell the transfer to delete source files, but it can't (due to permissions or somesuch)?

    Just an idea ...


  • 3.  only 2 files processed in file transfer

    Posted 01-23-2018 06:39 AM
    unmarked "erase source file" and all files transfered fine but they were also deleted from source host ???


  • 4.  only 2 files processed in file transfer

    Posted 01-23-2018 07:48 AM
    Does your RA FTP job run in a Workflow?


  • 5.  only 2 files processed in file transfer

    Posted 01-23-2018 09:17 AM
    yes, this is transfer from host A to B then from B to C, one more difference is that amount of all files for one flow was
    - 131256279 bytes inflow which transfer all file in one run
    - 38723806498 - flow witch this issue


  • 6.  only 2 files processed in file transfer

    Posted 01-23-2018 09:47 AM
    does the ftp job run fine, if you start it manually (without running in a workflow) ?


  • 7.  only 2 files processed in file transfer

    Posted 01-23-2018 10:06 AM
    yes, it went fine.


  • 8.  only 2 files processed in file transfer
    Best Answer

    Posted 01-23-2018 10:10 AM
    This could be an old issue with RA.FTP jobs and Task properties (just search in community)

    pls remove the jobs from that workflow - save - and insert them again + save

    then it should work again...


  • 9.  only 2 files processed in file transfer

    Posted 01-23-2018 10:12 AM
    ok, thx will test as new files will be in place


  • 10.  only 2 files processed in file transfer

    Posted 01-23-2018 11:51 AM

     

    I think it is because of a Job setting on the FTP tab. I have here screenshots of both AWI and Java Client.
    In  AWI,. under FTP tab, you have job settings and there are three settings that can be changed.

    The "number of transfer threads" Is by default 2. Once you change that , you should be able to transfer more files.

    xywcznll6v2u.jpghttps://us.v-cdn.net/5019921/uploads/editor/xh/xywcznll6v2u.jpg" width="802">1gztrzgulv4z.jpghttps://us.v-cdn.net/5019921/uploads/editor/6o/1gztrzgulv4z.jpg" width="937">

    Above is java Client



  • 11.  only 2 files processed in file transfer

    Posted 01-23-2018 11:54 AM
    The number of transfer threads is, in my humble understanding, the number of simultaneously running OS threads used to transfer files.

    Two threads should in no way limit the number of files that can be transfered, but the degree of parallelization.


  • 12.  only 2 files processed in file transfer

    Posted 01-23-2018 11:59 AM
    @Carsten Schmitz thankyou. For some reason I thought it was for controlling the number of file transfers.


  • 13.  only 2 files processed in file transfer

    Posted 01-23-2018 12:02 PM
    shravani_vedantham_6826 you're welcome, and thank you kindly for the "insightful".

    I was just imagining someone who wanted to transfer 10000 files cranking that thread count up to 10000, which would have been an exciting rapid automation experiment ;)


  • 14.  only 2 files processed in file transfer

    Posted 01-23-2018 12:03 PM
    :#   Still new to AE V12.


  • 15.  only 2 files processed in file transfer

    Posted 01-23-2018 12:05 PM
    :D @Carsten Schmitz no wonder that would have been me, if I had the same issue and didn't come across this thread.


  • 16.  only 2 files processed in file transfer

    Posted 01-23-2018 12:08 PM
    No worries. I would think Automic has put a limit in there ... I hope.

    When I get my RA agent to work (you're way ahead of me on that front ;) ) then I might give it a try and see what happens :)