Automic Workload Automation

Expand all | Collapse all

File transfers to/from Amazon S3

  • 1.  File transfers to/from Amazon S3

    Posted 05-10-2017 04:57 AM
    I am interested in automating file transfers to/from S3 using the Automation Engine. Has anyone worked with Amazon S3 using the AE? How did you do it? Is there an agent or RA solution for this?


  • 2.  File transfers to/from Amazon S3

    Posted 05-13-2017 08:03 AM
    Hello Michael,
    As far as I know, there is no file transfer to/from S3.  The closest solution we currently have is the Amazon AWS Action Pack, which is available in the Marketplace and requires ARA or Service Orchestration.  However, it currently does not do a file transfer.  These are its current features:

    • Allocate Elastic IP Address
      Associate/Disassociate Elastic IP Address
      Associate/Disassociate Route Table
      Attach/Detach Internet Gateway to VPC
      Attach/Detach Network Interface
      Attach/Detach EBS volume to instance
      Add rule to security group
      Create/Delete Security Group
      Create Instance
      Create/Delete Internet Gateway
      Create Network ACL in a VPC
      Create/Delete Network ACL Entry
      Create/Delete Network Interface
      Create/Delete Route
      Create Route Table
      Create Subnet
      Create/Delete EBS Volume
      Create/Delete VPC
      Allocate and Associate VPC Address
      Delete Network ACL
      Describe Addresses
      Modify Attribute of Instance
      Release Elastic IP Address
      Replace Network ACL Association
      Replace Route
      Replace Route Table Association
      Remove rule from security group
      Start/Stop Instance
      Terminate Instance
    You may have to log a Feature Enhancement Request.  


  • 3.  File transfers to/from Amazon S3

    Posted 05-15-2017 04:52 AM

    I have submitted a new enhancement request for this:

    Amazon S3 file transfers 

    If you like the idea, please vote for it.



  • 4.  Re: File transfers to/from Amazon S3

    Posted 09-26-2018 05:28 AM

    We have the same requirement so I have voted for the idea.!

     

    Is it possible to send files using the REST API ?



  • 5.  Re: File transfers to/from Amazon S3

    Posted 09-26-2018 10:25 AM

    If memory serves well, can't curl be used to upload data into S3 buckets?



  • 6.  Re: File transfers to/from Amazon S3

    Posted 09-27-2018 02:48 AM

    We have a dedicated server with AWS Api installed and configured.

    For a backup workflow, we run a JOBS on it who call the following command line

     

    aws s3 cp /opt/Automic/Backup_Files/&$CLIENT#/backup_file_&DATE#.tar.gz s3://&BUCKET#/automic/XML/&$CLIENT#/backup_file_&DATE#.tar.gz

     

    the AWS location is in Ireland

    this is work well

     

     

     

    PS: sorry for my english...



  • 7.  Re: File transfers to/from Amazon S3

    Posted 02-21-2019 04:09 PM

    I used the RA REST WebService version 4.4.2 to upload files to Amazon S3. But I haven't managed to get the download working. I always get a response type null and the job fails on processing the empty response. Full tracing doesn't reveal anything helpful. Download from Google Drive for example is working. Do you have any up-to-date experience?

     

    Update: Meanwhile I have realized that the connection in the customer installation is not directly with Amazon S3 but via another web service.

    Downloading binaries like exe or pdf directly from Amazon S3 produces corrupt files.



  • 8.  Re: File transfers to/from Amazon S3

    Posted 04-10-2019 10:16 AM

    Hello Siegfried, do you mind sharing a snapshot (or xml object) of the RA REST job you're using to upload to Amazon S3 and Google drive. I am very interested in accomplishing both task for two separate project. Thanks so much in advance. 

    John