Inside Automation Engine > FileTransfer > FileTransfers - Workflow

 File Transfer (FT) Procedure

OS agents are designed for the transfer of files. Doing so requires an agentA program that enables the de-centralized execution of processes (such as deployments) on target systems (computers or business solutions) or a service that provides connectivity to a target system (such as for databases or middleware). An agent is also an object type in the Automation Engine. [Formerly called "Executor."] See also: host to be installed on the source computer and on the target computer. Files are transferred in a secured and encrypted way.

Automation EngineThis component drives an Automation Engine system and consists of different types of server processes. version 9.00A provides an optimized and improved file transferTransfers files from one computer to another. A particular Automation Engine object type (FileTransfer object). procedure. This new protocol is only used if the participating agents are of version 9.00A or later. For compatibility reasons, the old procedure is used if one of the agents has an older version.

Old File Transfer Protocol (Up To 8.00A)

Both agents require a connection for the file transfer to start. One of the work processes contacts the communication processA communication process is part of the component Automation Engine. It is responsible for connecting the components. that is connected to the source agent and informs it about the connection request. The communication process passes the information on to the agent. The agent uses the included information in order to connect to the target agent.

If the Automation Engine's attempt to establish a connection fails (for example, because of Firewall settings), it uses the same information in order to contact the target agent. The target agent now tries to establish a connection to the source agent.

 

 

The file transfer can start as soon as the two agents are connected with each other. Status messages are regularly sent to the Automation Engine in order to track the progress. The file transfer taskAn executable object that is running. Tasks are also referred to as activities.'s Detail Window in the UserInterfaceThis is the Automation Engine's graphical user interface. [Formerly called the "Rich Client", "RichGUI" and "Dialog Client."] shows the number of bytes that have already been transferred.

Unlike with the new FT protocol, all file transfer files are sent via a connection (FT connection). Blocks of different files may be transferred alternately. The Automation Engine monitors the whole file transfer procedure and instructs the source agent to send the individual files.

 

The administrator can determine that the connection between the two agents should end after the files have been transferred. This is done in the variableIt stores or retrieves values dynamically at runtime. An individual Automation Engine object type. UC_HOSTCHAR_DEFAULT using the setting DISCONNECT_AFTER_FT. 

New File Transfer Protocol (as of 9.00A)

As of AE v9, the Automation Engine sends the complete file transfer request (including wildcard specifications in partially qualified file transfers) to the source agent. The sending agent is responsible for resolving the request (determining the files).

 

 

Connection establishment

The sending agent tries to establish a connection to the receiving agent. If this attempt fails (for example, because of Firewall settings), it notifies the Automation Engine. The FT request is then sent to the receiver which now tries to establish a connection to the sender. After the connection has been established, the receiving agent transfers the FT request to the sender.

 

 

Checking the disk space

Depending on OS, the system will check before starting a file transfer whether there is enough disk space on the target platform and, if not, will allocate it.

Handling file transfers

For each file transfer, the new protocol establishes a separate connection between the agents. The files are always sent through a connection one after the other. Each file transfer is handled in a separate thread or process if this is supported by the agent. Therefore, several file transfers can be processed independently of each other.

The setting DISCONNECT_AFTER_FT of the variable UC_HOSTCHAR_DEFAULT does not affect the new protocol because the system ends the connections after the Filetransfers have been completed.

The agents of the following operating systems support threads:

NSK handles each file transfer with a separate process. Therefore, the NSK agent has a second port especially for file transfers.

File transfers are not affected by any connection errors between Automation Engine and agents - they are continued. The file transfer's actual statusThis represents the condition of a task (such as active, blocked, generating). is sent to the Automation Engine as soon as the connection could be re-established.

The NFS security setting "root squash" causes problems in file transfers with UNIX agents if it is used in combination with the old FT protocol. The reason is that file transfers are always executed under the UNIX userIn the Automation Engine, a user is an instance of a User object, and generally the user is a specific person who works with Automic products. The User object is assigned a user ID and then a set of access rights to various parts of the Automation Engine system and product suite. These access rights come in the form of Automation Engine authorizations and privileges, Decision user roles and EventBase rights and ARA web application object rights. You can manage all these centrally in the ECC user management functions. See also, Unified user management. "root". This error does not occur in the new protocol because file transfers under UNIX always run under the user specified in the Login object.

 

 

The following file transfer procedures are provided in order to ensure a reliable transfer of files:

Transmission security

The accuracy of transferred data is verified with an MD5 checksum verifier that is embedded in the data stream. Data is verified in packets.

Consistency check for restarted file transfers

Unlike with the old protocol, you cannot repeat individual file transfer files selectively. Erroneous file transfers can be repeated from the last restartA restart refers to the repetition of an object's execution. This action differs from a new start in some parts. point.

At particular intervals, the agents automatically create restart points while the files are being transferred (setting FT_RESTARTINFO_INTERVAL in the variable UC_HOSTCHAR_DEFAULT). The agent stores this information locally on its computer in StatusStore files. If an error occurs, the file transfer can be restarted from the file's last restart point (Restart option: "From last restart position"). This functionPre-defined run book template in the Automation Engine. One single step only, e.g. Start Windows Service, Copy file,… saves time especially if most of a huge file has already been transferred.

You can use the settings FT_RESTARTINFO_LIFETIME and FT_RESTARTINFO_CHECK (UC_HOSTCHAR_DEFAULT) in order to specify that StatusStore files should be deleted.

In order to ensure that the target file complies with the source file after it has successfully been restarted, the transferred data is checked against an MD5 checksum.When it creates the restart point, the system also retrieves the MD5 checksum and stores it in the StatusStore file. The checksums differ if the partially transferred file has been changed on the receiving agent's computer, and the restart results in an error.

To save transmission time, MD5 checksums of files that are smaller than 1 MB are not calculated.

You can deactivate the MD5 checksum using the setting FT_USE_MD5 in the variable UC_HOSTCHAR_DEFAULT.

 

Depending on the particular agent platform, the StatusStore files are stored in the following directories: 

Platform Directory File name Peculiarity
Windows The agent's Temp directory

FTNNNNNNN.sts

NNNNNNN is the file transfer's RunIDShort for "run number". It is a number that provides unique information about a task's execution. The RunID can include 7 to 10 digits. It is assigned by the Automation Engine component . that has been converted to a 7-letter string. You can use the scriptA particular Automation Engine object type. element ALPHA2RUNNR in order to convert it to the regular 10-digit RunID.

StatusStore file per file transfer
BS2000 The agent's Temp directory FTNNNNNNN.sts StatusStore file per file transfer
Unix/VMS The agent's Temp directory FTNNNNNNN.sts StatusStore file per file transfer
OS/400

Depending on the INI-file parameter store_type=

IFS: FTNNNNNNN.sts
QSYS: Object with the name FTNNNNNNN and the type USRSPC
StatusStore file per file transfer
NSK

Sub-volume in the configurationA set of constituent components that make up a system. This includes information on how the components are connected including the settings applied. file (see the NSK agent installation).

The agent's INI file:

Section [FT-STATUS-STORE]

Four StatusStore files that include all restart information
z/OS  

StatusStore dataset

See: Installing the z/OS Agent

A StatusStore dataset that includes all restart information

 

See also:

 


Automic Documentation - Tutorials - Automic Blog - Resources - Training & Services - Automic YouTube Channel - Download Center - Support

Copyright © 2016 Automic Software GmbH