Prerequisites Step 1: Install File Transfer Step 2: Configure File Transfer

Getting started with Nimble Studio File Transfer

This tutorial shows how to install and configure File Transfer. Before you begin the following steps, make sure that the Setting up Nimble Studio File Transfer tutorial has been completed by your administrator.

If you encounter any problems while following this tutorial, see the Support for Nimble Studio File Transfer page.

Prerequisites

Complete the Setting up Nimble Studio File Transfer section before installing and configuring File Transfer.

Hardware configuration

We recommend that your computer meets the following requirements for you to use File Transfer. For more information about how to increase the speed of your transfers, see Performance optimization.

8 logical CPU cores
8 GB RAM

Your transfer speeds depend on your hardware, network configuration, and bandwidth. File Transfer can transfer files as your network and hardware permits. For example, if your machine has been allocated a network bandwidth of 500 Mbps, the fastest that File Transfer can try to complete transfers is 500 Mbps.

Step 1: Install File Transfer

The File Transfer installer will guide you through the setup, and it will set the correct threads and chunk size based on your machines.

Download and install the Nimble Studio File Transfer client from File Transfer clients.

Step 2: Configure File Transfer

With File Transfer, you can either use the graphical user interface (GUI) or the command line interface (CLI). If you use an operating system (OS) with a desktop, you can configure File Transfer in the GUI. If you aren’t using an OS with a desktop, you can use the File Transfer CLI to configure and run File Transfer. The CLI provides more flexibility for how you use File Transfer.

GUI

Before you can transfer files with File Transfer, you must add a remote configuration. Remote configurations are treated as different configurations that you can use to transfer files to different buckets and directories.

To add a remote configuration

Choose Create a Remote Configuration on the S3 Bucket file browser window of the GUI.
In the Add Remote Configuration section, enter the following information.
1. In Remote Configuration name, enter a name for the remote configuration. The name can't be modified once it is set.
2. In S3 Bucket, add the name of the S3 bucket you want to use for File Transfer. To create an S3 bucket, see Create an S3 bucket.
3. In AWS Region, enter the region where your S3 bucket is located.
4. In AWS named profile, enter an AWS named profile to use for S3 access. If you choose to use an AWS Access Key or an AWS Secret Key instead, leave this field blank. If you do not have an AWS named profile, see the Create an IAM Access Policy and Set up the AWS CLI topics.
Choose Add to continue.
In the Advanced section, enter the following information.
1. Storage Class specifies the storage class of the S3 Bucket. For information on storage classes, see Amazon S3 Storage Classes
2. Checksum Algorithm specifies which checksum algorithm is used for the checksum computations.
3. (Optional) Local directory specifies the default directory on your local machine.
4. (Optional) S3 bucket prefix specifies the default directory of the S3 bucket.
5. Filter specifies a filter for files based on format. This field accepts valid regular expressions. For example, "^.*\.(mov)$" only uploads files ending in .mov.
6. Max Age Limits File Transfer to files created or modified after the Max Age time value and before the current time. Units are expressed as m (minutes), h (hours), d (days), and w (weeks). If no units are supplied, the value is specified in seconds. For example, an input of 5d transfers files created or modified in the last five days. An input of 350 transfers files created or modified in the last 350 seconds.
7. (Optional) File order list sets transfer priority by file extension. Enter an extension to add it to the list. Files with extensions that aren't in the list are the lowest priority and are transferred in the order they are listed in the file system.
8. Transfer autotuning dynamically sets the number of threads and chunk size for file transfers when enabled. This is enabled by default. We recommend that you keep this option enabled.
  
  Number of threads
  
  Number of individual threads that are used to transfer each individual file.
  
  Chunk size
  
  Size of the chunk (in megabytes) that is delivered by each thread.
9. Enabling Use S3 Transfer Acceleration improves the speed of transfer for large files when enabled. The best practice is to keep S3 Transfer Acceleration off because it isn't required to achieve high speed transfers in File Transfer. If the bucket is geographically far from you, turn on this feature. For more information, see S3 Transfer Acceleration. If this is enabled, additional fees might apply. For more information, see Amazon S3 Pricing.
10. Enable metadata filter automatically filters system metadata files when enabled. These files include files that start with ._ as well as thumbs.db and .DS_Store files.
Choose Save

To add a studio ID

Open File Transfer.
1. Go to the Start Menu and search for File Transfer.
2. Select Nimble Studio File Transfer from the list.
From the dropdown menu ( ), choose Settings.
Enter the studio ID that you found in the Set up a studio in Nimble Studio tutorial.
Choose Save.

CLI

Now that File Transfer is installed, edit the configuration file.

To edit the configuration file

Open the configuration file with any text editing software on your computer.
1. Windows: Navigate to the User/<your username> folder on your computer. Open the .filetransfer folder and open the filetransfer.yaml file with a text editor.
2. macOS: Enter Cmd+Shift+G. Then enter ~/.filetransfer. Open the filetransfer.yaml file with a text editor.
3. Linux: Open the filetransfer.yaml file using any text editor. The file is located in ~/.filetransfer/configuration.yaml.
Define the following variables in the configuration file. The required values are populated. You must provide the optional values.
1. studio_id: Enter the studio ID that you found in the Set up a studio in Nimble Studio tutorial.
2. max_active_checksums: Enter the number of individual checksums that are processed at the same time. If this value exceeds the number of available CPU cores in the system, a warning message displays and the value is capped to the number of CPU cores. For more information, see File Transfer checksums.
3. max_active_transfers: Enter the number of individual files that are processed at the same time.

Define at least one remote configuration in the configuration file. The required values for the remote configuration aren't populated, so you must manually enter them. Define the following variables for each remote configuration.


protocols:
    s3:
        remote_configuration:
            demo:
                name: demo
                bucket: my-bucket
                region: us-west-2
                profile: my-profile
                storage_class: standard
                auto_tuning: true
                chunk_size: 25
                threads: 10
                checksum_algorithm: md5_hex
                max_age: ""
                accelerated: false
                file_order: []
                filter: ""
                upload_hot_folder:
                    enabled: false
                    local_source_folder: ""
                    s3_destination_folder: ""
                enable_metadata_filter: true
                paths:
                    local: ""
                    remote: ""

name: Enter a name for your transfer profile.
bucket: Enter the name of the S3 bucket that you want to upload to and download from. Your administrator should have created the bucket in Create an S3 bucket.
region: Enter the AWS Region that your bucket is located in.
profile: Enter the name of the profile that your admin created in step 4 of Set up the AWS CLI. This profile allows you to access Amazon Simple Storage Service (Amazon S3). To get a list of configured profiles, run the following command in a terminal window: aws configure list
storage_class: By default, this is set to standard. The accepted values for this variable are as follows:
- reduced_redundancy
- standard_ia
- onezone_ia
- intelligent_tiering
- glacier
- deep_archive
- glacier_ir
Note
You can upload directly to any storage class in Amazon S3. If you use S3 Glacier Deep Archive or S3 Glacier Flexible Retrieval, you can't download objects you uploaded with File Transfer from File Transfer directly. For more information about storage classes, see Amazon S3 Storage Classes.
auto_tuning: Automatically adjusts chunk size and max active transfers based on the size of the file. By default, this is set to true.
chunk_size: Enter the size of the chunk (in MB) that's delivered by each thread. This field isn't required if auto_tuning is set to true.
threads: Enter the number of individual threads that is used to transfer each individual file. This field isn't required if auto_tuning is set to true.
checksum_algorithm: This is the checksum algorithm that will be used when uploading your files to S3. You can choose md5-hex (the default value), xxhash, xxhash64, and xxh3 checksum algorithms.
(Optional) Max Age: Limits File Transfer to files created or modified after the Max Age time value and before the current time. Units are expressed as m (minutes), h (hours), d (days), and w (weeks). If no units are supplied, the value is specified in seconds. For example, an input of 5d transfers files created or modified in the last five days. An input of 350 transfers files created or modified in the last 350 seconds.
accelerated: Enables S3 Transfer Acceleration. By default, this is false. The best practice is to keep S3 Transfer Acceleration off because it isn't required to achieve high speed transfers in File Transfer. If the bucket is geographically far from you, turn on this feature. For more information, see S3 Transfer Acceleration. If this is enabled, additional fees might apply. For more information, see Amazon S3 Pricing.
(Optional) file_order: Enter a list of comma-separated file extensions to define a priority of files to transfer. Any file extensions that you don't list are transferred last in the order that the file system lists them. For example, ".mov,.txt" prioritizes transferring .mov files over .txt files.
(Optional) filter: Filter files being transferred based on format. The filter field accepts valid regular expressions. For example, "^.*\.(mov)$" only uploads files ending in .mov
(Optional) upload_hot_folder enabled: Turns on the upload hot folder functionality.
(Optional) upload_hot_folder local_source_folder: This is required when upload hot folder is enabled. File Transfer recursively monitors all file system events in the specified folder and initiates uploads when files are added or modified.
(Optional) upload_hot_folder s3_destination_folder: This is required when upload hot folder is enabled. Hot folder files that are uploaded are added to this folder.
enable_metadata_filter: When true, File Transfer filters system metadata files automatically. These files include Thumbs.db, .DS_Store, and files that start with ._.
(Optional) paths local: The root folder for File Transfer files in the local file system.
(Optional) paths remote: The root folder for File Transfer files in the S3 bucket.
(Optional) paths local: Enter a file path. Your transfers will begin at that local path in the file browser for the local file system.
(Optional) paths remote: Enter a file path. Your transfers will begin at that path in the file browser for the S3 bucket.
Save the configuration file.
Run the following command to validate your AWS credentials: filetransfer validate-credentials remote configuration
1. Replace remote configuration with the name of the configuration that you created.
2. This command checks the AWS credentials that you provided, such as your IAM key. This command checks that File Transfer is able to connect to Amazon S3, and it lists objects in the bucket that's specified in the configuration file.

Having completed installation and configuration, you’re now ready to use File Transfer.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Setting up

Using the GUI