Menu
Amazon Elastic File System
User Guide

Walkthrough 8: Sync a File System from Amazon EC2 to Amazon EFS Using EFS File Sync

This walkthrough shows the steps how to sync files from a file system that is in AWS to Amazon EFS using EFS File Sync.

Before You Begin

In this walkthrough, we assume the following:

  • You have a Network File System (NFS) file server on an Amazon EC2 instance.

  • You have created an Amazon EFS file system. If you don't have an Amazon EFS file system, create one now and come back to this walkthrough when you are done. For more information about how to create an Amazon EFS file system, see Getting Started with Amazon Elastic File System.

Step 1: Create a Sync Agent

To create a sync agent in Amazon EC2, you will use the AMI provided to create an Amazon EC2 instance that can mount the source file system in your AWS environment. This Amazon EC2 instance will run in the same AWS Region as your source file system. Once deployed, you will activate the agent to securely associate it with your AWS account.

To create a sync agent for data in AWS

  1. Open the Amazon EFS Management Console at https://console.aws.amazon.com/efs/ and choose the AWS Region where you created your destination file system.

  2. Choose File syncs. If you haven't used EFS File Sync in this AWS Region, you see an introductory page. Choose Get started to open the Select a host platform page.

    If you have previously used EFS File Sync in this AWS Region, choose Agents from the left navigation, and then choose Create sync agent to open the Select a host platform page.

  3. From Select a host platform page, choose Amazon EC2, choose the AWS Region where your source file system is located and then choose Launch instance. You will be redirected to the Choose an Instance Type page in the Amazon EC2 Management Console in that AWS Region, where you can choose an instance type.

    Note

    A sync agent syncs files to EFS file systems in the AWS region where the sync agent is activated. Standard Amazon EC2 rates apply to the instance.

  4. On the Choose an Instance Type page, choose the hardware configuration of your instance. When deploying your sync agent on Amazon EC2,we recommend choosing one of the Memory optimized r4.xlarge instance types for your sync agent. The instance size you choose must be at least xlarge.

  5. Choose Next: Configure Instance Details.

  6. On the Configure Instance Details page, choose a value for Auto-assign Public IP. If you want your instance to be accessible from the public internet, set Auto-assign Public IP to Enable. Otherwise, set Auto-assign Public IP to Disable.

  7. Choose Next: Add Storage and choose Next: Add tags. The EFS File Sync agent uses the root volume and doesn't require additional storage.

  8. On the Add Tags page, you can optionally add tags to your instance. Then choose Next: Configure Security Group.

  9. On the Configure Security Group page, add firewall rules for specific traffic to reach your instance. You can create a new security group or choose an existing security group.

    Important

    At a minimum, your security group must allow inbound access to HTTP port 80 from your web browser to activate your sync agent.

  10. Choose Review and Launch to review your configuration, then choose Launch to launch your instance. We recommend selecting an existing key pair or creating a new key pair for your instance. This key pair is not needed for normal operation of EFS File Sync, but it may be needed if you contact AWS to get support.

    A confirmation page appears to say that your instance is launching.

  11. Choose View Instances to close the confirmation page and return to the console. On the Instances screen, you can view the status of your instance. It takes a short time for an instance to launch. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running, and it's assigned a public DNS name and IP address.

  12. Choose your instance and take note of the public IP address in the Description tab. You use this IP address to connect to your sync agent.

    Note

    The IP address doesn’t need to be accessible from outside your network.

    Important

    If your source file system and destination Amazon EFS file system are in different AWS Regions, you open the Amazon EFS Console in the AWS Region where your destination Amazon EFS file system is located to connect.

    Your source and destination file system must be in different virtual private clouds (VPCs).

  13. Choose File syncs, choose Create sync agent, and then choose Next: Connect to agent on the Select host platform page.

  14. For IP address, type the Amazon EC2 instance IP address, and then choose Next: Activate agent. Your browser will connect to this IP address to get a unique activation key from your sync agent. This key securely associates your sync agent with your AWS account. This IP address doesn't need to be accessible from outside your network, but must be accessible from your browser.

  15. On the Activate agent page, type a name for your sync agent and choose Activate agent.

At this point, you should see your activated sync agent on the Amazon EFS console.

Step 2: Create a Sync Task

Create a sync task and configure the source and destination file systems.

To create a sync task

  1. Choose Create sync task. The Configure source location page appears.

  2. Provide the following information for the source file system:

    • For NFS server, type the domain name or IP address of the source NFS server.

    • For Mount Path, type the mount path for your source file system.

    • For Agent, choose the sync agent that you created earlier.

  3. Choose Next: Configure destination. The Configure destination location page appears.

  4. Provide the following information for the destination file system:

    • For Amazon EFS file system, choose the EFS file system you want to sync to. If you don't have an Amazon EFS file system, create one now and restart this walkthrough when you are done. For more information about how to create an Amazon EFS file system, see Getting Started with Amazon Elastic File System.

    • For File system path, type the path of the file system that you want to write data to. This path must exist in the destination file system.

    • For Security group, choose a security group that allows access to the destination Amazon EFS file system you selected.

  5. Choose Next: Configure settings. The Configure sync settings page appears.

  6. Configure the default settings that you want this sync task to use for synchronizing your files:

    Note

    You can override these settings later when you start a sync task.

    • Choose Ownership (User/Group ID) to copy the user and group IDs from the source files.

    • Choose Permissions to copy the source files permissions.

    • Choose Timestamps to copy time stamps from the source files.

    • Choose File deletion to keep all files in the destination that are not found in the source file system. If this box is cleared, all files in the destination that are not found in the source file system will be deleted.

    • Choose Verification mode to check that the destination file system is an exact copy of the source file system after the sync task completes. If you do not choose this option, only the data that is transferred is verified. Changes made to files while they are being actively transferred and changes made to files that are not actively being transferred, will not be discovered. We recommend choosing full verification.

  7. Choose Next: Review and Create, and then review your sync task settings. When you are ready, choose Create sync task.

Your sync task is created. The status of the task will show as Available when the source and destination file systems have been mounted.

The Details tab shows the status and settings for your source and destination file systems.

Step 3: Sync Your Source File System to Amazon EFS

Now that you have a sync task, you can start your sync task to begin syncing files from the source file system to the destination EFS File Sync file system.

To sync the source file system

  1. On the Tasks page, choose the sync task you just created. The Details tab shows the status of your sync task.

  2. In the Actions menu, choose Start.

  3. In the Start sync task dialog box, you can modify the settings for your sync task and choose Start.

  4. Choose Start to start syncing files.

  5. When the sync task starts, the Status column shows the progress of the sync task. As the sync task begins preparation, the status changes from Starting to Preparing. When the task starts to sync files, the status changes from Preparing to Syncing. When file consistency verification starts, the status changes to Verifying. When the sync task is done, the status changes to Success.

Step 4: Access Your Files

To access your files, connect to the Amazon EFS file system from an Amazon EC2 instance or use AWS Direct Connect.

For information about how to connect using Amazon EC2, see Connect to Your Amazon EC2 Instance and Mount the Amazon EFS File System.

For information about how to connect using AWS Direct Connect, see Walkthrough 5: Create and Mount a File System On-Premises with AWS Direct Connect.

Step 4: Clean Up

If you no longer need the resources you created, you should remove them to protect your account:

If you no longer need the resources you created, you should remove them:

  • Clean up your instance if you created your EFS File Sync on Amazon EC2. For more information, see Step 3: Clean Up Your Instance in the Amazon EC2 User Guide for Linux Instances.