Deploy an AWS DataSync Agent - AWS DataSync

Deploy an AWS DataSync Agent

Your agent can connect to public internet endpoints or private endpoints in virtual private clouds (VPCs). The activation process associates your agent with your AWS account.

Deploy Your DataSync Agent on VMware

You can download and deploy an AWS DataSync agent in your VMware environment and then activate it. You can also use an existing agent instead of deploying a new one. You can use a previously created agent if it can access your self-managed storage and if it's activated in the same AWS Region.

To deploy an agent on VMware

  1. Open the AWS DataSync console at https://console.aws.amazon.com/datasync/.

  2. If you don't have an agent, on the Create agent page in the console, choose Download image in the Deploy agent section. Doing this downloads the agent and deploys it in your VMware ESXi hypervisor. The agent is available as a VM. If you want to deploy the agent as an Amazon EC2 instance, see Deploy Your Agent as an EC2 Instance to Read Files from In-Cloud.

    AWS DataSync currently supports the VMware ESXi hypervisor. For information about hardware requirements for the VM, see Virtual Machine Requirements. For information about how to deploy an .ova file in a VMware host, see the documentation for your hypervisor.

    If you have previously activated an agent in this AWS Region and want to use that agent, choose that agent and choose Create agent. The Configure a Source Location page appears.

  3. Power on your hypervisor, log in to your VM, and get the IP address of the agent. You need this IP address to activate the agent.

    Note

    The VM's default credentials are the login admin and the password password.

    You can change the password on the local console. You don't need to log in to the VM for DataSync functionality. Login is mainly required for troubleshooting, such as running a connectivity test or opening a support channel with AWS. It's also required for network-specific settings, such as setting up a static IP address.

After you have deployed an agent, you choose a service endpoint.

Deploy Your DataSync Agent on KVM

You can download and deploy an AWS DataSync agent in your KVM environment and then activate it. You can also use an existing agent instead of deploying a new one. You can use a previously created agent if it can access your self-managed storage and if it is activated in the same AWS Region.

To deploy an agent on KVM

  1. Open the AWS DataSync console at https://console.aws.amazon.com/datasync/.

  2. If you don't have an agent, on the Create agent page in the console, choose Download image in the Deploy agent section. Doing this downloads the agent in a .zip file that contains a .qcow2 image file that can be deployed in your KVM hypervisor.

    The agent is available as a VM. If you want to deploy the agent as an Amazon EC2 instance, see Deploy Your Agent as an EC2 Instance to Read Files from In-Cloud.

    AWS DataSync currently supports the KVM hypervisor. For information about hardware requirements for the VM, see Virtual Machine Requirements. For information about how to deploy an .qcow2 file in a KVM host, see the documentation for your hypervisor.

    If you previously activated an agent in this AWS Region and want to use that agent, choose that agent, and then choose Create agent. The Configure a Source Location page appears.

  3. Power on your hypervisor, log in to your VM, and get the IP address of the agent. You need this IP address to activate the agent.

    Note

    The VM's default credentials are the login admin and the password password.

    You can change the password on the local console. You don't need to log in to the VM for DataSync functionality. Login is mainly required for troubleshooting, such as running a connectivity test or opening a support channel with AWS. It's also required for network-specific settings, such as setting up a static IP address.

After you deploy an agent, you choose a service endpoint.

Deploy Your DataSync Agent on Hyper-V

You can download and deploy an AWS DataSync agent in your Hyper-V environment and then activate it. You can also use an existing agent instead of deploying a new one. You can use a previously created agent if it can access your self-managed storage and if it is activated in the same AWS Region.

To deploy an agent on Hyper-V

  1. Open the AWS DataSync console at https://console.aws.amazon.com/datasync/.

  2. If you don't have an agent, on the Create agent page in the console, choose Download image in the Deploy agent section. Doing this downloads the agent in a .zip file that contains a .vhdx image file that can be deployed in your Hyper-V hypervisor.

    The agent is available as a VM. If you want to deploy the agent as an Amazon EC2 instance, see Deploy Your Agent as an EC2 Instance to Read Files from In-Cloud.

    AWS DataSync currently supports the Hyper-V hypervisor. For information about hardware requirements for the VM, see Virtual Machine Requirements. For information about how to deploy a .vhdx file in a Hyper-V host, see the documentation for your hypervisor.

    If you previously activated an agent in this AWS Region and want to use that agent, choose that agent, and then choose Create agent. The Configure a Source Location page appears.

  3. Power on your hypervisor, log in to your VM, and get the IP address of the agent. You need this IP address to activate the agent.

    Note

    The VM's default credentials are the login admin and the password password.

    You can change the password on the local console. You don't need to log in to the VM for DataSync functionality. Login is mainly required for troubleshooting, such as running a connectivity test or opening a support channel with AWS. It's also required for network-specific settings, such as setting up a static IP address.

After you deploy an agent, you choose a service endpoint.

Deploy Your Agent as an EC2 Instance to Read Files from In-Cloud

You can use your agent to transfer data between two locations in AWS, including cross-Region and cross-account transfers. Doing this enables you to perform the following tasks:

  • Transfer data from one Amazon EFS or Amazon FSx for Windows File Server file system to another – Migrate data from one AWS account to another, or periodically copy recently added files to a second EFS file system.

  • Migrate from a self-managed file system to Amazon FSx for Windows File Server – Migrate to benefit from a more scalable, fully managed, elastic, and highly available file storage that has an NFS interface.

  • Transfer data between Amazon S3 and in-cloud file systems – Use this approach for cases such as high performance computing (HPC) processing.

To get started, choose the Amazon Machine Image (AMI) for your agent for the AWS Region where your EFS or self-managed NFS file system resides:

  • To copy between Amazon EFS or Amazon FSx for Windows File Server file systems, or from a self-managed file system to a managed file system, create the EC2 agent in the source AWS Region.

  • To copy from Amazon S3, create the agent in the destination AWS Region.

Important

Deploy your agent such that it does not require network traffic between Availability Zones (to avoid charges for such traffic).

  • To access your Amazon EFS or Amazon FSx for Windows File Server file system, deploy the agent in an Availability Zone that has a mount target to your file system.

  • For self-managed file systems, deploy the agent in the Availability Zone where your file system resides.

To learn more about data transfer prices for all AWS Regions, see Amazon EC2 On-Demand Pricing.

Warning

We don't recommend using a DataSync agent that is deployed as an Amazon EC2 instance to read data from an on-premises source location. This approach doesn't deliver maximum throughput.

To choose the agent AMI for your AWS Region

  • Use the following CLI command to programmatically get the latest DataSync AMI ID for the specified Region.

    aws ssm get-parameter --name /aws/service/datasync/ami --region $region

    Example command and output

    aws ssm get-parameter --name /aws/service/datasync/ami --region us-east-1 { "Parameter": { "Name": "/aws/service/datasync/ami", "Type": "String", "Value": "ami-id", "Version": 6, "LastModifiedDate": 1569946277.996, "ARN": "arn:aws:ssm:us-east-1::parameter/aws/service/datasync/ami" } }

    For the recommended instance types, see Amazon EC2 Instance Requirements.

    If you activate an agent in this Region that has access to your file system using a mount target in the same Availability Zone and you want to use that agent, choose the agent, and then choose Create agent. The Configure a Source Location page appears.

To deploy your DataSync agent as an EC2 instance

  1. From the AWS account where the source file system resides, launch the agent using your AMI from the Amazon EC2 launch wizard. Use the following URL to launch the AMI.

    https://console.aws.amazon.com/ec2/v2/home?region=source-file-system-region#LaunchInstanceWizard:ami=ami-id

    In the URL, replace the source-file-system-region and ami-id with your own source AWS Region and AMI ID. The Choose an Instance Type page appears on the Amazon EC2 console. Use the .AMI-command CLI command described in the preceding section to find the DataSync AMI ID for a specified AWS Region.

  2. Choose one of the recommended instance types for your use case, and choose Next: Configure Instance Details. For the recommended instance types, see Amazon EC2 Instance Requirements.

  3. On the Configure Instance Details page, do the following:

    1. For Network, choose the virtual private cloud (VPC) where your source EFS or NFS file system is located.

    2. For Auto-assign Public IP, choose a value. For your instance to be accessible from the public internet, set Auto-assign Public IP to Enable. Otherwise, set Auto-assign Public IP to Disable. If a public IP address isn't assigned, activate the agent in your VPC using its private IP address.

      When you transfer files from an in-cloud file system, to increase performance we recommend that you choose a Placement Group value where your NFS server resides.

  4. Choose Next: Add Storage. The agent doesn't require additional storage, so you can skip this step and choose Next: Add tags.

  5. (Optional) On the Add Tags page, you can add tags to your EC2 instance. When you're finished on the page, choose Next: Configure Security Group.

  6. On the Configure Security Group page, do the following:

    1. Make sure that the selected security group allows inbound access to HTTP port 80 from the web browser that you plan to use to activate the agent.

    2. Make sure that the security group of the source file system allows inbound traffic from the agent. In addition, make sure that the agent allows outbound traffic to the source file system. If you deploy your agent using a VPC endpoint, you need to allow additional ports. For more information, see How DataSync Works with VPC Endpoints .

    For the complete set of network requirements for DataSync, see Network Requirements for DataSync.

  7. Choose Review and Launch to review your configuration, then choose Launch to launch your instance. Remember to use a key pair that's accessible to you. A confirmation page appears and indicates that your instance is launching.

  8. Choose View Instances to close the confirmation page and return to the EC2 instances screen. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running. At this point, it is assigned a public Domain Name System (DNS) name and IP address, which can be found in the Descriptions tab.

  9. If you set Auto-assign Public IP to Enable, choose your instance and note the public IP address in the Description tab. You use this IP address later to connect to your sync agent.

    If you set Auto-assign Public IP to Disable, launch or use an existing instance in your VPC to activate the agent. In this case, you use the private IP address of the sync agent to activate the agent from this instance in the VPC.

After you have deployed an agent, you choose a service endpoint.

Deploy Your DataSync Agent on an AWS Snowcone Device

The DataSync agent AMI is pre-installed on your Snowcone device. You can use AWS OpsHub for Snow Family or the AWS Snowball Edge CLI command line tool to launch the agent and attach a virtual interface to the agent. Then, use the virtual interface’s IP address to activate the agent.

For instructions on launching the agent using AWS OpsHub, see Using DataSync to Transfer Files to AWS.

For instructions on launching the agent using the Snowball CLI, see Launching AWS DataSync AMI.

For information about using the AWS Snowcone client, see Using the Snowcone Client.