Getting started with AWS Database Migration Service - AWS Database Migration Service

Getting started with AWS Database Migration Service

AWS Database Migration Service (AWS DMS) helps you migrate databases easily and securely. You can migrate your data to and from most widely used commercial and open-source databases. AWS DMS supports nearly all of today's most popular database engines as data sources, and provides a broad coverage of target engines. For more information about source and target endpoints, see Working with AWS DMS endpoints.

At the basic level, AWS DMS is a server in the AWS Cloud that runs replication software. You create a source and target connection to tell AWS DMS where to extract from and load to. Then you schedule a task that runs on this server to move your data. AWS DMS creates the tables and associated primary keys if they don’t exist on the target. You can precreate the target tables yourself if you prefer. Or you can use AWS Schema Conversion Tool (AWS SCT) to create some or all of the target tables, indexes, views, triggers, and so on.

The following diagram illustrates the AWS DMS replication process.


            Getting started with AWS DMS

To perform a database migration, you do the following:

  • Complete the steps in Setting up for AWS Database Migration Service.

  • Create a replication instance to perform all the processes for the migration.

  • Specify source and target database endpoints.

  • Create a task or set of tasks to define what tables and replication processes you want to use.

AWS DMS can then create your replication instance and perform the tasks on the data being migrated.

You begin a database migration in one of two ways:

  1. You can choose the AWS DMS console and perform each step there.

  2. You can use the AWS Command Line Interface (AWS CLI). For more information about using the CLI with AWS DMS, see AWS CLI for AWS DMS.

You can use the steps following to perform a database migration using the AWS DMS console.

Step 1: Create a replication instance using the AWS DMS console

To start work with AWS DMS, create a replication instance. A replication instance performs the actual data migration between source and target endpoints. Your instance needs enough storage and processing power to perform the tasks that migrate data from your source database to your target database. How large this replication instance should be depends on the amount of data to migrate and the tasks your instance needs to do. For more information about replication instances, see Working with an AWS DMS replication instance.

You can use the following procedure to create a replication instance with the AWS DMS console. Before you get started, make sure that you complete the steps in Setting up for AWS Database Migration Service.


                Create replication instance

To create a replication instance using the console

  1. Sign in to the AWS Management Console and open the AWS DMS console at https://console.aws.amazon.com/dms/v2/.

  2. At upper right, choose the AWS Region to create your replication instance in.

  3. In the navigation pane, choose Replication instances, and then choose Create replication instance.

  4. On the Create replication instance page, specify your replication instance configuration:

    1. For Name, enter a unique name for your replication instance.

      Your name can contain up to 63 printable ASCII characters (excluding /,", and @). The first character must be a letter, and the name can't end with a hyphen. The name should be unique for your AWS account and for the AWS Region that you chose.

    2. For Description, enter a short description for your replication instance.

    3. For Instance class, choose an instance class with the configuration that you need for your migration.

      The instance needs enough storage, networking, and processing power for your migration. For more information about how to choose an instance class, see Choosing the right AWS DMS replication instance for your migration.

    4. For Engine version, choose an AWS DMS version to run your replication instance. For this getting started exercise, accept the default.

    5. For Allocated storage (GiB), choose the amount of storage space that you want for your replication instance. The default is 50 GiB.

      In DMS, storage is mostly used by log files and cached transactions. For cache transactions, storage is used only when the cached transactions need to be written to disk. Thus, AWS DMS doesn't use a significant amount of storage.

    6. For VPC, choose a virtual private cloud (VPC) based on the Amazon Virtual Private Cloud (Amazon VPC) service. Your replication instance runs in this VPC.

      To help choose your VPC, consider the following:

      • If your source or your target database is in a VPC, choose that VPC.

      • If your source and your target databases are in different VPCs, make sure that they are both in public subnets. Also, make sure that both source and target are publicly accessible.

        After you do this, choose the VPC where you want your replication instance. Make sure that your replication instance can access the data in the source VPC.

      • If neither your source or target database is in a VPC, choose a VPC where the replication instance is to be located.

  5. Choose the Advanced security and network configuration tab to set values for network and encryption settings if you need them:

    1. For Replication subnet group, choose the replication subnet group in the VPC where you're creating your replication instance.

      If your source database is in a VPC, choose the subnet group that contains the source database as the location for your replication instance.

    2. For Availability zone, choose the Availability Zone (AZ) where you want your replication instance to run. The default is No preference, so AWS DMS determines which AZ to use. For this getting started exercise, keep the default.

    3. For VPC security group(s), choose a security group for your replication instance.

      The security group specifies inbound and outbound rules to control network access to your replication instance.

    4. For KMS master key(s), choose (Default) aws/dms to encrypt replication storage and connection information. (Default) aws/dms is the default AWS Key Management Service (AWS KMS) key associated with your account and AWS Region.

  6. Choose the Maintenance tab, and choose a weekly day and time range when system maintenance can occur, in Universal Coordinated Time (UTC).

    The default is a 30-minute window selected at random from an 8-hour block of time for each AWS Region, occurring on a random day of the week.

  7. Choose Create.

At this point, AWS DMS creates a replication instance to perform your migration.

Step 2: Specify source and target endpoints

While your replication instance is being created, you can specify the source and target data store endpoints. The source and target data stores can be on an Amazon Elastic Compute Cloud (Amazon EC2) instance. Or they can be on an Amazon Relational Database Service (Amazon RDS) DB instance or an on-premises database. You create each endpoint separately.


                Create endpoint

To specify a source endpoint and database endpoint using the AWS DMS console

  1. On the console, choose Endpoints from the navigation pane and then choose Create Endpoint.

  2. On the Create endpoint page, choose the Source endpoint type.

  3. If your data store is an Amazon RDS DB instance, choose the Select RDS DB instance option.

  4. In the Endpoint configuration section, enter a name for your endpoint for Endpoint identifier. For example, include the type of endpoint in the name, such as oracle-source or PostgreSQL-target.

  5. For Source engine, choose the type of database engine you want this endpoint to connect.

  6. Choose the Endpoint-specific settings tab. For now, keep Extra connection attributes empty.

    However, you can enter any extra connection attributes for your source endpoint. For more information about extra connection attributes for a source endpoint, see Sources for data migration.

  7. Choose the KMS master key tab, and choose (Default) aws/dms to encrypt replication storage and connection information. (Default) aws/dms is the default AWS Key Management Service (AWS KMS) key associated with your account and AWS Region.

  8. Choose the Test endpoint connection (optional) tab. For VPC, choose the Amazon VPC security group that your replication instance was created in.

    If your source database is in a VPC, choose the VPC security group that provides access to the DB instance where the database resides.

  9. For Replication instance, choose the name of the replication instance you previously entered on the Create replication instance page.

  10. Choose Run test.

    After you choose Run test, AWS DMS creates the endpoint with the details that you provided and connects to it. If the connection fails, edit the endpoint definition and test the connection again. You can also delete the endpoint manually.

  11. After you have a successful test, choose Create endpoint.

  12. Specify a target database endpoint using the AWS DMS console. To do this, repeat the steps preceding, but choose Target endpoint as your endpoint type.

When you're finished providing all information for your endpoints, AWS DMS creates your source and target endpoints for use during database migration.

Step 3: Create a task and migrate data

In this step, you create a task to specify what tables to migrate. Your task also maps data using a target schema and creates new tables for the target database.

As part of creating a task, you can choose the type of migration to perform:

  • Migrate existing data—full load only.

  • Migrate existing data and replicate ongoing changes—full load + change data capture (CDC).

  • Replicate data changes only—CDC only.

You can start your task as soon as you finish specifying information for it on the Create task page. Or you can start the task from the navigation pane after you specify task information.


                Create migration task

To create a migration task and start your database migration

  1. In the console navigation pane, choose Database migration tasks, and then choose Create task. The Create database migration task page opens.

  2. In the Task configuration section, specify the following task options:

    • Task identifier – Enter a unique name for the task.

    • Replication instance – Choose your replication instance to use.

    • Source database endpoint – Choose your source database endpoint to use.

    • Target database endpoint – Choose your target database endpoint to use.

    • Migration type – Choose Migrate existing data as the migration method that you want to use for this getting started exercise.

      Other practical options include Migrate existing data and replicate on-going changes and Replicate data changes only.

  3. Choose the Task settings tab.

    The task settings that appear depend on the migration type. Because we choose Migrate existing data as our migration type, do the following:

    1. For Target table preparation mode, choose one of the following:

      • Do nothing – Data and metadata of the target tables aren't changed.

      • Drop tables on target – Tables are dropped, and new tables are created in their place.

      • Truncate – Tables are truncated without affecting table metadata.

    2. For Include LOB columns in replication, choose one of the following:

      • Don't include LOB columns – LOB columns are excluded from the migration.

      • Full LOB mode – Migrate complete LOBs regardless of size. LOBs are migrated piecewise in chunks controlled by the LOB chunk size. This method is slower than using limited LOB mode.

      • Limited LOB mode – Truncate LOBs to the maximum LOB size. This method is faster than using full LOB mode.

    3. For Maximum LOB size (KB), set a maximum size for LOBs in AWS DMS.

  4. Choose the Advanced task settings tab to specify table and tuning settings:

    1. For Create control table in target using schema, enter the name of the target database schema where AWS DMS can create its control tables.

    2. For History timeslot (minutes), enter 5 as the number of minutes that you want each history record to stay in the replication history table.

    3. In the Control table settings section, turn Enable control table settings on or off. For Apply Exceptions, awsdms_apply_exceptions is always enabled.

    4. For Full load tuning, specify values for Maximum number of tables to load in parallel, Transaction consistency timeout, and Commit rate during full load.

  5. Choose the Migration task startup configuration tab, and then choose Automatically on create.

  6. Choose Create task.

If you chose Automatically on create, your task begins immediately to migrate your data when you choose Create task. If you didn't, start your task from the Database migration tasks page. On that page, choose your task, and then choose Start for Actions.

Step 4: Monitor your task

Now that your migration task is running, you can monitor the progress of your database migration while it happens.

To view migration task metrics

  1. In the DMS console navigation pane, choose Database migration tasks.

  2. Choose the name of the running task that you want to monitor.

  3. Choose Table statistics.

For more information about monitoring, see Monitoring AWS DMS tasks.

Step 5: Deleting AWS DMS components using the console

After you complete the getting started exercise, you can delete your database migration components when you no longer need them as reference. You can use the AWS DMS console to remove them. Make sure to delete the migration tasks before deleting the replication instance and endpoints.

To delete a migration task using the console

  1. In the AWS DMS console navigation pane, choose Database migration tasks.

  2. Choose a migration task that is no longer wanted.

  3. For Actions, choose Delete.

  4. From the Delete database migration tasks dialog box, choose Delete.

AWS DMS deletes the migration task and removes it from the Database migration tasks page.

To delete a replication instance using the console

  1. In the AWS DMS console navigation pane, choose Replication instances.

  2. Choose the replication instance to delete.

  3. For Actions, choose Delete.

  4. From the Delete replication instance dialog box, choose Delete.

AWS DMS deletes the replication instance and removes it from the Replication instances page.

To remove obsolete endpoints using the console

  1. In the AWS DMS console navigation pane, choose Endpoints.

  2. Choose the endpoint to delete.

  3. For Actions, choose Delete.

  4. From the Delete endpoint dialog box, choose Delete.

AWS DMS deletes the endpoint and removes it from the Endpoints page.

Additional resources

The remaining sections in this user guide describe how to use AWS DMS to migrate your data to and from the most widely used commercial and open-source databases. We also recommend that you check the following resources as you prepare and perform a database migration project: