Menu
AWS Database Migration Service
User Guide (Version API Version 2016-01-01)

AWS DMS Components

The components you work with when using AWS DMS include the following:

Replication instance

An AWS DMS replication instance runs on an Amazon Elastic Compute Cloud (Amazon EC2) instance. The replication instance provides high-availability and failover support using a Multi-AZ deployment. In a Multi-AZ deployment, AWS DMS automatically provisions and maintains a synchronous standby replica of the replication instance in a different Availability Zone. The primary replication instance is synchronously replicated across Availability Zones to a standby replica. This approach provides data redundancy, eliminate I/O freezes, and minimize latency spikes during system backups.

AWS DMS uses a replication server that connects to the source data store, reads the source data, formats the data for consumption by the target data store, and loads the data into the target data store. Most of this processing happens in memory. However, large transactions might require some buffering on disk. Cached transactions and log files are also written to disk. When creating your replication server, you should consider the following:

  • EC2 instance class — Some of the smaller EC2 instance classes are sufficient for testing the service or for small migrations. If your migration involves a large number of tables, or if you intend to run multiple concurrent replication tasks, you should consider using one of the larger instances. We recommend this approach because AWS DMS consumes a fair amount of memory and CPU.

  • Storage — Depending on the EC2 instance class you select, your replication server comes with either 50 GB or 100 GB of data storage. This storage is used for log files and any cached changes collected during the load. If your source system is busy or takes large transactions, or if you’re running multiple tasks on the replication server, you might need to increase this amount of storage. Usually the default amount is sufficient.

Endpoints

An endpoint provides connection, data store type, and location information about your data store. AWS DMS uses this information to connect to a data store and migrate data from a source endpoint to a target endpoint. You can specify additional connection attributes for an endpoint by using extra connection attributes. These attributes can control logging, file size, and other parameters; for more information about extra connection attributes, see Using Extra Connection Attributes with AWS Database Migration Service or see the documentation section for your data store. For a list of supported source and target data stores, see Sources for AWS Database Migration Service and Targets for AWS Database Migration Service.

Tasks

An AWS DMS task is where all the work happens. You use tasks to migrate data from the source endpoint to the target endpoint, and the task processing is done on the replication instance. You specify what tables and schemas to use for your migration and any special processing, such as logging requirements, control table data, and error handling.

You can create one of three possible types of migration tasks:

  • Migrate existing data — If you can afford an outage long enough to copy your existing data, this option is a good one to choose. This option simply migrates the data from your source database to your target database, creating tables when necessary.

  • Migrate existing data and replicate ongoing changes — This option performs a full data load while capturing changes on the source. Once the full load is complete, captured changes are applied to the target. Eventually the application of changes reaches a steady state. At this point you can shut down your applications, let the remaining changes flow through to the target, and then restart your applications pointing at the target.

  • Replicate data changes only — In some situations it might be more efficient to copy existing data using a method other than AWS DMS. For example, in a homogeneous migration, using native export/import tools might be more efficient at loading the bulk data. In this situation, you can use AWS DMS to replicate changes starting when you start your bulk load to bring and keep your source and target databases in sync

By default AWS DMS starts your task as soon as you create it. However, in some situations, you might want to postpone the start of the task. For example, when using the AWS Command Line Interface (AWS CLI), you might have a process that creates a task and a different process that starts the task based on some triggering event. As needed, you can postpone your task's start.

Ongoing replication (Change data capture (CDC))

AWS DMS can be used to capture ongoing changes to the data store while you are migrating you data. The change capture process that AWS DMS uses when replicating ongoing changes from a source endpoint collects changes to the database logs by using the database engine's native API. Each source engine has specific configuration requirements for exposing this change stream to a given user account. Most engines require some additional configuration to make the change data consumable in a meaningful way, without data loss, for the capture process. For example, Oracle requires the addition of supplemental logging, and MySQL requires row-level bin logging. When using Amazon RDS as a source, we recommend ensuring that backups are enabled and that the source database is configured to retain change logs for a sufficient time (24 hours is usually enough).

Schema and code migration

AWS DMS doesn't perform schema or code conversion. You can use tools such as Oracle SQL Developer, MySQL Workbench, or pgAdmin III to move your schema if your source and target are the same database engine. If you want to convert an existing schema to a different database engine, you can use the AWS Schema Conversion Tool (SCT). It can create a target schema and also can generate and create an entire schema: tables, indexes, views, and so on. You can also use the AWS SCT to convert PL/SQL or TSQL to PgSQL and other formats. For more information on the AWS SCT, see AWS Schema Conversion Tool.

Whenever possible, AWS DMS attempts to create the target schema for you. Sometimes, AWS DMS can't create the schema—for example, AWS DMS won't create a target Oracle schema for security reasons. For MySQL database targets, you can use extra connection attributes to have AWS DMS migrate all objects to the specified database and schema or create each database and schema for you as it finds the schema on the source.