AWS Database Migration Service
User Guide (Version API Version 2016-01-01)

Walkthrough: Migrating from MongoDB to Amazon DocumentDB

Use the following walkthrough to guide you through the process of migrating from MongoDB to Amazon DocumentDB (with MongoDB compatibility). In this walkthrough, you do the following:

  • Install MongoDB on an Amazon EC2 instance.

  • Populate MongoDB with sample data.

  • Create an AWS DMS replication instance, a source endpoint (for MongoDB), and a target endpoint (for Amazon DocumentDB).

  • Run an AWS DMS task to migrate the data from the source endpoint to the target endpoint.

Important

Before you begin, make sure to launch an Amazon DocumentDB cluster in your default virtual private cloud (VPC). For more information, see Getting Started in the Amazon DocumentDB Developer Guide.

Step 1: Launch an Amazon EC2 Instance

For this walkthrough, you launch an Amazon EC2 instance into your default VPC.

To launch an Amazon EC2 instance

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. Choose Launch Instance, and do the following:

    1. On the Choose an Amazon Machine Image (AMI) page, at the top of the list of AMIs, go to Amazon Linux AMI and choose Select.

    2. On the Choose an Instance Type page, at the top of the list of instance types, choose t2.micro. Then choose Next: Configure Instance Details.

    3. On the Configure Instance Details page, for Network, choose your default VPC. Then choose Next: Add Storage.

    4. On the Add Storage page, skip this step by choosing Next: Add Tags.

    5. On the Add Tags page, skip this step by choosing Next: Configure Security Group.

    6. On the Configure Security Group page, do the following:

      1. Choose Select an existing security group.

      2. In the list of security groups, choose default. Doing this chooses the default security group for your VPC. By default, the security group accepts inbound Secure Shell (SSH) connections on TPC port 22. If this isn't the case for your VPC, add this rule; for more information, see What Is Amazon VPC? in the Amazon VPC User Guide.

      3. Choose Next: Review and Launch.

    7. Review the information, and choose Launch.

  3. In the Select an existing key pair or create a new key pair window, do one of the following:

    • If you don't have an Amazon EC2 key pair, choose Create a new key pair and follow the instructions. You are asked to download a private key file (.pem file). You need this file later when you log in to your Amazon EC2 instance.

    • If you already have an Amazon EC2 key pair, for Select a key pair choose your key pair from the list. You must already have the private key file (.pem file) available in order to log in to your Amazon EC2 instance.

  4. After you configure your key pair, choose Launch Instances.

    In the console navigation pane, choose EC2 Dashboard, and then choose the instance that you launched. In the lower pane, on the Description tab, find the Public DNS location for your instance, for example: ec2-11-22-33-44.us-west-2.compute.amazonaws.com.

    It takes a few minutes for your Amazon EC2 instance to become available.

  5. Use the ssh command to log in to your Amazon EC2 instance, as in the following example.

    chmod 400 my-keypair.pem ssh -i my-keypair.pem ec2-user@public-dns-name

    Specify your private key file (.pem file) and the public DNS name of your EC2 instance. The login ID is ec2-user. No password is required.

    For further details about connecting to your EC instance, see Connecting to Your Linux Instance Using SSH in the Amazon EC2 User Guide for Linux Instances.

Step 2: Install and Configure MongoDB Community Edition

Perform these steps on the Amazon EC2 instance that you launched in Step 1: Launch an Amazon EC2 Instance.

To install and configure MongoDB Community Edition on your EC2 instance

  1. Go to Install MongoDB Community Edition on Amazon Linux in the MongoDB documentation and follow the instructions there.

  2. By default, the MongoDB server (mongod) only allows loopback connections from IP address 127.0.0.1 (localhost). To allow connections from elsewhere in your Amazon VPC, do the following:

    1. Edit the /etc/mongod.conf file and look for the following lines.

      # network interfaces net: port: 27017 bindIp: 127.0.0.1 # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
    2. Modify the bindIp line so that it looks like the following.

      bindIp: public-dns-name
    3. Replace public-dns-name with the actual public DNS name for your instance, for example ec2-11-22-33-44.us-west-2.compute.amazonaws.com.

    4. Save the /etc/mongod.conf file, and then restart mongod.

      sudo service mongod restart
  3. Populate your MongoDB instance with data by doing the following:

    1. Use the wget command to download a JSON file containing sample data.

      wget http://media.mongodb.org/zips.json
    2. Use the mongoimport command to import the data into a new database (zips-db).

      mongoimport --host public-dns-name:27017 --db zips-db --file zips.json
    3. After the import completes, use the mongo shell to connect to MongoDB and verify that the data was loaded successfully.

      mongo --host public-dns-name:27017
    4. Replace public-dns-name with the actual public DNS name for your instance.

    5. At the mongo shell prompt, enter the following commands.

      use zips-db db.zips.count() db.zips.aggregate( [ { $group: { _id: { state: "$state", city: "$city" }, pop: { $sum: "$pop" } } }, { $group: { _id: "$_id.state", avgCityPop: { $avg: "$pop" } } } ] )

      The output should display the following:

      • The name of the database (zips-db)

      • The number of documents in the zips collection (29353)

      • The average population for cities in each state

    6. Exit from the mongo shell and return to the command prompt by using the following command.

      exit

Step 3: Create an AWS DMS Replication Instance

To perform replication in AWS DMS, you need a replication instance.

To create an AWS DMS replication instance

  1. Open the AWS DMS console at https://console.aws.amazon.com/dms/.

  2. In the navigation pane, choose Replication instances.

  3. Choose Create replication instance and enter the following information:

    • For Name, enter mongodb2docdb.

    • For Description, enter MongoDB to Amazon DocumentDB replication instance.

    • For Instance class, keep the default value.

    • For Engine version, keep the default value.

    • For VPC, choose your default VPC.

    • For Multi-AZ, choose No.

    • For Publicly accessible, enable this option.

    When the settings are as you want them, choose Create replication instance.

Note

You can begin using your replication instance when its status becomes available. This can take several minutes.

Step 4: Create Source and Target Endpoints

The source endpoint is the endpoint for your MongoDB installation running on your Amazon EC2 instance.

To create a source endpoint

  1. Open the AWS DMS console at https://console.aws.amazon.com/dms/.

  2. In the navigation pane, choose Endpoints.

  3. Choose Create endpoint and enter the following information:

    • For Endpoint type, choose Source.

    • For Endpoint identifier, enter a name that's easy to remember, for example mongodb-source.

    • For Source engine, choose mongodb.

    • For Server name, enter the public DNS name of your Amazon EC2 instance, for example ec2-11-22-33-44.us-west-2.compute.amazonaws.com.

    • For Port, enter 27017.

    • For SSL mode, choose none.

    • For Authentication mode, choose none.

    • For Database name, enter zips-db.

    • For Authentication mechanism, choose default.

    • For Metadata mode, choose document.

    When the settings are as you want them, choose Create endpoint.

Next, you create a target endpoint. This endpoint is for your Amazon DocumentDB cluster, which should already be running. For more information on launching your Amazon DocumentDB cluster, see Getting Started in the Amazon DocumentDB Developer Guide.

Important

Before you proceed, do the following:

  • Have available the master user name and password for your Amazon DocumentDB cluster.

  • Have available the DNS name and port number of your Amazon DocumentDB cluster, so that AWS DMS can connect to it. To determine this information, use the following AWS CLI command, replacing cluster-id with the name of your Amazon DocumentDB cluster.

    aws docdb describe-db-clusters \ --db-cluster-identifier cluster-id \ --query "DBClusters[*].[Endpoint,Port]"
  • Download a certificate bundle that Amazon DocumentDB can use to verify SSL connections. To do this, enter the following command. Here, aws-api-domain completes the Amazon S3 domain in your region required to access the specified S3 bucket and the rds-combined-ca-bundle.pem file that it provides.

    wget https://s3.aws-api-domain/rds-downloads/rds-combined-ca-bundle.pem

To create a target endpoint

  1. In the navigation pane, choose Endpoints.

  2. Choose Create endpoint and enter the following information:

    • For Endpoint type, choose Target.

    • For Endpoint identifier, enter a name that's easy to remember, for example docdb-target.

    • For Target engine, choose docdb.

    • For Server name, enter the DNS name of your Amazon DocumentDB cluster.

    • For Port, enter the port number of your Amazon DocumentDB cluster.

    • For SSL mode, choose verify-full.

    • For CA certificate, choose Add new CA certificate, and then for Certificate Identifier, enter rds-combined-ca-bundle. For Import file, navigate to the rds-combined-ca-bundle.pem file that you downloaded. When you are finished, choose Add new CA certificate.

    • For User name, enter the master user name of your Amazon DocumentDB cluster.

    • For Password, enter the master password of your Amazon DocumentDB cluster.

    • For Database name, enter zips-db.

    When the settings are as you want them, choose Create endpoint.

Now that you've created the source and target endpoints, test them to ensure that they work correctly. Also, to ensure that AWS DMS can access the database objects at each endpoint, refresh the endpoints' schemas.

To test an endpoint

  1. In the navigation pane, choose Endpoints.

  2. Choose the source endpoint (mongodb-source), and then choose Test connection.

  3. Choose your replication instance (mongodb2docdb), and then choose Run test. It takes a few minutes for the test to complete, and for the Status to change to successful.

    If the Status changes to failed instead, review the failure message. Correct any errors that might be present, and test the endpoint again.

Note

Repeat this procedure for the target endpoint (docdb-target).

To refresh schemas

  1. In the navigation pane, choose Endpoints.

  2. Choose the source endpoint (mongodb-source), and then choose Refresh schemas.

  3. Choose your replication instance (mongodb2docdb), and then choose Refresh schemas.

Note

Repeat this procedure for the target endpoint (docdb-target).

Step 5: Create and Run a Migration Task

You are now ready to launch an AWS DMS migration task, to migrate the zips data from MongoDB to Amazon DocumentDB.

  1. Open the AWS DMS console at https://console.aws.amazon.com/dms/.

  2. In the navigation pane, choose Tasks.

  3. Choose Create task and enter the following information:

    In the Task Settings section, keep all of the options at their default values.

    In the Table mappings section, choose the Guided tab, and then enter the following information:

    • For Schema name is, choose Enter a schema.

    • For Schema name is like, keep this at its default setting (%).

    • For Table name is like, keep this at its default setting (%).

    Choose Add selection rule to confirm that the information is correct.

    When the settings are as you want them, choose Create task.

AWS DMS now begins migrating data from MongoDB to Amazon DocumentDB. The task status changes from Starting to Running. You can monitor the progress by choosing Tasks in the AWS DMS console. After several minutes, the status changes to Load complete.

Note

After the migration is complete, you can use the mongo shell to connect to your Amazon DocumentDB cluster and view the zips data. For more information, see Access Your Amazon DocumentDB Cluster Using the mongo Shell in the Amazon DocumentDB Developer Guide.