Migrate a relational database to MongoDB Atlas on AWS - AWS Prescriptive Guidance

Migrate a relational database to MongoDB Atlas on AWS

Created by Battulga Purevragchaa (AWS), Babu Srinivasan (MongoDB), and Igor Alekseev (AWS)

Summary

This pattern describes the steps for migrating from a relational database such as SQL Server, MySQL, or PostgreSQL to MongoDB Atlas in the AWS Cloud. It uses MongoDB Relational Migrator to help accelerate the data migration from relational databases to MongoDB Atlas.

The pattern accompanies the guide Migrating to MongoDB Atlas on AWS on the AWS Prescriptive Guidance website. It provides the implementation steps for one of the migration scenarios that are discussed in that guide. For additional migration scenarios, see the following patterns on the AWS Prescriptive Guidance website:

The pattern is intended for AWS System Integrator (SI) Partners and AWS users.

Prerequisites and limitations

Prerequisites

  • A source relational database (Oracle Database, SQL Server, PostgreSQL, MySQL, SAP/Sybase ASE, and others) to migrate to MongoDB Atlas.

  • Familiarity with relational databases, MongoDB Atlas, and AWS services. This pattern explains some of the migration steps at a high level. Additional details will be added in future versions.

Product versions

  • MongoDB version 5.0 or later

Architecture

The following diagram shows the migration from a relational database management system (RDBMS) database to MongoDB Atlas on AWS.

Architecture for migrating from RDBMS to MongoDB Atlas on AWS.

For MongoDB Atlas reference architectures that support different usage scenarios, see Migrating to MongoDB Atlas on AWS on the AWS Prescriptive Guidance website.

Tools

  • MongoDB Atlas is a fully managed database as a service (DBaaS) for deploying and managing MongoDB databases in the cloud.

  • MongoDB Relational Migrator provides a smooth transition of data from traditional relational databases to MongoDB. It helps automate the conversion process and transforms the structured data model of relational databases into the flexible document format provided by MongoDB. The Relational Migrator preserves data integrity and relationships to simplify the migration. Organizations can take advantage of the scalability, performance, and versatility benefits that MongoDB provides while retaining the familiarity of their existing data.

Best practices

For best practices for using MongoDB on AWS, see the posts on the AWS Partner Network Blog.

Epics

TaskDescriptionSkills required

Determine the parameters and size of the relational database.

Estimate the working set size by using the Relational Migrator recommendations and information from db.stats() for the total index space. Assume that a percentage of your data space will be accessed frequently. This task should take approximately one week. For more information and examples for this and the other stories in this epic, see the Related resources section.

App owner, DBA

Estimate network bandwidth requirements.

To estimate your network bandwidth requirements, multiply the average document size by the number of documents served per second. Consider the maximum traffic that any node on your cluster will bear as the basis. To calculate downstream data transfer rates from your cluster to client applications, use the sum of the total documents returned over a period of time. If your applications read from secondary nodes, divide this number of total documents by the number of nodes that can serve read operations. To find the average document size for a database, use the db.stats().avgObjSize command. This task will typically take one day.

DBA

Select the Atlas tier.

Follow the instructions in the MongoDB documentation to select the correct Atlas cluster tier.

DBA

Plan for cutover.

Plan for application cutover.

DBA, App owner
TaskDescriptionSkills required

Create a new MongoDB Atlas cluster on AWS.

In MongoDB Atlas, choose Build a Cluster. In the Create New Cluster dialog box, select AWS as the cloud provider.

DBA

Select AWS Regions and global cluster configuration.

Select from the list of available AWS Regions for your Atlas cluster. Configure global clusters if required.

DBA

Select the cluster tier.

Select your preferred cluster tier. Your tier selection determines factors such as memory, storage, and IOPS specification.

DBA

Configure additional cluster settings.

Configure additional cluster settings such as MongoDB version, backup, and encryption options. For more information about these options, see the Related resources section.

DBA
TaskDescriptionSkills required

Configure the access list.

To connect to the Atlas cluster, you must add an entry to the project’s access list. Atlas uses TLS/SSL to encrypt the connections to the virtual private cloud (VPC) for your database. To set up the access list for the project and for more information about the stories in this epic, see the Related resources section.

DBA

Authenticate and authorize users.

You must create and authenticate the database users who will access the MongoDB Atlas clusters. To access clusters in a project, users must belong to that project, and they can belong to multiple projects.

DBA

Create custom roles.

(Optional) Atlas supports creating custom roles in cases where the built-in Atlas database user permissions don’t cover the set of permissions you want.

DBA

Set up VPC peering.

(Optional) Atlas supports VPC peering with other VPCs on AWS.

AWS administrator

Set up an AWS PrivateLink endpoint.

(Optional) You can set up private endpoints on AWS by using AWS PrivateLink. For more information, see the Amazon VPC documentation.

AWS administrator

Enable two-factor authentication.

(Optional) Atlas supports two-factor authentication (2FA) to help users control access to their Atlas accounts.

AWS administrator

Set up user authentication and authorization with LDAP.

(Optional) Atlas supports performing user authentication and authorization with Lightweight Directory Access Protocol (LDAP).

DBA

Set up unified AWS access.

(Optional) Some Atlas features, including Atlas Data Lake and encryption at rest using customer key management, use AWS Identity and Access Management (IAM) roles for authentication.

AWS administrator

Set up encryption at rest using AWS KMS.

(Optional) Atlas supports using AWS Key Management Service (AWS KMS) to encrypt storage engines and cloud provider backups.

AWS administrator

Set up client-side field-level encryption.

(Optional) Atlas supports client-side field level encryption, including automatic encryption of fields.

AWS administrator
TaskDescriptionSkills required

Add MongoDB Relational Migrator to the access list.

Add the Relational Migrator to the access list for your source database. This helps prepare the source environment to connect to the target Atlas cluster.

DBA

Assess the relational database objects.

Launch MongoDB Relational Migrator and connect to your relational database. Start the assessment.

DBA

Accept the migration patterns or choose to alter them based on your business needs.

Either accept the database patterns recommended by the Relational Migrator based on the initial assessment and performance parameters, or choose to alter them based on your business requirements.

DBA

Launch your target replica set in MongoDB Atlas.

Launch your target replica set in MongoDB Atlas. In Relational Migrator, choose I'm ready to migrate.

DBA
TaskDescriptionSkills required

Connect to the MongoDB Atlas cluster.

Make sure that MongoDB Atlas cluster connectivity works as expected.

App owner

Interact with cluster data.

Verify the cluster data.

DBA

Monitor your clusters.

Verify that your clusters are set up correctly.

DBA

Back up and restore cluster data.

Schedule backups with a regular cadence for cluster data.

DBA

Related resources

All of the following links, unless noted otherwise, go to webpages in the MongoDB documentation.

Migration guide

Discovery and assessment

Configuring security and compliance

Setting up a new MongoDB Atlas environment on AWS

Migrating data

Monitoring clusters

Integrating operations

Blog posts