Migrate a self-hosted MongoDB environment to MongoDB Atlas on the AWS Cloud
Created by Suresh Veeragoni (AWS)
Source: MongoDB | Target: MongoDB Atlas on AWS | R Type: Replatform |
Environment: Production | Technologies: Migration; Analytics; Databases | Workload: All other workloads |
AWS services: Amazon EC2; Amazon VPC |
Summary
This pattern describes the steps for migrating from a self-managed MongoDB environment (including MongoDB Community Server, Enterprise Server, Enterprise Advanced, mLab, or any managed MongoDB cluster) to MongoDB Atlas on the Amazon Web Services (AWS) Cloud. It uses the Atlas Live Migration Service
The pattern accompanies the guide Migrating from MongoDB to MongoDB Atlas on the AWS Cloud on the AWS Prescriptive Guidance website. It provides the implementation steps for the migration.
The pattern is intended for AWS Service Integrator Partners (SI Partners) and AWS users.
Prerequisites and limitations
Prerequisites
A source MongoDB environment to migrate to MongoDB Atlas
Expertise
This pattern requires familiarity with MongoDB, MongoDB Atlas, and AWS services. For more information, see Roles and responsibilities in the guide Migrating from MongoDB to MongoDB Atlas on the AWS Cloud on the AWS Prescriptive Guidance website.
Product versions
MongoDB version 2.6 or later
Architecture
For MongoDB Atlas reference architectures that support different usage scenarios, see MongoDB Atlas reference architectures on AWS in the guide Migrating from MongoDB to MongoDB Atlas on the AWS Cloud on the AWS Prescriptive Guidance website.
Tools
Atlas Live Migration Service
– A free MongoDB utility that helps migrate databases to Atlas. This service keeps the source database in sync with the destination database until cutover. When you’re ready to cut over, you stop your application instances, point them to the destination Atlas cluster, and restart them.
Epics
| Task | Description | Skills required |
|---|---|---|
| Determine the cluster size. | Estimate the working set size by using the information from db.stats() for the total index space. Assume that a percentage of your data space will be accessed frequently. Or, you can estimate your memory requirements based on your own assumptions. This task should take approximately one week. For more information and examples for this and the other stories in this epic, see the links in the “Related resources” section. | MongoDB DBA, Application architect |
| Estimate network bandwidth requirements. | To estimate your network bandwidth requirements, multiply the average document size by the number of documents served per second. Consider the maximum traffic that any node on your cluster will bear as the basis. To calculate downstream data transfer rates from your cluster to client applications, use the sum of the total documents returned over a period of time. If your applications read from secondary nodes, divide this number of total documents by the number of nodes that can serve read operations. To find the average document size for a database, use the db.stats().avgObjSize command. This task will typically take one day. | MongoDB DBA |
| Select the Atlas tier. | Follow the instructions in the MongoDB documentation to select the correct Atlas cluster tier. | MongoDB DBA |
| Plan for application cutover. | MongoDB DBA, Application architect |
| Task | Description | Skills required |
|---|---|---|
| Create a new MongoDB Atlas cluster on AWS. | In MongoDB Atlas, choose “Build a Cluster” to display the “Create New Cluster” dialog box. Select AWS as the cloud provider. | MongoDB DBA |
| Select Regions and global cluster configuration. | Select from the list of available AWS Regions for your Atlas cluster. Configure global clusters if required. | MongoDB DBA |
| Select the cluster tier. | Select your preferred cluster tier. Your tier selection determines factors such as memory, storage, and IOPS specification. | MongoDB DBA |
| Configure additional cluster settings. | Configure additional cluster settings such as MongoDB version, backup, and encryption options. For more information about these options, see the links in the “Related resources” section. | MongoDB DBA |
| Task | Description | Skills required |
|---|---|---|
| Configure the access list. | To connect to the Atlas cluster, you must add an entry to the project’s access list. Atlas uses Transport Layer Security (TLS) / Secure Sockets Layer (SSL) to encrypt the connections to the virtual private cloud (VPC) for your database. To set up the access list for the project and for more information about the stories in this epic, see the links in the “Related resources” section. | MongoDB DBA |
| Authenticate and authorize users. | You must create and authenticate the database users who will access the MongoDB Atlas clusters. To access clusters in a project, users must belong to that project, and they can belong to multiple projects. | MongoDB DBA |
| Create custom roles. | (Optional) Atlas supports creating custom roles in cases where the built-in Atlas database user privileges don’t cover your desired set of privileges. | MongoDB DBA |
| Set up VPC peering. | (Optional) Atlas supports VPC peering with other AWS, Azure, or Google Cloud Platform (GCP) VPCs. | MongoDB DBA |
| Set up an AWS PrivateLink endpoint. | (Optional) You can set up private endpoints on AWS by using AWS PrivateLink. | MongoDB DBA |
| Enable two-factor authentication. | (Optional) Atlas supports two-factor authentication (2FA) to help users control access to their Atlas accounts. | MongoDB DBA |
| Set up user authentication and authorization with LDAP. | (Optional) Atlas supports performing user authentication and authorization with Lightweight Directory Access Protocol (LDAP). | MongoDB DBA |
| Set up unified AWS access. | (Optional) Some Atlas features, including Atlas Data Lake and encryption at rest using customer key management, use AWS Identity and Access Management (AWS IAM) roles for authentication. | MongoDB DBA |
| Set up encryption at rest using AWS KMS. | (Optional) Atlas supports using AWS Key Management System (AWS KMS) to encrypt storage engines and cloud provider backups. | MongoDB DBA |
| Set up client-side field-level encryption. | (Optional) Atlas supports client-side field level encryption, including automatic encryption of fields. | MongoDB DBA |
| Task | Description | Skills required |
|---|---|---|
| Launch your target replica set in MongoDB Atlas. | Launch your target replica set in MongoDB Atlas. In Atlas Live Migration Service, choose “I'm ready to migrate.” | MongoDB DBA |
| Add the Atlas Live Migration Service to the access list in your AWS source cluster. | This helps prepare the source environment to connect to the target Atlas cluster. | MongoDB DBA |
| Validate your AWS credentials with Atlas Live Migration Service. | Choose “Start migration.” When the “Prepare to Cutover” button turns green, perform the cutover. Review Atlas cluster performance metrics. | MongoDB DBA |
| Task | Description | Skills required |
|---|---|---|
| Connect to the MongoDB Atlas cluster. | Application developer | |
| Interact with cluster data. | Application developer | |
| Monitor your clusters. | MongoDB DBA | |
| Back up and restore cluster data. | MongoDB DBA |
Related resources
Migration guide
Discovery and assessment
Configuring security and compliance
Setting up a new MongoDB Atlas environment on AWS
Migrating data
Integrating operations
