Build an environment to upgrade, migrate, and compare OpenSearch clusters
Publication date: November 2023 (last update: September 2024)
OpenSearch is widely adopted for log analytics and search functionalities. However, self-managing OpenSearch can be operationally demanding. Amazon OpenSearch Service and Amazon OpenSearch Service Serverless offer more manageable alternatives, but transitioning to these services or updating to the latest OpenSearch version has historically been complex. Also, it can be difficult for a customer to predict the outcome of a migration. The Migration Assistant for Amazon OpenSearch Service solution addresses these challenges - it simplifies the migration process, ensures integrity, and validates performance post-migration.
The Migration Assistant for Amazon OpenSearch Service solution is a toolkit designed to ease the transition to OpenSearch, facilitate upgrades to the latest OpenSearch versions, and refine cluster configurations based on observed traffic patterns. Whether you're looking to set up a proof-of-concept in AWS, transition production workloads with confidence, or enhance your current OpenSearch clusters, this guide provides references to step-by-step instructions, best practices, and insights to leverage the full potential of the OpenSearch migrations package.
Benefits of using this solution:
-
Migrate cluster metadata, including index settings, type mappings, index templates, and aliases.
-
Migrate existing data from legacy clusters to OpenSearch clusters, including Amazon OpenSearch Service (AOS) Domains and Amazon OpenSearch Service Serverless (AOSS) collections.
-
Intercept and redirect live traffic from self-managed Elasticsearch or OpenSearch clusters to Amazon OpenSearch Service domains and Amazon OpenSearch Service Serverless collections with minimal latency.
-
Replicate production traffic on target clusters to validate and ensure accuracy.
-
Simulate real-world traffic by capturing and replaying request patterns to fine-tune system performance.
-
Deploy across the most common AWS Regions for global reach and scalability.
-
Provides a recommended path for migration while continuing to maintain service availability.
Note
The Migration Assistant for Amazon OpenSearch Service solution supports migrating from Elasticsearch versions 6.x and 7.x, and OpenSearch 1.x and 2.x. The following migrations paths are tested regularly:
Elasticsearch 6.8 to OpenSearch 2.14
Elasticsearch 7.10.2 to OpenSearch 2.14
Elasticsearch 7.17 to OpenSearch 2.14
OpenSearch 1.3 to OpenSearch 2.14
We expect other minor versions within these major versions to work. If you prefer another migration path or find a bug with an existing path, open a feature request or bug accordingly.
This implementation guide provides an overview of the Migration
Assistant for Amazon OpenSearch Service solution, its reference
architecture and components, considerations for planning the
deployment, and configuration steps for deploying the solution to
the Amazon Web Services (AWS) Cloud. It also references the
solution's open-source documentation on
GitHub
The intended audience for using this solution's features and capabilities in their environment includes solution architects, business decision makers, DevOps engineers, data scientists, and cloud professionals.
Use this navigation table to quickly find answers to these questions:
| If you want to . . . | Read . . . |
|---|---|
|
Know the cost for running this solution. The estimated cost for running this solution in the US East (N. Virginia) Region is approximately USD $3,096 for a 15-day migration with 100 TB of existing data and 15 MBps of live traffic. |
Cost |
| Understand the security considerations for this solution. | Security |
| Know how to plan for quotas for this solution. | Quotas |
| Know which AWS Regions support this solution. | Supported AWS Regions |
| View or download the AWS CloudFormation template included in this solution to automatically deploy the infrastructure resources (the "stack") for this solution. | AWS CloudFormation template |
| Access the source code and optionally use the AWS Cloud Development Kit (AWS CDK) to deploy the solution. | GitHub repository |
