Migration Assistant for Amazon OpenSearch Service

Flexible multi-version workload migration with rollback capability and no downtime

Overview

Migration Assistant for Amazon OpenSearch Service assists you in migrating or upgrading your Solr, Elasticsearch and OpenSearch workloads to Amazon OpenSearch Service Service or Amazon OpenSearch Service Serverless. This AWS Solution automates manual tasks with a low-risk and prescriptive migration path for existing and live data. It also includes advanced features, such as a metadata migration tool and capture and replay comparison tooling to help you identify potential migration and upgrade issues earlier. The migration process is streamlined, performance and behavioral comparisons based on real customer workloads are enabled, and the pre-migration, migration, and validation phases are accelerated.

This solution prescribes a systematic migration workflow to upgrade, migrate, recover, and modify an OpenSearch Service cluster. The workflow includes a migration console command line interface (CLI) for management, a dedicated scaling group for existing data backfill, and a replayer to synchronize live traffic between source and target clusters. Users can pause or abort the migration without affecting production traffic, thereby reducing risk. Additionally, the backfill functionality minimizes further risk by retrieving data from a snapshot, leaving the source cluster unaffected, and supporting multi-hop migrations, which decreases the overall number of migrations required.

Benefits

Simplified management experience

Transfer data from an originating (source) cluster to a designated target (OpenSearch Service cluster).

Adaptable, low-risk migration

Safely capture and replay traffic on source and target clusters to identify optimal performance while reducing migration risk through abort capabilities, comparison tools, source preservation, and multi-hop support.

Centralized location for data analysis

Record requests and responses between the source and destination clusters for comparison, then forward the latency metrics and response codes to an analytics hub. You can analyze the data essential for transitioning your traffic from a legacy system to a new OpenSearch Service destination.

How it works

You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template.

Architecture diagram Step 1

Client traffic is directed to the existing cluster.

Step 2

An Network Load Balancer is positioned in front of the traffic capture proxy to route traffic as needed. The Network Load Balancer forwards traffic to the capture proxy, which then relays it to the source while simultaneously replicating the traffic to Apache Kafka.

Step 3

With continuous traffic capture in place, a Reindex-from-Snapshot (RFS) is initiated by the user through the Migration Management Console.

Step 4

After a backfill has been completed, the captured traffic is replayed by the user with a Traffic Replayer.

Step 5

The performance and behavior of traffic routed to the source and target clusters are analyzed by reviewing relevant logs and metrics.

Step 6

After confirming the target cluster’s functionality meets expectations, the user redirects clients to the new target. Additionally, the user can retire and discard the old cluster’s infrastructure.

Deploy with confidence

We'll walk you through it

Get started fast. Read the implementation guide for deployment steps, architecture details, cost information, and customization options.Open guide

Deployment Options

Launch Migration Assistant with EKS in a new VPC

Deploy using Amazon Elastic Kubernetes Service

Launch Migration Assistant with EKS in an existing VPC

Deploy using Amazon Elastic Kubernetes Service

AllCloud

"Our work using Migration Assistant for Amazon OpenSearch Service reflects AllCloud’s commitment to customer success."

Peter Nebel, Chief Strategy Officer

Read the case study