Cost

You are responsible for the cost of the AWS services used while running this solution. As of this revision, the cost for running this solution with the default settings in the US East (N. Virginia) Region is approximately $3,096 for a 15-day migration with 100 TB of existing data and 15 MBps of live traffic. These costs are for the resources shown in the sample cost table.

We recommend creating a budget through AWS Cost Explorer to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this solution.

A migration typically has different duration and data volume for the individual steps described in the OpenSearch Migrations Wiki. Customers typically wait to remove infrastructure or delete data until after a migration is complete. It’s crucial to understand the volume and duration of each of the steps to estimate the cost of the solution.

In the following example, we outline the cost of a 15-day migration with the following schedule:

Example 15-day migration schedule

To understand the costs, we need to map these steps into cost components, including data retention periods as applicable. This yields the following schedule:

Example 15-day migration schedule mapped to solution components

• Core Services – 15 days

• Capture Proxy – 6 days

• Capture Proxy Data Retention Period – 4 days

• Snapshot – 1 day

• Snapshot Data Retention Period – 9 days

• Reindex from Snapshot – 3 days

• Traffic Replayer – 2 days

• Traffic Replayer Data Retention Period – 4 days

• Target Proxy – 4 days

Using this schedule, the following is an example of a customer performing a zero-downtime migration of a cluster with 100 TB of primary shard data with 15 MBps aggregated request-response throughput. The ongoing request throughput over the 6-day capture timeline gives a capture/replay volume of about 7.8 TB.

Assumptions:

• No data is retained after the teardown step, representing no resources and cost after day 15.

• Deployment in US East (N. Virginia).

• The Source Cluster, Target Cluster, and VPC are imported and cost excluded from this calculation, including data transfer cost based on VPC setup. We recommend that the VPC includes an Amazon S3 Gateway Endpoint and cluster data communication over VPC Interface Endpoints to reduce data transfer cost.

This gives us the following cost table:

AWS service	Dimensions	Cost [USD]
Core services
Amazon ECS	Migration Management Console: Task vCPU: 1 task × 24 hours per day × 15 days × (0.50 vCPU × 0.04048 USD per hour) = $7.29 Task Memory: 1 task × 24 hours per day × 15 days × (1 GB × 0.004445 USD per hour) = $1.60	$8.89
AWS Lambda	< 10 requests × 0.20 per 1M requests = ~$0.00 < 10 seconds × $0.0000166667 per GB-second = ~$0.00	~$0.00
AWS Secrets Manager	($0.40 per secret-month × 10 secrets × 15 days) / 30 days per month = $2.00	$2.00
Amazon Route 53	($0.50 per Hosted Zone × 15 days) / 30 days per month = $0.25	$0.25
Amazon Elastic Container Registry (Amazon ECR)	(3 GB × $0.10 GB/month × 15 days) / 30 days per month = $0.15	$0.15
Amazon EC2	Bootstrap Box (t3.large): 1 × $0.0832/hr × 24 hours per day × 15 days = $29.95	$29.95
Amazon EBS	Bootstrap Box: (50 GB × $0.08/GB-month × 15 days) / 30 days per month = $2.00	$2.00
Amazon Virtual Private Cloud (Amazon VPC)	Bootstrap Box VPC – NAT Gateway: 1 × 24 hours per day × 15 days × $0.045/hour = $16.20 30 GB × $0.045/GB = $1.35	$17.55
Elastic Load Balancing	Application Load Balancer hours: 15 days × 24 hours × $0.0225 per Application Load Balancer-hour = $8.10	$8.10
Core services total:		$68.89
Miscellaneous services
Amazon CloudWatch	Note This is a high estimate for aggregated charges over all components for a large migration. Log data ingested (200 GB): $0.50 per GB × 200 GB = $100.00 Archived log charges (assume log data compresses to 30 GB): ($0.03 per GB-month × 30 GB × 15 days) / 30 days per month = $0.45 Metrics (200 metrics): (200 metrics × $0.30 per metric-month × 15 days) / 30 days per month = $60.00 Note Metrics are metered only while being sent. Actual usage and cost might be lower.	$160.45
AWS X-Ray	1M traces × $5.00 per million traces = $5.00	$5.00
Miscellaneous services total:		$165.45
Optional services
Capture proxy (7.776 TB), 6 days running; 4 days data retention)
Amazon Managed Streaming for Apache Kafka (Amazon MSK)	Note Assume deployment in 2 Availability Zones. Broker (6 days running + 4 days data retention): 2 nodes (M5.large) × $0.21 node per hour × 10 days × 24 hours per day = $151.20 Storage (9,331 GB Provisioned (20% buffer) for 6 days running + 4 days data retention): (2 nodes × 9,331 GB × $0.10 per GB-month × 10 days) / 30 days per month = $620.07 Data transfer: 7,776 GB × $0.01 per GB inbound = $77.76	$849.03
Amazon ECS	Task vCPU: 4 tasks × 0.5 vCPU × 0.04048 USD per vCPU-hour × 6 days × 24 hours per day = $11.66 Task memory: 4 tasks × 2 GB × 0.004445 USD per GB-hour × 6 days × 24 hours per day = $5.12	$16.78
Elastic Load Balancing	Load Balancer Capacity Unity (LCU) (assuming GB is the cost dimension): 7,776 GB × $0.008 per LCU = $62.21	$62.21
Capture proxy total:		$928.02
Target proxy (15 MBps for 4 days (5.184 TB))
Amazon ECS	Task vCPU: 4 tasks × 0.5 vCPU × 0.04048 USD per vCPU-hour) × 4 days × 24 hours per day = $7.77 Task memory: 4 tasks × 2 GB × 0.004445 USD per GB-hour × 4 days × 24 hours per day = $3.41	$11.18
Elastic Load Balancing	LCU (assuming GB is the cost dimension): 5,184 GB × $0.008 per LCU = $41.47	$41.47
Target proxy total:		$52.65
Replay (7.776 TB over 2 days, retention 4 days)
Amazon ECS	Task vCPU: 1 task × 1 vCPU × 0.04048 USD per vCPU-hour × 2 days × 24 hours per day = $1.94 Task memory: 1 task × 4 GB × 0.004445 USD per GB-hour × 2 days × 24 hours per day = $0.86	$2.80
Amazon MSK	Intra-Region: 7,776 GB × 0.01 USD per GB outbound = $77.76	$77.76
Amazon Elastic File System (Amazon EFS)	Note Calculating cost based on constant size of 15.552 TB (7.776 TB × 2). Actual cost might differ due to some data spending less than 5 days in EFS Infrequent Access or data inflation. EFS Standard (infrequent after 1 day): 15,552 GB × $0.30 per GB-month × 1 day / 30 days per month = $155.52 EFS Infrequent Access (5 days maximum): 15,552 GB × $0.025 per GB-month × 5 days / 30 days per month = $64.80 EFS Infrequent Access – Tiering: 15,552 GB × $0.01 per GB = $155.52	$375.84
Replay total:		$456.40
Historical Backfill (100 TB Snapshot over 1 month)
Amazon ECS	RFS: Each task is capable of delivering approximately 5 MBps of snapshot data. To deliver 100 TB over 3 days, 77 tasks are needed. Task vCPU: 77 tasks × 3 days × 24 hours per day × 2 vCPUs × $0.04048 per hour = $448.84 Task memory: 77 tasks × 3 days × 24 hours per day × 4 GB × $0.004445 per hour = $98.57 Task storage: 77 tasks × 3 days × 24 hours per day × (200GB - 20GB Free Tier) × $0.000111 per hour = $110.77	$657.34
Amazon S3	Note This calculates the cost for all the data occurring in the S3 bucket for the entire duration. The actual cost might be less for storage due to partial rate while the snapshot is being taken. Snapshot storage: (100 TB × 1,000 GB per TB × $0.023 per GB-month × 10 days) / 30 days per month = $766.66	$766.66
Historical Backfill total:		$1,424.00
Total for all components:		$3,095.41

The Capture and Replay yields an effective cost per TB of $199.87, and the Historic Backfill is $15.41 per TB. For a given cluster, the amount of data in Historic Backfill is largely fixed, while the data for Capture and Replay is based on the time needed to capture. For this reason, it can be cost advantageous to scale up the target OpenSearch cluster beyond the final intended capacity for the Historic Backfill period to reduce the duration and quantity of data for Capture and Replay.

Taking the table above and dividing out the components cost, we can determine the following calculation method for cost. This is a broad estimation making assumptions on traffic pattern including size per request, and number of new connections. We can also subdivide this by migration type applicable.

Component	Applicable migration type	Cost [USD]
Core services	Metadata only Historical only Live only Full migration	$4.59/day
Miscellaneous services	Metadata only Historical only Live only Full migration	$165.45 per large migration
Capture runtime	Live only Full migration	$17.22/day + $8.00/TB-day + $18.00/TB
Capture data retention period	Live only Full migration	$15.12/day + $8.00/TB-day
Snapshot	Historical only Full migration	$0.7667/TB-day
Snapshot data retention period	Historical only Full migration	$0.7667/TB-day
Reindex from snapshot	Historical only Full migration	$6.57/TB
Traffic Replayer	Live only Full migration	$1.40/day + $50.00/TB + $1.6667/TB-day Note Due to Amazon EFS-Intelligent Tiering, $1.6667/TB-day is not paid for the first day.
Traffic Replayer data retention period	Historical only Full migration	$1.6667/TB-day
Target proxy	Historical only Full migration	$2.10/day + $8.00/TB

We can also identify the cost for full migrations of different sizes that follow the 15-day migration timeline. By applying the durations shown previously, as well as a conversion from TB/day to MB/s, we get the following formula for a representative cost estimate:

15-day migration = $390.35 + $14.24 × HistoricalTB + $71.86 × LiveMBps

This results in the following cost estimates table:

Note

All costs are rounded to the nearest dollar.

15-day full migration cost ($ USD/migration)
	Live traffic throughput
Historical data volume	5 MBps	20 MBps	50 MBps
1 TB	$764	$1,842	$4,125
10 TB	$892	$1,970	$3,252
100 TB	$2,174	$3,252	$5,407