What is Amazon OpenSearch Serverless?

Amazon OpenSearch Serverless is an on-demand serverless configuration for Amazon OpenSearch Service. Serverless removes the operational complexities of provisioning, configuring, and tuning your OpenSearch clusters. It's a good option for organizations that don't want to self-manage their OpenSearch clusters, or organizations that don't have the dedicated resources or expertise to operate large clusters. With OpenSearch Serverless, you can easily search and analyze a large volume of data without having to worry about the underlying infrastructure and data management.

An OpenSearch Serverless collection is a group of OpenSearch indexes that work together to support a specific workload or use case. Collections are easier to use than self-managed OpenSearch clusters, which require manual provisioning.

Collections have the same kind of high-capacity, distributed, and highly available storage volume that's used by provisioned OpenSearch Service domains, but they remove more complexity because they don't require manual configuration and tuning. Data is encrypted in transit within a collection. OpenSearch Serverless also supports OpenSearch Dashboards, which provides an intuitive interface for analyzing data.

Serverless collections currently run OpenSearch version 2.0.x. As new versions are released, OpenSearch Serverless will automatically upgrade your collections to consume new features, bug fixes, and performance improvements.

Use cases for OpenSearch Serverless

OpenSearch Serverless supports two primary use cases:

• Log analytics - The log analytics segment focuses on analyzing large volumes of semi-structured, machine-generated time series data for operational and user behavior insights.

• Full-text search - The full-text search segment powers applications in your internal networks (content management systems, legal documents) and internet-facing applications, such as ecommerce website content search.

When you create a collection, you choose one of these use cases. For more information, see Choosing a collection type.

Getting started

To get started with OpenSearch Serverless, create one or more collections using the OpenSearch Service console, the AWS CLI, or one of the AWS SDKs. For a tutorial that helps you get a collection up and running quickly, see Getting started with Amazon OpenSearch Serverless.

OpenSearch Serverless supports the same ingest and query API operations as the OpenSearch open source suite, so you can continue to use your existing clients and applications. Your clients must be compatible with OpenSearch 2.x in order to work with OpenSearch Serverless. For more information, see Ingesting data into Amazon OpenSearch Serverless collections.

How it works

Traditional OpenSearch clusters have a single set of instances that perform both indexing and search operations, and index storage is tightly coupled with compute capacity. By contrast, OpenSearch Serverless uses a cloud-native architecture that separates the indexing (ingest) components from the search (query) components, with Amazon S3 as the primary data storage for indexes.

This decoupled architecture lets you scale search and indexing functions independently of each other, and independently of the indexed data in S3. The architecture also provides isolation for ingest and query operations so that they can run concurrently without resource contention.

When you write data to a collection, OpenSearch Serverless distributes it to the indexing compute units. The indexing compute units ingest the incoming data and move the indexes to S3. When you perform a search on the collection data, OpenSearch Serverless routes requests to the search compute units that hold the data being queried. The search compute units download the indexed data directly from S3 (if it's not already cached locally), run search operations, and perform aggregations.

The following image illustrates this decoupled architecture:

OpenSearch Serverless compute capacity for data ingestion, searching, and querying are measured in OpenSearch Compute Units (OCUs). Each OCU is a combination of 6 GiB of memory and corresponding virtual CPU (vCPU), as well as data transfer to Amazon S3. Each OCU includes enough hot ephemeral storage for 120 GiB of index data.

When you create your first collection, OpenSearch Serverless instantiates two OCUs—one for indexing and one for search. To ensure high availability, it also launches a standby set of nodes in another Availability Zone. For development and testing purposes, you can disable the Enable redundancy setting for a collection, which eliminates the two standby replicas and only instantiates two OCUs. By default, the redundant active replicas are enabled, which means that a total of four OCUs are instantiated for the first collection in an account.

These OCUs exist even when there's no activity on any collection endpoints. All subsequent collections share these OCUs. When you create additional collections in the same account, OpenSearch Serverless only adds additional OCUs for search and ingest as needed to support the collections, according to the capacity limits that you specify. Capacity scales back down as your compute usage decreases.

For information about how you're billed for these OCUs, see Pricing for OpenSearch Serverless.

Choosing a collection type

OpenSearch Serverless supports three primary collection types:

Time series – The log analytics segment that focuses on analyzing large volumes of semi-structured, machine-generated data in real-time for operational, security, user behavior, and business insights.

Search – Full-text search that powers applications in your internal networks (content management systems, legal documents) and internet-facing applications, such as ecommerce website search and content search.

Vector search – Semantic search on vector embeddings that simplifies vector data management and powers machine learning (ML) augmented search experiences and generative AI applications, such as chatbots, personal assistants, and fraud detection.

You choose a collection type when you first create a collection:

The collection type that you choose depends on the kind of data that you plan to ingest into the collection, and how you plan to query it. You can't change the collection type after you create it.

The collection types have the following notable differences:

• For search and vector search collections, all data is stored in hot storage to ensure fast query response times. Time series collections use a combination of hot and warm storage, where the most recent data is kept in hot storage to optimize query response times for more frequently accessed data.

• For time series and vector search collections, you can't index by custom document ID or update by upsert requests. This operation is reserved for search use cases. You can update by document ID instead. For more information, see Supported OpenSearch API operations and permissions.

• For search and time series collections, you can't use k-NN type indexes.

Pricing for OpenSearch Serverless

In OpenSearch Serverless, you're charged for the following components:

• Data ingestion compute

• Search and query compute

• Storage retained in Amazon S3

OCUs are billed on an hourly basis, with per-second granularity. In your account statement, you see an entry for compute in OCU-hours with a label for data ingestion and a label for search. You're also billed on a monthly basis for data stored in Amazon S3. You aren't charged for using OpenSearch Dashboards.

You're billed for a minimum of four OCUs that are allocated for your workloads when you create a collection and enable redundant active replicas. You're billed for a minimum of two OCUs for the first collection in your account if you disable redundant active replicas. All subsequent collections can share those OCUs.

OpenSearch Serverless adds additional OCUs based on the compute needed to support your collections. If your workload uses a fractional OCU, the pricing is proportionate. You can configure a maximum number of OCUs for your account in order to control costs.

Note

Collections with unique AWS KMS keys can't share OCUs with other collections.

OpenSearch Serverless attempts to use the minimum required resources to account for changing workloads. The number of OCUs provisioned at any given time can vary and isn't exact. Over time, the algorithm that OpenSearch Serverless uses will continue to improve in order to better minimize system usage.

For full pricing details, see Amazon OpenSearch Service pricing.

Supported AWS Regions

OpenSearch Serverless is available in a subset of AWS Regions that OpenSearch Service is available in. For a list of supported Regions, see Amazon OpenSearch Service endpoints and quotas in the AWS General Reference.

Limitations

OpenSearch Serverless has the following limitations:

• Some OpenSearch API operations aren't supported. See Supported OpenSearch API operations and permissions.

• Some OpenSearch plugins aren't supported. See Supported OpenSearch plugins.

• There's currently no way to automatically migrate your data from a managed OpenSearch Service domain to a serverless collection. You must reindex your data from a domain to a collection.

• Cross-account access to collections isn't supported. You can't include collections from other accounts in your encryption or data access policies.

• Custom OpenSearch plugins aren't supported.

• You can't take or restore snapshots of OpenSearch Serverless collections.

• Cross-Region search and replication aren't supported.

• There are limits on the number of serverless resources that you can have in a single account and Region. See OpenSearch Serverless quotas.

• The refresh interval for indexes in vector search and time series collections is approximately 60 seconds. The refresh interval for indexes in search collections is approximately 10 seconds.

• The number of shards, number of intervals, and refresh interval are not modifiable and are handled by OpenSearch Serverless. The sharding strategy is based off the collection type and traffic. For example, a time series collection scales primary shards based on write traffic bottlenecks.

• Geospatial features available on OpenSearch versions up to 2.1 are supported.

Comparing OpenSearch Service and OpenSearch Serverless

In OpenSearch Serverless, some concepts and features are different than their corresponding feature for a provisioned OpenSearch Service domain. For example, one important difference is that OpenSearch Serverless doesn't have the concept of a cluster or node.

The following table describes how important features and concepts in OpenSearch Serverless differ from the equivalent feature in a provisioned OpenSearch Service domain.

Feature	OpenSearch Service	OpenSearch Serverless
Domains versus collections	Indexes are held in domains, which are pre-provisioned OpenSearch clusters. For more information, see Creating and managing Amazon OpenSearch Service domains.	Indexes are held in collections, which are logical groupings of indexes that represent a specific workload or use case. For more information, see Creating, listing, and deleting Amazon OpenSearch Serverless collections.
Node types and capacity management	You build a cluster with node types that meet your cost and performance specifications. You must calculate your own storage requirements and choose an instance type for your domain. For more information, see Sizing Amazon OpenSearch Service domains.	OpenSearch Serverless automatically scales and provisions additional compute units for your account based on your capacity usage. For more information, see Managing capacity limits for Amazon OpenSearch Serverless.
Billing	You pay for each hour of use of an EC2 instance and for the cumulative size of any EBS storage volumes attached to your instances. For more information, see Pricing for Amazon OpenSearch Service.	You're charged in OCU-hours for compute for data ingestion, compute for search and query, and storage retained in S3. For more information, see Pricing for OpenSearch Serverless.
Encryption	Encryption at rest is optional for domains. For more information, see Encryption of data at rest for Amazon OpenSearch Service.	Encryption at rest is required for collections. For more information, see Encryption in Amazon OpenSearch Serverless.
Data access control	Access to the data within domains is determined by IAM policies and fine-grained access control.	Access to data within collections is determined by data access policies.
Supported OpenSearch operations	OpenSearch Service supports a subset of all of the OpenSearch API operations. For more information, see Supported operations in Amazon OpenSearch Service.	OpenSearch Serverless supports a different subset of OpenSearch API operations. For more information, see Supported operations and plugins in Amazon OpenSearch Serverless.
Dashboards sign-in	Sign in with a username and password. For more information, see Accessing OpenSearch Dashboards as the master user.	If you're logged into the AWS console and navigate to your Dashboard URL, you'll automatically log in. For more information, see Accessing OpenSearch Dashboards.
APIs	Interact programmatically with OpenSearch Service using the OpenSearch Service API operations.	Interact programmatically with OpenSearch Serverless using the OpenSearch Serverless API operations.
Network access	Network settings for a domain apply to the domain endpoint as well as the OpenSearch Dashboards endpoint. Network access for both is tightly coupled.	Network settings for the domain endpoint and the OpenSearch Dashboards endpoint are decoupled. You can choose to not configure network access for OpenSearch Dashboards. For more information, see Network access for Amazon OpenSearch Serverless.
Signing requests	Use the OpenSearch high and low-level REST clients to sign requests. Specify the service name as es.	At this time, OpenSearch Serverless supports a subset of clients that OpenSearch Service supports. When you sign requests, specify the service name as aoss. The x-amz-content-sha256 header is required. For more information, see Signing HTTP requests with other clients.
OpenSearch version upgrades	You manually upgrade your domains as new versions of OpenSearch become available. You're responsible for ensuring that your domain meets the upgrade requirements, and that you've addressed any breaking changes.	OpenSearch Serverless automatically upgrades your collections to new OpenSearch versions. Upgrades don't necessarily happen as soon as a new version is available.
Service software updates	You manually apply service software updates to your domain as they become available.	OpenSearch Serverless automatically updates your collections to consume the latest bug fixes, features, and performance improvements.
VPC access	You can provision your domain within a VPC. You can also create additional OpenSearch Service-managed VPC endpoints to access the domain.	You create one or more OpenSearch Serverless-managed VPC endpoints for your account. Then, you include these endpoints within network policies.
SAML authentication	You enable SAML authentication on a per-domain basis. For more information, see SAML authentication for OpenSearch Dashboards.	You configure one or more SAML providers at the account level, then you include the associated user and group IDs within data access policies. For more information, see SAML authentication for Amazon OpenSearch Serverless.
Transport Security Layer (TSL)	OpenSearch Service supports TLS 1.2 but it is recommend you use TLS 1.3.	OpenSearch Serverless supports TLS 1.2 but it is recommended you use TLS 1.3.