Amazon OpenSearch Service rename - Summary of changes
On September 8, 2021, our search and analytics suite was renamed to Amazon OpenSearch Service. OpenSearch Service supports OpenSearch as well as legacy Elasticsearch OSS. The following sections describe the different parts of the service that changed with the rename, and what actions you need to take to ensure that your domains continue to function properly.
Some of these changes only apply when you upgrade your domains from Elasticsearch to OpenSearch. In other cases, such as in the Billing and Cost Management console, the experience changes immediately.
Note that this list is not exhaustive. While other parts of the product also changed, these updates are the most relevant.
New API version
The new version of the OpenSearch Service configuration API (2021-01-01) works with OpenSearch as
well as legacy Elasticsearch OSS. 21 API operations were replaced with more concise and
engine-agnostic names (for example, CreateElasticsearchDomain
changed to
CreateDomain
), but OpenSearch Service continues to support both API versions.
We recommend that you use the new API operations
to create and manage domains going forward. Note that when you use the new API
operations to create a domain, you need to specify the EngineVersion
parameter in the format Elasticsearch_X.Y
or OpenSearch_X.Y
,
rather than just the version number. If you don't specify a version, it defaults to the
latest version of OpenSearch.
Upgrade your AWS CLI to version 1.20.40 or later in order to use aws opensearch
...
to create and manage your domains. For the new CLI format, see the OpenSearch CLI reference
Renamed instance types
Instance types in Amazon OpenSearch Service are now in the format
<type>.<size>.search
—for example,
m6g.large.search
rather than
m6g.large.elasticsearch
. You don't need to
take any action. Existing domains will start automatically referring to the new instance
types within the API and in the Billing and Cost Management console.
If you have Reserved Instances (RIs), your contract won't be impacted by the change. The old configuration API version is still compatible with the old naming format, but if you want to use the new API version, you need to use the new format.
Access policy changes
The following sections describe what actions you need to take to update your access policies.
IAM policies
We recommend that you update your IAM policies to use
the renamed API operations. However, OpenSearch Service will continue to respect existing policies
by internally replicating the old API permissions. For example, if you currently
have permission to perform the CreateElasticsearchDomain
operation, you
can now make calls to both CreateElasticsearchDomain
(old API
operation) and CreateDomain
(new API operation). The same applies to
explicit denies. For a list of updated API operations, see the policy element reference.
SCP policies
Service control policies (SCPs) introduce an additional layer of
complexity compared to standard IAM. To prevent your SCP policies from breaking,
you need to add both the old and the new API operations to each
of your SCP policies. For example, if a user currently has allow permissions for
CreateElasticsearchDomain
, you also need to grant them allow
permissions for CreateDomain
so they can retain the ability to create
domains. The same applies to explicit denies.
For example:
"Statement": [ { "Effect": "Allow", "Action": [ "es:CreateElasticsearchDomain", "es:CreateDomain" ... ], }, "Effect": "Deny", "Action:" [ "es:DeleteElasticsearchDomain", "es:DeleteDomain" ...
New resource types
OpenSearch Service introduces the following new resource types:
Resource | Description |
---|---|
|
Represents an Amazon OpenSearch Service domain. This resource exists at the service
level and isn't specific to the software running on the domain. It
applies to services like AWS CloudFormation For instructions to upgrade domains defined within CloudFormation from Elasticsearch to OpenSearch, see Remarks in the CloudFormation User Guide. |
|
Represents OpenSearch/Elasticsearch software running on a domain.
This resource applies to services like AWS CloudTrailAWS::Elasticsearch::Domain ) versus domains
running OpenSearch (AWS::OpenSearch::Domain ). |
Note
In AWS ConfigAWS::Elasticsearch::Domain
resource type
for several weeks, even if you upgrade one or more domains to OpenSearch.
Kibana renamed to OpenSearch Dashboards
OpenSearch Dashboards, the AWS alternative to
Kibana, is an open-source visualization tool designed to work with OpenSearch. After
you upgrade a domain from Elasticsearch to OpenSearch, the
/_plugin/kibana
endpoint changes to /_dashboards
. OpenSearch Service
will redirect all requests to the new endpoint, but if you use the Kibana endpoint in
any of your IAM policies, update those policies to include the new
/_dashboards
endpoint as well.
If you're using SAML authentication for OpenSearch Dashboards, before you upgrade your domain to
OpenSearch, you need to change all Kibana URLs configured in your identity provider
(IdP) from /_plugin/kibana
to /_dashboards
. The most common
URLs are assertion consumer service (ACS) URLs and recipient URLs.
The default kibana_read_only
role for OpenSearch Dashboards was renamed to
opensearch_dashboards_read_only
, and the kibana_user
role
was renamed to opensearch_dashboards_user
. The change applies to all
newly-created OpenSearch 1.x domains
running service software R20211203 or later. If you upgrade an existing domain to
service software R20211203, the role names remain the same.
Renamed CloudWatch metrics
Several CloudWatch metrics change for domains running OpenSearch. When you upgrade a domain to OpenSearch, the metrics change automatically and your current CloudWatch alarms will break. Before upgrading your cluster from an Elasticsearch version to an OpenSearch version, make sure to update your CloudWatch alarms to use the new metrics.
The following metrics changed:
Original metric name | New name |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For a full list of metrics that OpenSearch Service sends to Amazon CloudWatch, see Monitoring OpenSearch cluster metrics with Amazon CloudWatch.
Billing and Cost Management console changes
Historic data in the Billing and Cost Management
In addition to the service name, the following fields will change for all reports, bills, and price list API operations:
Field | Old format | New format |
---|---|---|
Instance type |
|
|
Product family |
Elasticsearch Instance Elasticsearch Volume |
Amazon OpenSearch Service Instance Amazon OpenSearch Service Volume |
Pricing description |
$5.098 per c5.18xlarge.elasticsearch instance hour (or partial hour) - EU |
$5.098 per c5.18xlarge.search instance hour (or partial hour) - EU |
Instance family |
|
|
New event format
The format of events that OpenSearch Service sends to Amazon EventBridge and Amazon CloudWatch has changed,
specifically the detail-type
field. The source field (aws.es
)
remains the same. For the complete format for each event type, see Monitoring OpenSearch Service events with Amazon EventBridge. If you have existing event rules that depend on the
old format, make sure to update them to conform to the new format.
What's staying the same?
The following features and functionality, among others not listed, will remain the same:
-
Service principal (
es.amazonaws.com
) -
Vendor code
-
Domain ARNs
-
Domain endpoints
Get started: Upgrade your domains to OpenSearch 1.x
OpenSearch 1.x supports upgrades from Elasticsearch versions 6.8
and 7.x. For instructions to upgrade your domain, see
Upgrading a domain (console). If you're using the AWS CLI or configuration API
to upgrade your domain, you need to specify the TargetVersion
as
OpenSearch_1.x
.
OpenSearch 1.x introduces an additional domain setting called Enable compatibility mode. Because certain Elasticsearch OSS clients and plugins check the cluster version before connecting, compatibility mode sets OpenSearch to report its version as 7.10 so these clients continue to work.
You can enable compatibility mode when you create OpenSearch domains for the first
time, or when you upgrade to OpenSearch from an Elasticsearch version. If it's not set,
the parameter defaults to false
when you create a domain, and
true
when you upgrade a domain.
To enable compatibility mode using the configuration API, set override_main_response_version
to
true
:
POST https://es.us-east-1.amazonaws.com/2021-01-01/opensearch/upgradeDomain { "DomainName": "
domain-name
", "TargetVersion": "OpenSearch_1.0", "AdvancedOptions": { "override_main_response_version": "true" } }
To enable or disable compatibility mode on existing OpenSearch
domains, you need to use the OpenSearch _cluster/settings
PUT /_cluster/settings { "persistent" : { "compatibility.override_main_response_version" : true } }