AWS::OpenSearchService::Domain - AWS CloudFormation

AWS::OpenSearchService::Domain

The AWS::OpenSearchService::Domain resource creates an Amazon OpenSearch Service domain.

Important

The AWS::OpenSearchService::Domain resource replaces the legacy AWS::Elasticsearch::Domain resource. While the Elasticsearch resource and options are still supported, we recommend modifying your existing Cloudformation templates to use the new OpenSearch Service resource, which supports both OpenSearch and legacy Elasticsearch engines. For instructions to upgrade domains defined within CloudFormation from Elasticsearch to OpenSearch, see Remarks.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

Properties

AccessPolicies

An AWS Identity and Access Management (IAM) policy document that specifies who can access the OpenSearch Service domain and their permissions. For more information, see Configuring access policies in the Amazon OpenSearch Service Developer Guide.

Required: No

Type: Json

Update requires: No interruption

AdvancedOptions

Additional options to specify for the OpenSearch Service domain. For more information, see AdvancedOptions in the OpenSearch Service configuration API reference.

Required: No

Type: Map of String

Update requires: No interruption

AdvancedSecurityOptions

Specifies options for fine-grained access control.

Required: No

Type: AdvancedSecurityOptionsInput

Update requires: No interruption

ClusterConfig

ClusterConfig is a property of the AWS::OpenSearchService::Domain resource that configures an Amazon OpenSearch Service cluster.

Required: No

Type: ClusterConfig

Update requires: No interruption

CognitoOptions

Configures OpenSearch Service to use Amazon Cognito authentication for OpenSearch Dashboards.

Required: No

Type: CognitoOptions

Update requires: No interruption

DomainEndpointOptions

Specifies additional options for the domain endpoint, such as whether to require HTTPS for all traffic or whether to use a custom endpoint rather than the default endpoint.

Required: No

Type: DomainEndpointOptions

Update requires: No interruption

DomainName

A name for the OpenSearch Service domain. For valid values, see the DomainName data type in the Amazon OpenSearch Service Developer Guide. If you don't specify a name, AWS CloudFormation generates a unique physical ID and uses that ID for the domain name. For more information, see Name Type.

Required when creating a new domain.

Important

If you specify a name, you can't perform updates that require replacement of this resource. You can perform updates that require no or some interruption. If you must replace the resource, specify a new name.

Required: Conditional

Type: String

Update requires: Replacement

EBSOptions

The configurations of Amazon Elastic Block Store (Amazon EBS) volumes that are attached to data nodes in the OpenSearch Service domain. For more information, see EBS volume size limits in the Amazon OpenSearch Service Developer Guide.

Required: No

Type: EBSOptions

Update requires: No interruption

EncryptionAtRestOptions

Whether the domain should encrypt data at rest, and if so, the AWS KMS key to use. See Encryption of data at rest for Amazon OpenSearch Service.

Required: No

Type: EncryptionAtRestOptions

Update requires: No interruption

EngineVersion

The version of OpenSearch to use. The value must be in the format OpenSearch_X.Y or Elasticsearch_X.Y. If not specified, the latest version of OpenSearch is used. For information about the versions that OpenSearch Service supports, see Supported versions of OpenSearch and Elasticsearch in the Amazon OpenSearch Service Developer Guide.

If you set the EnableVersionUpgrade update policy to true, you can update EngineVersion without interruption. When EnableVersionUpgrade is set to false, or is not specified, updating EngineVersion results in replacement.

Required: Conditional

Type: String

Minimum: 14

Maximum: 18

Pattern: ^Elasticsearch_[0-9]{1}\.[0-9]{1,2}$|^OpenSearch_[0-9]{1,2}\.[0-9]{1,2}$

Update requires: No interruption

LogPublishingOptions

An object with one or more of the following keys: SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, INDEX_SLOW_LOGS, AUDIT_LOGS, depending on the types of logs you want to publish. Each key needs a valid LogPublishingOption value. For the full syntax, see the examples.

Required: No

Type: Map of LogPublishingOption

Update requires: No interruption

NodeToNodeEncryptionOptions

Specifies whether node-to-node encryption is enabled. See Node-to-node encryption for Amazon OpenSearch Service.

Required: No

Type: NodeToNodeEncryptionOptions

Update requires: No interruption

SnapshotOptions

DEPRECATED. The automated snapshot configuration for the OpenSearch Service domain indices.

Required: No

Type: SnapshotOptions

Update requires: No interruption

Tags

An arbitrary set of tags (key–value pairs) to associate with the OpenSearch Service domain.

Required: No

Type: List of Tag

Update requires: No interruption

VPCOptions

The virtual private cloud (VPC) configuration for the OpenSearch Service domain. For more information, see Launching your Amazon OpenSearch Service domains within a VPC in the Amazon OpenSearch Service Developer Guide.

If you remove this entity altogether, along with its associated properties, it causes a replacement. You might encounter this scenario if you're updating your security configuration from a VPC to a public endpoint.

Required: No

Type: VPCOptions

Update requires: Some interruptions

Return values

Ref

When the logical ID of this resource is provided to the Ref intrinsic function, Ref returns the resource name, such as mystack-abc1d2efg3h4. For more information about using the Ref function, see Ref.

Fn::GetAtt

GetAtt returns a value for a specified attribute of this type. For more information, see Fn::GetAtt. The following are the available attributes and sample return values.

Arn

The Amazon Resource Name (ARN) of the domain, such as arn:aws:es:us-west-2:123456789012:domain/mystack-1ab2cdefghij.

DomainArn

Not currently supported by AWS CloudFormation.

DomainEndpoint

The domain-specific endpoint used for requests to the OpenSearch APIs, such as search-mystack-1ab2cdefghij-ab1c2deckoyb3hofw7wpqa3cm.us-west-1.es.amazonaws.com.

Id

The resource ID. For example, 123456789012/my-domain.

Remarks

Migrating stacks from Elasticsearch to OpenSearch

Important

You can't directly update CloudFormation templates to use the AWS::OpenSearchService::Domain resource in place of AWS::Elasticsearch::Domain, otherwise the corresponding domain will be deleted along with all of its data.

Perform the following steps to migrate an Elasticsearch domain to an OpenSearch domain if the domain is defined within CloudFormation.

Step 1: Prepare your existing stack for deprecation

Make a copy of your original CloudFormation template, which contains the Elasticsearch domain resource, for use in step 3. Then add the following attributes to the Elasticsearch domain resource at the same level as Type and Properties.

DeletionPolicy: Retain

UpdateReplacePolicy: Retain

These settings ensure that CloudFormation doesn't delete or modify the corresponding domain when you delete this resource from your stack. If you have other custom resources defined in the stack that aren't critically important during the short migration period, you can delete them from the template and they'll be recreated when you create the new stack.

Step 2: Upgrade your domain to OpenSearch

After you add the two policy attributes to your template, upgrade your domain to an OpenSearch version using the normal upgrade process. For instructions, see Starting an upgrade. Make sure to take a snapshot of your domain before upgrading it to prevent accidental loss of data.

Step 3: Create a new CloudFormation template

While you wait for the upgrade to complete, prepare your new OpenSearch template. Using the copy of your original template that you made in step 1, make the following changes:

  • Change the domain resource type from AWS::Elasticsearch::Domain to AWS::OpenSearchService::Domain.

  • Add the DeletionPolicy and UpdateReplacePolicy attributes to the resource, as you did in step 1.

  • Change ElasticsearchVersion to EngineVersion and set its value to OpenSearch_1.0 (or whichever version of OpenSearch you want to upgrade to).

  • If your resource contains ElasticsearchClusterConfig, change it to ClusterConfig.

  • Change the suffixes of all instance types from .elasticsearch to .search.

  • If your template includes ColdStorageOptions, remove it, as it's not currently supported for OpenSearch resources.

  • If there are any Fn::GetAtt or !GetAtt references to your domain ARN, change them to !GetAtt MyDomain.Arn.

  • Comment out any resources not currently within the stack (most likely everything except AWS::OpenSearchService::Domain).

See the next section for examples that demonstrate the new format.

Step 4: Import the OpenSearch stack

Once your domain upgrade finishes, you can import the new stack. Within CloudFormation, choose Create stack and With existing resources (import resources), then upload the template you created in the previous step.

CloudFormation prompts you for the name (identifier value) of the existing domain. Copy the domain name directly from the OpenSearch console. Give the stack a name that's different from the current one, then choose Import resources.

After the stack is created, uncomment any related resources from the stack and update it to ensure they're recreated. You can remove the DeletionPolicy and UpdateReplacePolicy attributes if you want, but they can help prevent accidental deletions in the future.

Step 5: Delete the Elasticsearch stack

Now that your new stack is created, delete the old stack which contains the legacy Elasticsearch resource.

Important

Before deleting the old stack, make absolutely sure that the template contains the DeletionPolicy: Retain attribute.

*The above steps were partially derived from this blog post.

Examples

Create an OpenSearch Service domain that contains two data nodes and three master nodes

The following example creates an OpenSearch Service domain running OpenSearch 1.0 that contains two data nodes and three dedicated master nodes. The domain has 40 GiB of storage and enables log publishing for application logs, search slow logs, and index slow logs. The access policy permits the root user for the AWS account to make all HTTP requests to the domain, such as indexing documents or searching indices.

JSON

"OpenSearchServiceDomain": { "Type":"AWS::OpenSearchService::Domain", "Properties": { "DomainName": "test", "EngineVersion": "OpenSearch_1.0", "ClusterConfig": { "DedicatedMasterEnabled": true, "InstanceCount": "2", "ZoneAwarenessEnabled": true, "InstanceType": "m3.medium.search", "DedicatedMasterType": "m3.medium.search", "DedicatedMasterCount": "3" }, "EBSOptions":{ "EBSEnabled": true, "Iops": "0", "VolumeSize": "20", "VolumeType": "gp2" }, "AccessPolicies": { "Version":"2012-10-17", "Statement":[ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:user/opensearch-user" }, "Action":"es:*", "Resource": "arn:aws:es:us-east-1:123456789012:domain/test/*" } ] }, "LogPublishingOptions": { "ES_APPLICATION_LOGS": { "CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-application-logs", "Enabled": true }, "SEARCH_SLOW_LOGS": { "CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-slow-logs", "Enabled": true }, "INDEX_SLOW_LOGS": { "CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-index-slow-logs", "Enabled": true } }, "AdvancedOptions": { "rest.action.multi.allow_explicit_index": true, "override_main_response_version": true } } }

YAML

OpenSearchServiceDomain: Type: AWS::OpenSearchService::Domain Properties: DomainName: 'test' EngineVersion: 'OpenSearch_1.0' ClusterConfig: DedicatedMasterEnabled: true InstanceCount: '2' ZoneAwarenessEnabled: true InstanceType: 'm3.medium.search' DedicatedMasterType: 'm3.medium.search' DedicatedMasterCount: '3' EBSOptions: EBSEnabled: true Iops: '0' VolumeSize: '20' VolumeType: 'gp2' AccessPolicies: Version: '2012-10-17' Statement: - Effect: 'Allow' Principal: AWS: 'arn:aws:iam::123456789012:user/opensearch-user' Action: 'es:*' Resource: 'arn:aws:es:us-east-1:846973539254:domain/test/*' LogPublishingOptions: ES_APPLICATION_LOGS: CloudWatchLogsLogGroupArn: 'arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-application-logs' Enabled: true SEARCH_SLOW_LOGS: CloudWatchLogsLogGroupArn: 'arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-slow-logs' Enabled: true INDEX_SLOW_LOGS: CloudWatchLogsLogGroupArn: 'arn:aws:logs:us-east-1:123456789012:log-group:/aws/opensearch/domains/opensearch-index-slow-logs' Enabled: true AdvancedOptions: rest.action.multi.allow_explicit_index: true override_main_response_version: true

Create a domain with VPC options

The following example creates a domain with VPC options.

JSON

{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "OpenSearchServiceDomain resource", "Parameters": { "DomainName": { "Description": "User-defined OpenSearch domain name", "Type": "String" }, "EngineVersion": { "Description": "User-defined OpenSearch version", "Type": "String" }, "InstanceType": { "Type": "String" }, "AvailabilityZone": { "Type": "String" }, "CidrBlock": { "Type": "String" }, "GroupDescription": { "Type": "String" }, "SGName": { "Type": "String" } }, "Resources": { "OpenSearchServiceDomain": { "Type": "AWS::OpenSearchService::Domain", "Properties": { "DomainName": { "Ref": "DomainName" }, "EngineVersion": { "Ref": "EngineVersion" }, "ClusterConfig": { "InstanceCount": "1", "InstanceType": { "Ref": "InstanceType" } }, "EBSOptions": { "EBSEnabled": true, "Iops": "0", "VolumeSize": "10", "VolumeType": "standard" }, "AccessPolicies": { "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": { "AWS": "*" }, "Action": "es:*", "Resource": "*" } ] }, "AdvancedOptions": { "rest.action.multi.allow_explicit_index": true, "override_main_response_version": true }, "Tags": [ { "Key": "foo", "Value": "bar" } ], "VPCOptions": { "SubnetIds": [ { "Ref": "subnet" } ], "SecurityGroupIds": [ { "Ref": "mySecurityGroup" } ] } } }, "vpc": { "Type": "AWS::EC2::VPC", "Properties": { "CidrBlock": "10.0.0.0/16" } }, "subnet": { "Type": "AWS::EC2::Subnet", "Properties": { "VpcId": { "Ref": "vpc" }, "CidrBlock": { "Ref": "CidrBlock" }, "AvailabilityZone": { "Ref": "AvailabilityZone" } } }, "mySecurityGroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": { "Ref": "GroupDescription" }, "VpcId": { "Ref": "vpc" }, "GroupName": { "Ref": "SGName" }, "SecurityGroupIngress": [ { "FromPort": 443, "IpProtocol": "tcp", "ToPort": 443, "CidrIp": "0.0.0.0/0" } ] } } }, "Outputs": { "Arn": { "Value": { "Fn::GetAtt": [ "OpenSearchServiceDomain", "Arn" ] } }, "DomainEndpoint": { "Value": { "Fn::GetAtt": [ "OpenSearchServiceDomain", "DomainEndpoint" ] } }, "SecurityGroupId": { "Value": { "Ref": "mySecurityGroup" } }, "SubnetId": { "Value": { "Ref": "subnet" } } } }

YAML

AWSTemplateFormatVersion: '2010-09-09' Description: OpenSearchServiceDomain resource Parameters: DomainName: Description: User-defined OpenSearch domain name Type: String EngineVersion: Description: User-defined OpenSearch version Type: String InstanceType: Type: String AvailabilityZone: Type: String CidrBlock: Type: String GroupDescription: Type: String SGName: Type: String Resources: OpenSearchServiceDomain: Type: 'AWS::OpenSearchService::Domain' Properties: DomainName: Ref: DomainName EngineVersion: Ref: EngineVersion ClusterConfig: InstanceCount: '1' InstanceType: Ref: InstanceType EBSOptions: EBSEnabled: true Iops: '0' VolumeSize: '10' VolumeType: 'standard' AccessPolicies: Version: '2012-10-17' Statement: - Effect: Deny Principal: AWS: '*' Action: 'es:*' Resource: '*' AdvancedOptions: rest.action.multi.allow_explicit_index: true override_main_response_version: true Tags: - Key: foo Value: bar VPCOptions: SubnetIds: - Ref: subnet SecurityGroupIds: - Ref: mySecurityGroup vpc: Type: 'AWS::EC2::VPC' Properties: CidrBlock: 10.0.0.0/16 subnet: Type: 'AWS::EC2::Subnet' Properties: VpcId: Ref: vpc CidrBlock: Ref: CidrBlock AvailabilityZone: Ref: AvailabilityZone mySecurityGroup: Type: 'AWS::EC2::SecurityGroup' Properties: GroupDescription: Ref: GroupDescription VpcId: Ref: vpc GroupName: Ref: SGName SecurityGroupIngress: - FromPort: 443 IpProtocol: tcp ToPort: 443 CidrIp: 0.0.0.0/0 Outputs: Arn: Value: 'Fn::GetAtt': - OpenSearchServiceDomain - Arn DomainEndpoint: Value: 'Fn::GetAtt': - OpenSearchServiceDomain - DomainEndpoint SecurityGroupId: Value: Ref: mySecurityGroup SubnetId: Value: Ref: subnet