Menu
Amazon Elasticsearch Service
Developer Guide (API Version 2015-01-01)

VPC Support for Amazon Elasticsearch Service Domains

A virtual private cloud (VPC) is a virtual network that is dedicated to your AWS account. It's logically isolated from other virtual networks in the AWS Cloud. You can launch AWS resources, such as Amazon ES domains, into your VPC.

Placing an Amazon ES domain within a VPC enables secure communication between Amazon ES and other services without the need for an Internet gateway, NAT device, or VPN connection. All traffic remains securely within the AWS Cloud. Domains that reside within a VPC have an extra layer of security when compared to domains that use public endpoints: you can use security groups as well as IAM policies to control access to the domain.

To support VPCs, Amazon ES places an endpoint into either one or two subnets of your VPC. A subnet is a range of IP addresses in your VPC. If you enable zone awareness for your domain, Amazon ES places an endpoint into two subnets. The subnets must be in different Availability Zones in the same region. If you don't enable zone awareness, Amazon ES places an endpoint into only one subnet.

The following illustration shows the VPC architecture if zone awareness is not enabled.

The following illustration shows the VPC architecture if zone awareness is enabled.

Amazon ES also places elastic network interface (ENIs) in the VPC for each of your data nodes. Amazon ES assigns each ENI a private IP address from the IPv4 address range of your subnet and also assigns a public DNS hostname (which is the domain endpoint) for the IP addresses. You must use a public DNS service to resolve the endpoint (which is a DNS hostname) to the appropriate IP addresses for the data nodes:

  • If your VPC uses the Amazon-provided DNS server by setting the enableDnsSupport option to true (the default value), resolution for the Amazon ES endpoint will succeed.

  • If your VPC uses a private DNS server and the server can reach the pubic authoritative DNS servers to resolve DNS hostnames, resolution for the Amazon ES endpoint will also succeed.

Because the IP addresses might change, you should resolve the domain endpoint periodically so that you can always access the correct data nodes. We recommend that you set the DNS resolution interval to one minute. If you’re using a client, you should also ensure that the DNS cache in the client is cleared.

Note

Amazon ES doesn't support IPv6 addresses with a VPC. You can use a VPC that has IPv6 enabled, but the domain will use IPv4 addresses.

Limitations

Currently, operating an Amazon ES domain within a VPC has the following limitations:

  • You can either launch your domain within a VPC or use a public endpoint, but you can't do both. You must choose one or the other when you create your domain.

  • If you launch a new domain within a VPC, you can't later switch it to use a public endpoint. The reverse is also true: If you create a domain with a public endpoint, you can't later place it within a VPC. Instead, you must create a new domain and migrate your data.

  • You can't launch your domain within a VPC that uses dedicated tenancy. You must use a VPC with tenancy set to Default.

  • After you place a domain within a VPC, you can't move it to a different VPC. However, you can change the subnets and security group settings.

  • Currently, Amazon ES does not support integration with Amazon Kinesis Firehose for domains that reside within a VPC. To use this service with Amazon ES, you must use a domain with public access.

  • To access the default installation of Kibana for a domain that resides within a VPC, users must have access to the VPC. This process varies by network configuration, but likely involves connecting to a VPN or managed network or using a proxy server. To learn more, see the Amazon VPC User Guide and Using a Proxy to Access Amazon ES from Kibana.

Before You Begin: Prerequisites for VPC Access

Before you can enable a connection between a VPC and your new Amazon ES domain, you must do the following:

  • Create a VPC

    To create your VPC, you can use the Amazon VPC console, the AWS CLI, or one of the AWS SDKs. You must create a subnet in the VPC, or two subnets if you enable zone awareness. For more information, see Creating A VPC. If you already have a VPC, you can skip this step.

  • Reserve IP addresses

    Amazon ES enables the connection of a VPC to a domain by placing network interfaces in a subnet of the VPC. Each network interface is associated with an IP address. You must reserve a sufficient number of IP addresses in the subnet for the network interfaces. For more information, see Reserving IP Addresses in a VPC Subnet.

Creating a VPC

To create your VPC, you can use one of the following: the Amazon VPC console, the AWS CLI, or one of the AWS SDKs. The VPC must have a subnet, or two subnets if you enable zone awareness. The two subnets must be in different Availability Zones in the same region.

The following procedure shows how to use the Amazon VPC console to create a VPC with a public subnet, reserve IP addresses for the subnet, and create a security group to control access to your Amazon ES domain. For other VPC configurations, see Scenarios and Examples in the Amazon VPC User Guide.

To create a VPC (console)

  1. Sign in to the AWS Management Console, and open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the navigation pane, choose VPC Dashboard.

  3. Choose Start VPC Wizard.

  4. On the Select a VPC Configuration page, select VPC with a Single Public Subnet.

  5. On the VPC with a Single Public Subnet page, keep the default options, and then choose Create VPC.

  6. In the confirmation message that appears, choose Close.

  7. If you intend to enable zone awareness for your Amazon ES domain, you must create a second subnet in a different Availability Zone in the same region. If you don't intend to enable zone awareness, skip to step 8.

    1. In the navigation pane, choose Subnets.

    2. Choose Create Subnet.

    3. In the Create Subnet dialog box, optionally create a name tag to help you identify the subnet later.

    4. For VPC, choose the VPC that you just created.

    5. For Availability Zone, choose an Availability Zone that differs from that of the first subnet. The Availability Zones for both subnets must be in the same region.

    6. For IPv4 CIDR block, configure a CIDR block large enough to provide sufficient IP addresses for Amazon ES to use during maintenance activities. For more information, see Reserving IP Addresses in a VPC Subnet.

      Note

      Amazon ES domains using VPC access don't support IPv6 addresses. You can use a VPC that has IPv6 enabled, but the ENIs will have IPv4 addresses.

    7. Choose Yes, Create.

  8. In the navigation pane, choose Subnets.

  9. In the list of subnets, find your subnet (or subnets, if you created a second subnet in step 7), and in the Available IPv4 column, confirm that you have a sufficient number of IPv4 addresses.

  10. Make a note of the subnet ID and Availability Zone. You will need this information later when you launch your Amazon ES domain and add an Amazon EC2 instance to your VPC.

  11. Create an Amazon VPC security group. You will use this security group to control access to your Amazon ES domain.

    1. In the navigation pane, choose Security Groups.

    2. Choose Create Security Group.

    3. In the Create Security Group dialog box, type a name tag, a group name, and a description. For VPC, choose the ID of your VPC.

    4. Choose Yes, Create.

  12. Define a network ingress rule for your security group. This rule will allow you to connect to your Amazon ES domain.

    1. In the navigation pane, choose Security Groups, and then select the security group that you just created.

    2. At the bottom of the page, choose the Inbound Rules tab.

    3. Choose Edit, and then choose HTTPS (443).

    4. Choose Save.

Now you are ready to launch an Amazon ES domain in your Amazon VPC.

Reserving IP Addresses in a VPC Subnet

Amazon ES connects a domain to a VPC by placing network interfaces in a subnet of the VPC (or two subnets of the VPC if you enable zone awareness). Each network interface is associated with an IP address. Before you create your Amazon ES domain, you must have a sufficient number of IP addresses available in the VPC subnet to accommodate the network interfaces.

The number of IP addresses that Amazon ES needs depends on the following:

  • Number of data nodes in your domain. (Master nodes are not included in the number.)

  • Whether you enable zone awareness. If you enable zone awareness, you need only half the number of IP addresses that you need if you don't enable zone awareness.

The basic formula follows: the number of IP addresses reserved in each subnet is three times the number of nodes, divided by two if zone awareness is enabled.

Examples

  • If a domain has 10 data nodes and zone awareness is enabled, then the IP count is 10 / 2 * 3 = 15.

  • If a domain has 10 data nodes and zone awareness is disabled, then the IP count is 10 * 3 = 30.

When you create the domain, Amazon ES reserves the IP addresses. You can see the network interfaces and their associated IP addresses in the Network Interfaces section of the Amazon EC2 console at https://console.aws.amazon.com/ec2/. The Description column shows which Amazon ES domain the network interface is associated with.

Tip

We recommend creating dedicated subnets for the Amazon ES reserved IP addresses. By using dedicated subnets, you avoid overlap with other applications and services and ensure that you can reserve additional IP addresses if you need to scale your cluster in the future. To learn more, see Creating a Subnet in Your VPC.

Service-Linked Role for VPC Access

A service-linked role is a unique type of IAM role that delegates permissions to a service so that it can create and manage resources on your behalf. Amazon ES requires a service-linked role to access your VPC, create the domain endpoint, and place network interfaces in a subnet of your VPC.

Amazon ES automatically creates the role when you use the Amazon ES console to create a domain within a VPC. In order for this automatic creation to succeed, you must have permissions for the iam:CreateServiceLinkedRole action. To learn more, see Service-Linked Role Permissions in the IAM User Guide.

After Amazon ES creates the role, you can view it (AWSServiceRoleForAmazonElasticsearchService) using the IAM console.

Note

If you create a domain that uses a public endpoint, Amazon ES doesn’t need the service-linked role and doesn't create it.

For full information on this role's permissions and how to delete it, see Using Service-Linked Roles for Amazon ES.

Migrating from Public Access to VPC Access

When you create a domain, you specify whether it should have a public endpoint or reside within a VPC. Once created, you cannot switch from one to the other. Instead, you must create a new domain and either manually reindex or migrate your data. Snapshots offer a convenient means of migrating data. For information on taking and restoring snapshots, see Working with Amazon Elasticsearch Service Index Snapshots.

About Access Policies on VPC Domains

Placing your Amazon ES domain within a VPC provides an inherent, strong layer of security. The VPC lets you manage access to the domain through security groups. For many use cases, this level of security is sufficient, and you might feel comfortable applying an open access policy to the domain.

Operating with an open access policy does not mean that anyone on the Internet can access the Amazon ES domain. Rather, it means that if a request comes from the proper security group, the domain accepts it without further security checks.

For an additional layer of security, we recommend applying IAM user- and role-based policies to the Amazon ES domain. Applying one of these policies means that, for the domain to accept a request, it must come from the proper security group and be signed with valid credentials.

Note

Because security groups already enforce IP-based access policies, you can't apply IP-based access policies to Amazon ES domains that reside within a VPC. If you are using a public endpoint, IP-based policies are still available.

Amazon VPC Documentation

Amazon VPC has its own set of documentation to describe how to create and use your Amazon VPC. The following table provides links to the Amazon VPC guides.

Description Documentation
How to get started using Amazon VPC Amazon VPC Getting Started Guide
How to use Amazon VPC through the AWS Management Console Amazon VPC User Guide
Complete descriptions of all the Amazon VPC commands Amazon EC2 Command Line Reference (the Amazon VPC commands are part of the Amazon EC2 reference)
Complete descriptions of the Amazon VPC API actions, data types, and errors Amazon EC2 API Reference (the Amazon VPC API actions are part of the Amazon EC2 reference)
Information for the network administrator who needs to configure the gateway at your end of an optional IPsec VPN connection Amazon VPC Network Administrator Guide

For more detailed information about Amazon Virtual Private Cloud, see Amazon Virtual Private Cloud.