Amazon EKS VPC and subnet requirements and considerations
When you create a cluster, you specify a VPC and at least two subnets that are in different Availability Zones. This topic provides an overview of Amazon EKS specific requirements and considerations for the VPC and subnets that you use with your cluster. If you don't have a VPC to use with Amazon EKS, you can create one using an Amazon EKS provided AWS CloudFormation template. If you're creating a local or extended cluster on AWS Outposts, see Amazon EKS local cluster VPC and subnet requirements and considerations instead of this topic.
VPC requirements and considerations
When you create a cluster, the VPC that you specify must meet the following requirements and considerations:
-
The VPC must have a sufficient number of IP addresses available for the cluster, any nodes, and other Kubernetes resources that you want to create. If the VPC that you want to use doesn't have a sufficient number of IP addresses, try to increase the number of available IP addresses. You can do this by associating additional Classless Inter-Domain Routing (CIDR) blocks with your VPC. You can associate private (RFC 1918) and public (non-RFC 1918) CIDR blocks to your VPC either before or after you create your cluster. It can take a cluster up to five hours for a CIDR block that you associated with a VPC to be recognized.
You can conserve IP address utilization by using a transit gateway with a shared services VPC. For more information, see Isolated VPCs with shared services and Amazon EKS VPC routable IP address conservation patterns in a hybrid network
. -
If you want Kubernetes to assign
IPv6
addresses to Pods and services, associate anIPv6
CIDR block with your VPC. For more information, see Associate anIPv6
CIDR block with your VPC in the Amazon VPC User Guide. -
The VPC must have
DNS
hostname andDNS
resolution support. Otherwise, nodes can't register to your cluster. For more information, see DNS attributes for your VPC in the Amazon VPC User Guide. -
The VPC might require VPC endpoints using AWS PrivateLink. For more information, see Subnet requirements and considerations.
If you created a cluster with Kubernetes 1.14
or earlier, Amazon EKS added the
following tag to your VPC:
Key | Value |
---|---|
kubernetes.io/cluster/ |
owned |
This tag was only used by Amazon EKS. You can remove the tag without impacting your
services. It's not used with clusters that are version 1.15
or
later.
Subnet requirements and considerations
When you create a cluster, Amazon EKS creates 2–4 elastic network interfaces in
the subnets that you specify. These network interfaces enable communication between your
cluster and your VPC. These network interfaces also enable Kubernetes features such as
kubectl exec
and kubectl logs
. Each Amazon EKS created network
interface has the text Amazon EKS
in its
description.cluster-name
Amazon EKS can create its network interfaces in any subnet that you specify when you create a cluster. You can't change which subnets Amazon EKS creates its network interfaces in after your cluster is created. When you update the Kubernetes version of a cluster, Amazon EKS deletes the original network interfaces that it created, and creates new network interfaces. These network interfaces might be created in the same subnets as the original network interfaces or in different subnets than the original network interfaces. To control which subnets network interfaces are created in, you can limit the number of subnets you specify to only two when you create a cluster.
The subnets that you specify when you create a cluster must meet the following requirements:
-
The subnets must each have at least six IP addresses for use by Amazon EKS. However, we recommend at least 16 IP addresses.
-
The subnets can't reside in AWS Outposts, AWS Wavelength, or an AWS Local Zone. However, if you have them in your VPC, you can deploy self-managed nodes and Kubernetes resources to these types of subnets.
-
The subnets can be a public or private. However, we recommend that you specify private subnets, if possible. A public subnet is a subnet with a route table that includes a route to an internet gateway, whereas a private subnet is a subnet with a route table that doesn't include a route to an internet gateway.
You can deploy nodes and Kubernetes resources to the same subnets that you specify when you create your cluster. However, this isn't necessary. This is because you can also deploy nodes and Kubernetes resources to subnets that you didn't specify when you created the cluster. If you deploy nodes to different subnets, Amazon EKS doesn't create cluster network interfaces in those subnets. Any subnet that you deploy nodes and Kubernetes resources to must meet the following requirements:
-
The subnets must have enough available IP addresses to deploy all of your nodes and Kubernetes resources to.
-
If you want Kubernetes to assign
IPv6
addresses to Pods and services, then you must have oneIPv6
CIDR block and oneIPv4
CIDR block that are associated with your subnet. For more information, see Associate anIPv6
CIDR block with your subnet in the Amazon VPC User Guide. The route tables that are associated with the subnets must include routes toIPv4
andIPv6
addresses. For more information, see Routes in the Amazon VPC User Guide. Pods are assigned only anIPv6
address. However the network interfaces that Amazon EKS creates for your cluster and your nodes are assigned anIPv4
and anIPv6
address. -
If you need inbound access from the internet to your Pods, make sure to have at least one public subnet with enough available IP addresses to deploy load balancers and ingresses to. You can deploy load balancers to public subnets. Load balancers can load balance to Pods in private or public subnets. We recommend deploying your nodes to private subnets, if possible.
-
If you plan to deploy nodes to a public subnet, the subnet must auto-assign
IPv4
public addresses orIPv6
addresses. If you deploy nodes to a private subnet that has an associatedIPv6
CIDR block, the private subnet must also auto-assignIPv6
addresses. If you used an Amazon EKS AWS CloudFormation template to deploy your VPC after March 26, 2020, this setting is enabled. If you used the templates to deploy your VPC before this date or you use your own VPC, you must enable this setting manually. For more information, see Modify the publicIPv4
addressing attribute for your subnet and Modify theIPv6
addressing attribute for your subnet in the Amazon VPC User Guide. -
If the subnet that you deploy a node to is a private subnet and its route table doesn't include a route to a network address translation (NAT) device (
IPv4
) or an egress-only gateway (IPv6
), add VPC endpoints using AWS PrivateLink to your VPC. VPC endpoints are needed for all the AWS services that your nodes and Pods need to communicate with. Examples include Amazon ECR, Elastic Load Balancing, Amazon CloudWatch, AWS Security Token Service, and Amazon Simple Storage Service (Amazon S3). The endpoint must include the subnet that the nodes are in. Not all AWS services support VPC endpoints. For more information, see What is AWS PrivateLink? and AWS services that integrate with AWS PrivateLink. For a list of more Amazon EKS requirements, see Private cluster requirements. -
If you want to deploy load balancers to a subnet, the subnet must have the following tag:
-
Private subnets
Key Value kubernetes.io/role/internal-elb
1
-
Public subnets
Key Value kubernetes.io/role/elb
1
-
When a Kubernetes cluster that's version 1.18
and earlier was created, Amazon EKS
added the following tag to all of the subnets that were specified.
Key | Value |
---|---|
kubernetes.io/cluster/ |
shared |
When you create a new Kubernetes cluster now, Amazon EKS doesn't add the tag to your subnets. If
the tag was on subnets that were used by a cluster that was previously a version earlier
than 1.19
, the tag wasn't automatically removed from the subnets when the
cluster was updated to a newer version. Version 2.1.1
or earlier of the
AWS Load Balancer Controller requires this tag. If you
are using a newer version of the Load Balancer Controller, you can remove the tag
without interrupting your services.
If you deployed a VPC by using eksctl
or any
of the Amazon EKS AWS CloudFormation VPC templates, the following applies:
-
On or after March 26, 2020 – Public
IPv4
addresses are automatically assigned by public subnets to new nodes that are deployed to public subnets. -
Before March 26, 2020 – Public
IPv4
addresses aren't automatically assigned by public subnets to new nodes that are deployed to public subnets.
This change impacts new node groups that are deployed to public subnets in the following ways:
-
Managed node groups – If the node group is deployed to a public subnet on or after April 22, 2020, automatic assignment of public IP addresses must be enabled for the public subnet. For more information, see Modifying the public
IPv4
addressing attribute for your subnet. -
Linux, Windows, or Arm self-managed node groups – If the node group is deployed to a public subnet on or after March 26, 2020, automatic assignment of public IP addresses must be enabled for the public subnet. Otherwise, the nodes must be launched with a public IP address instead. For more information, see Modifying the public
IPv4
addressing attribute for your subnet or Assigning a publicIPv4
address during instance launch.
Shared subnet requirements and considerations
You can use VPC sharing to share subnets with other AWS accounts within the same AWS Organizations. You can create Amazon EKS clusters in shared subnets, with the following considerations:
-
The owner of the VPC subnet must share a subnet with a participant account before that account can create an Amazon EKS cluster in it.
-
You can't launch resources using the default security group for the VPC because it belongs to the owner. Additionally, participants can't launch resources using security groups that are owned by other participants or the owner.
-
In a shared subnet, the participant and the owner separately controls the security groups within each respective account. The subnet owner can see security groups that are created by the participants but cannot perform any actions on them. If the subnet owner wants to remove or modify these security groups, the participant that created the security group must take the action.
-
If a cluster is created by a participant, the following considerations apply:
Cluster IAM role and Node IAM roles must be created in that account. For more information, see Amazon EKS cluster IAM role and Amazon EKS node IAM role.
-
All nodes must be made by the same participant, including managed node groups.
-
The shared VPC owner cannot view, update or delete a cluster that a participant creates in the shared subnet. This is in addition to the VPC resources that each account has different access to. For more information, see Responsibilities and permissions for owners and participants in the Amazon VPC User Guide.
-
If you use the custom networking feature of the Amazon VPC CNI plugin for Kubernetes, you need to use the Availability Zone ID mappings listed in the owner account to create each
ENIConfig
. For more information, see Custom networking for pods.
For more information about VPC subnet sharing, see Share your VPC with other accounts in the Amazon VPC User Guide.