Private clusters - Amazon EKS

Private clusters

This topic describes how to deploy a private cluster without outbound internet access. If you're not familiar with Amazon EKS networking, see De-mystifying cluster networking for Amazon EKS worker nodes.

Requirements

The following requirements must be met to run Amazon EKS in a private cluster without outbound internet access.

  • A container image must be in or copied to Amazon Elastic Container Registry (Amazon ECR) or to a registry inside the VPC to be pulled. For more information, see Creating local copies of container images.

  • Endpoint private access is required for nodes to register with the cluster endpoint. Endpoint public access is optional. For more information, see Amazon EKS cluster endpoint access control.

  • You may need to include the VPC endpoints found at VPC endpoints for private clusters.

  • You must include the following text to the bootstrap arguments when launching self-managed nodes. This text bypasses the Amazon EKS introspection and does not require access to the Amazon EKS API from within the VPC. Replace <cluster-endpoint> and <cluster-certificate-authority> with the values from your Amazon EKS cluster.

    --apiserver-endpoint <cluster-endpoint> --b64-cluster-ca <cluster-certificate-authority>
  • The aws-auth ConfigMap must be created from within the VPC. For more information about create the aws-auth ConfigMap, see Managing users or IAM roles for your cluster.

Considerations

Here are some things to consider when running Amazon EKS in a private cluster without outbound internet access.

Creating local copies of container images

Because a private cluster has no outbound internet access, container images cannot be pulled from external sources such as Docker Hub. Instead, container images must be copied locally to Amazon ECR or to an alternative registry accessible in the VPC. A container image can be copied to Amazon ECR from outside the private VPC. The private cluster accesses the Amazon ECR repository using the Amazon ECR VPC endpoints. You must have Docker and the AWS CLI installed on the workstation that you use to create the local copy.

To create a local copy of a container image

  1. Create an Amazon ECR repository. For more information, see Creating a repository.

  2. Pull the container image from the external registry using docker pull.

  3. Tag your image with the Amazon ECR registry, repository, and optional image tag name combination using docker tag.

  4. Authenticate to the registry. For more information, see Registry authentication.

  5. Push the image to Amazon ECR using docker push.

    Note

    Be sure to update your resource configuration to use the new image location.

    The following example pulls the amazon/aws-node-termination-handler image, using tag v1.3.1-linux-amd64, from Docker Hub and creates a local copy in Amazon ECR.

    aws ecr create-repository --repository-name amazon/aws-node-termination-handler docker pull amazon/aws-node-termination-handler:v1.3.1-linux-amd64 docker tag amazon/aws-node-termination-handler <111122223333>.dkr.ecr.<region-code>.amazonaws.com/amazon/aws-node-termination-handler:v1.3.1-linux-amd64 aws ecr get-login-password --region <region-code> | docker login --username AWS --password-stdin <111122223333>.dkr.ecr.<region-code>.amazonaws.com docker push <111122223333>.dkr.ecr.<region-code>.amazonaws.com/amazon/aws-node-termination-handler:v1.3.1-linux-amd64

VPC endpoints for private clusters

The following VPC endpoints may be required.

  • com.amazonaws.<region>.ec2

  • com.amazonaws.<region>.ecr.api

  • com.amazonaws.<region>.ecr.dkr

  • com.amazonaws.<region>.s3 – For pulling container images

  • com.amazonaws.<region>.logs – For CloudWatch Logs

  • com.amazonaws.<region>.sts – If using AWS Fargate or IAM roles for service accounts

  • com.amazonaws.<region>.elasticloadbalancing – If using Application Load Balancers

  • com.amazonaws.<region>.autoscaling – If using Cluster Autoscaler

  • com.amazonaws.<region>.appmesh-envoy-management – If using App Mesh