Set up an Amazon EKS cluster - Amazon EMR

Set up an Amazon EKS cluster

Amazon EKS is a managed service that makes it easy for you to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane or nodes. Follow the steps outlined below to create a new Kubernetes cluster with nodes in Amazon EKS.

Prerequisites

Before creating an Amazon EKS cluster, you must install and configure the following tools and resources that you need to create and manage an Amazon EKS cluster,

  • The latest version of AWS CLI.

  • kubectl version 1.20 or later.

  • The latest version of eksctl .

For more information, see Install the AWS CLI, Installing kubectl, Install eksctl.

Create an Amazon EKS cluster using eksctl

Take the following steps to create an Amazon EKS cluster using eksctl.

Important

To get started quickly, you can create an EKS cluster and the nodes with default settings. But for production use, we recommend that you customize the settings for the cluster and nodes to meet your specific requirements. For a list of all settings and options, run the command eksctl create cluster -h. For more information, see Creating and Managing Clusters in the eksctl documentation.

  1. Create an Amazon EC2 key pair.

    If you don't have an existing key pair, you can run the following command to create a new key pair. Replace us-west-2 with the Region where you want to create your cluster.

    aws ec2 create-key-pair --region us-west-2 --key-name myKeyPair

    Save the returned output in a file on your local computer. For more information, see Creating or importing a key pair in the Amazon EC2 User Guide for Linux Instances.

    Note

    A key pair is not required for creating an EKS cluster. But specifying the key pair allows you to SSH to nodes once they're created. You can specify a key pair only when you create the node group.

  2. Create an EKS cluster.

    Run the following command to create an EKS cluster and nodes. Replace my-cluster and myKeyPair with your own cluster name and key pair name. Replace us-west-2 with the Region where you want to create your cluster. For more information about Amazon EKS supported Regions, see Amazon Elastic Kubernetes Service endpoints and quotas.

    eksctl create cluster \ --name my-cluster \ --region us-west-2 \ --with-oidc \ --ssh-access \ --ssh-public-key myKeyPair \ --instance-types=m5.xlarge \ --managed
    Important

    When creating an EKS cluster, use m5.xlarge as the instance type, or any other instance type with a higher CPU and memory. Using an instance type with lower CPU or memory compared to m5.xlarge may lead to job failure due to insufficient resources available in the cluster. To see all resources created, view the stack named eksctl-my-cluster-cluster in the AWS Cloud Formation console.

    The cluster and node creation process takes several minutes. You'll see several lines of output when the cluster and nodes are created. The following example demonstrates the last line of output.

    ... [✓] EKS cluster "my-cluster" in "us-west-2" region is ready

    eksctl created a kubectl config file in ~/.kube or added the new cluster's configuration within an existing config file in ~/.kube.

  3. View and validate resources

    Run the following command to view your cluster nodes.

    kubectl get nodes -o wide

    The following shows an example output.

    Amazon EC2 node output NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ip-192-168-12-49.us-west-2.compute.internal Ready none 6m7s v1.18.9-eks-d1db3c 192.168.12.49 52.35.116.65 Amazon Linux 2 4.14.209-160.335.amzn2.x86_64 docker://19.3.6 ip-192-168-72-129.us-west-2.compute.internal Ready none 6m4s v1.18.9-eks-d1db3c 192.168.72.129 44.242.140.21 Amazon Linux 2 4.14.209-160.335.amzn2.x86_64 docker://19.3.6

    For more information, see View nodes.

    Use the following command to view the workloads running on your cluster.

    kubectl get pods --all-namespaces -o wide

    The following shows an example output.

    Amazon EC2 output NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system aws-node-6ctpm 1/1 Running 0 7m43s 192.168.72.129 ip-192-168-72-129.us-west-2.compute.internal none none kube-system aws-node-cbntg 1/1 Running 0 7m46s 192.168.12.49 ip-192-168-12-49.us-west-2.compute.internal none none kube-system coredns-559b5db75d-26t47 1/1 Running 0 14m 192.168.78.81 ip-192-168-72-129.us-west-2.compute.internal none none kube-system coredns-559b5db75d-9rvnk 1/1 Running 0 14m 192.168.29.248 ip-192-168-12-49.us-west-2.compute.internal none none kube-system kube-proxy-l8pbd 1/1 Running 0 7m46s 192.168.12.49 ip-192-168-12-49.us-west-2.compute.internal none none kube-system kube-proxy-zh85h 1/1 Running 0 7m43s 192.168.72.129 ip-192-168-72-129.us-west-2.compute.internal none none

    For more information about what you see here, see View workloads.

Create an EKS cluster using AWS Management Console and AWS CLI

You can also use AWS Management Console and AWS CLI to create an EKS cluster. Follow the steps at Getting started with Amazon EKS – AWS Management Console and AWS CLI. This way gives you visibility into how each resource is created for the EKS cluster and how the resources interact with each other.

Important

When creating nodes for an EKS cluster, use m5.xlarge as the instance type, or any other instance type with a higher CPU and memory.

Create an EKS cluster with AWS Fargate

You can also create an EKS cluster with pods running on AWS Fargate.

  1. To create an EKS cluster with pods running on Fargate, follow the steps outlined at Getting Started with AWS Fargate using Amazon EKS.

    Note

    Amazon EMR on EKS needs CoreDNS for running jobs on EKS cluster. If you want to run your pods only on Fargate, you must follow the steps at Updating CoreDNS.

  2. Run the following command to view your cluster nodes.

    kubectl get nodes -o wide

    The following shows an example Fargate output.

    Fargate node output NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME fargate-ip-192-168-141-147.us-west-2.compute.internal Ready none 8m3s v1.18.8-eks-7c9bda 192.168.141.147 none Amazon Linux 2 4.14.209-160.335.amzn2.x86_64 containerd://1.3.2 fargate-ip-192-168-164-53.us-west-2.compute.internal Ready none 7m30s v1.18.8-eks-7c9bda 192.168.164.53 none Amazon Linux 2 4.14.209-160.335.amzn2.x86_64 containerd://1.3.2

    For more information, see View nodes.

  3. Run the following command to view the workloads running on your cluster.

    kubectl get pods --all-namespaces -o wide

    The following shows an example Fargate output.

    Fargate output NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system coredns-69dfb8f894-9z95l 1/1 Running 0 18m 192.168.164.53 fargate-ip-192-168-164-53.us-west-2.compute.internal none none kube-system coredns-69dfb8f894-c8v66 1/1 Running 0 18m 192.168.141.147 fargate-ip-192-168-141-147.us-west-2.compute.internal none none

    For more information, see View workloads.