Provisioning production-ready Amazon EKS clusters using Terraform

Jomcy Pappachen, Vara Bonthu, and Ulaganathan N, Amazon Web Services (AWS)

July 2021 (document history)

Kubernetes is an open-source system for automating and managing containerized applications at scale. Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service that runs container application workloads and helps standardize operations across your environments (for example, production or development environments). You can manage modern infrastructures by using infrastructure as code (IaC) practices with tools such as AWS CloudFormation, AWS Cloud Development Kit (AWS CDK) , or Terraform by Hashicorp. This guide is intended for solution architects and technical leaders who are responsible for designing production-ready Amazon EKS clusters to run modernized workloads. The solution uses Terraform to build an IaC framework that provisions a multi-tenant Amazon EKS cluster. The guide describes the outcomes, design, architecture, and implementation of Amazon EKS clusters for running modernized application workloads.

By using this guide's solution, you can quickly create the infrastructure to migrate live-traffic serving self-hosted Kubernetes clusters to Amazon EKS on the AWS Cloud. The guide also provides a framework to help you design and create Amazon EKS clusters, each with a unique Terraform configuration and state file, in different environments across multiple AWS accounts and AWS Regions. When you want to modernize your applications with microservices and Kubernetes, you can use this guide and its reference code in the GitHub aws-eks-accelerator-for-terraform repository to build the Amazon EKS infrastructure on the AWS Cloud. This provisions Amazon EKS clusters, managed node groups with On-Demand and Spot Amazon Elastic Compute Cloud (Amazon EC2) instance types, AWS Fargate profiles, and plugins or add-ons for creating production-ready Amazon EKS clusters. The Terraform Helm provider also deploys common Kubernetes add-ons by using Helm charts.

The guide has the following four sections:

• Development environment requirements and code repository – Provides the software, tools, and GitHub repository to implement this guide's solution.

• High-level architecture – Explains the high-level architectural design of the guide's solution.

• Helm add-ons and Kubernetes Cluster Autoscaler – Describes how to implement the Helm modules by using Terraform Helm provider and how the Kubernetes Cluster Autoscaler helps scale Amazon EKS clusters.

• Logging and monitoring Amazon EKS clusters – Discusses the centralized logging and monitoring solutions that can be implemented for Amazon EKS clusters.

After provisioning the Amazon EKS clusters, you can use the code examples from the GitHub aws-eks-accelerator-for-terraform repository. However, this guide doesn't provide a complete overview of all implementations and we recommend that you carefully evaluate all third-party or open-source tools according to your organization's policies and requirements.

Solution outcomes

You should expect the following eight outcomes from deploying this guide’s solution in your AWS accounts:

• Enable your cross-functional teams to use the same Amazon EKS cluster by provisioning Amazon EKS clusters that support multi-tenancy based on applications and namespaces.

• Provision Amazon EKS clusters in new or existing virtual private clouds (VPCs), which means that you can use existing VPCs if required.

• Define your scaling metrics as a Kubernetes manifest by using Kubernetes Horizontal Pod Autoscaling and configurable options for expanding resource quotas and pod security policies.

• Ensure role-based access control (RBAC) for your developers and administrators by using AWS Identity and Access Management (IAM) roles.

• Deploy a private Amazon EKS cluster to secure your application and meet your compliance requirements.

• Monitor and log applications and system pods by using Amazon CloudWatch to collect and track metrics.

• Flexibly provision your Amazon EKS clusters with different node group types by running a combination of self-managed nodes, Amazon EKS managed node groups, and Fargate.

• Deploy a Bottlerocket Amazon Machine Image (AMI) in self-managed node groups to run container workloads in a purpose-built operating system (OS) on the AWS Cloud.