Help improve this page
Want to contribute to this user guide? Choose the đź“ť Edit this page on GitHub link that is located at the bottom of every page. Your contributions will help make our user guide better for everyone.
Create a Node Pool for EKS Auto Mode
Amazon EKS node pools provide a flexible way to manage compute resources in your Kubernetes cluster. This topic demonstrates how to create and configure node pools using Karpenter, a node provisioning tool that helps optimize cluster scaling and resource utilization. With Karpenter’s NodePool resource, you can define specific requirements for your compute resources, including instance types, availability zones, architectures, and capacity types.
You cannot modify the built in system
and general-purpose
node pools. You can only enable or disable them. For more information, see Enable or Disable Built-in NodePools.
The NodePool specification allows for fine-grained control over your EKS cluster’s compute resources through various supported labels and requirements. These include options for specifying EC2 instance categories, CPU configurations, availability zones, architectures (ARM64/AMD64), and capacity types (spot/on-demand). You can also set resource limits for CPU and memory usage, ensuring your cluster stays within desired operational boundaries.
EKS Auto Mode leverages well-known Kubernetes labels to provide consistent and standardized ways of identifying node characteristics. These labels, such as topology.kubernetes.io/zone
for availability zones and kubernetes.io/arch
for CPU architecture, follow established Kubernetes conventions. Additionally, EKS-specific labels (prefixed with eks.amazonaws.com/
) extend this functionality with AWS-specific attributes like instance types, CPU manufacturers, GPU capabilities, and networking specifications. This standardized labeling system enables seamless integration with existing Kubernetes tooling while providing deep AWS infrastructure integration.
Create a NodePool
Follow these steps to create a NodePool for your Amazon EKS cluster:
-
Create a YAML file named
nodepool.yaml
with your desired NodePool configuration. You can use the sample configuration below. -
Apply the NodePool to your cluster:
kubectl apply -f nodepool.yaml
-
Verify that the NodePool was created successfully:
kubectl get nodepools
-
(Optional) Monitor the NodePool status:
kubectl describe nodepool default
Ensure that your NodePool references a valid NodeClass that exists in your cluster. The NodeClass defines AWS-specific configurations for your compute resources. For more information, see Create a Node Class for Amazon EKS.
Sample NodePool
apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: my-node-pool spec: template: metadata: labels: billing-team: my-team spec: nodeClassRef: group: eks.amazonaws.com kind: NodeClass name: default requirements: - key: "eks.amazonaws.com/instance-category" operator: In values: ["c", "m", "r"] - key: "eks.amazonaws.com/instance-cpu" operator: In values: ["4", "8", "16", "32"] - key: "topology.kubernetes.io/zone" operator: In values: ["us-west-2a", "us-west-2b"] - key: "kubernetes.io/arch" operator: In values: ["arm64", "amd64"] limits: cpu: "1000" memory: 1000Gi
EKS Auto Mode Supported Labels
EKS Auto Mode supports the following well known labels.
Label | Example | Description |
---|---|---|
topology.kubernetes.io/zone |
us-east-2a |
AWS region |
node.kubernetes.io/instance-type |
g4dn.8xlarge |
AWS instance type |
kubernetes.io/arch |
amd64 |
Architectures are defined by GOARCH values |
karpenter.sh/capacity-type |
spot |
Capacity types include |
eks.amazonaws.com/instance-hypervisor |
nitro |
Instance types that use a specific hypervisor |
eks.amazonaws.com/compute-type |
auto |
Identifies EKS Auto Mode managed nodes |
eks.amazonaws.com/instance-encryption-in-transit-supported |
true |
Instance types that support (or not) in-transit encryption |
eks.amazonaws.com/instance-category |
g |
Instance types of the same category, usually the string before the generation number |
eks.amazonaws.com/instance-generation |
4 |
Instance type generation number within an instance category |
eks.amazonaws.com/instance-family |
g4dn |
Instance types of similar properties but different resource quantities |
eks.amazonaws.com/instance-size |
8xlarge |
Instance types of similar resource quantities but different properties |
eks.amazonaws.com/instance-cpu |
32 |
Number of CPUs on the instance |
eks.amazonaws.com/instance-cpu-manufacturer |
aws |
Name of the CPU manufacturer |
eks.amazonaws.com/instance-memory |
131072 |
Number of mebibytes of memory on the instance |
eks.amazonaws.com/instance-ebs-bandwidth |
9500 |
Number of maximum megabits of EBS available on the instance |
eks.amazonaws.com/instance-network-bandwidth |
131072 |
Number of baseline megabits available on the instance |
eks.amazonaws.com/instance-gpu-name |
t4 |
Name of the GPU on the instance, if available |
eks.amazonaws.com/instance-gpu-manufacturer |
nvidia |
Name of the GPU manufacturer |
eks.amazonaws.com/instance-gpu-count |
1 |
Number of GPUs on the instance |
eks.amazonaws.com/instance-gpu-memory |
16384 |
Number of mebibytes of memory on the GPU |
eks.amazonaws.com/instance-local-nvme |
900 |
Number of gibibytes of local nvme storage on the instance |
Note
EKS Auto Mode only supports certain instances, and has minimum size requirements. For more information, see EKS Auto Mode supported instance reference.
EKS Auto Mode Not Supported Labels
EKS Auto Mode does not support the following labels.
-
EKS Auto Mode only supports Linux
-
node.kubernetes.io/windows-build
-
kubernetes.io/os
-
Disable built-in node pools
If you create custom node pools, you can disable the built-in node pools. For more information, see Enable or Disable Built-in NodePools.
Cluster without built-in node pools
You can create a cluster without the built-in node pools. This is helpful when your organization has created customized node pools.
Overview:
-
Create an EKS cluster with the both
nodePools
andnodeRoleArn
values empty.-
Sample eksctl
autoModeConfig
:autoModeConfig: enabled: true nodePools: [] # Do not set a nodeRoleARN
For more information, see Create an EKS Auto Mode Cluster with the eksctl CLI
-
-
Create a custom node class with a node role ARN
-
For more information, see Create a Node Class for Amazon EKS
-
-
Create an access entry for the custom node class
-
For more information, see Create node class access entry
-
-
Create a custom node pool, as described above.
Disruption
You can configure EKS Auto Mode to disrupt Nodes through your NodePool in multiple ways. You can use spec.disruption.consolidationPolicy
, spec.disruption.consolidateAfter
, or spec.template.spec.expireAfter
. You can also rate limit EKS Auto Mode’s disruption through the NodePool’s spec.disruption.budgets
. You can also control the time windows and number of simultaneous Nodes disrupted. For instructions on configuring this behavior, see Disruption
You can configure disruption for node pools to:
-
Identify when instances are underutilized, and consolidate workloads.
-
Create a node pool disruption budget to rate limit node terminations due to drift, emptiness, and consolidation.
By default, EKS Auto Mode:
-
Consolidates underutilized instances.
-
Terminates instances after 720 hours.
-
Sets a single disruption budget of 10% of nodes.