Help improve this page
To contribute to this user guide, choose the Edit this page on GitHub link that is located in the right pane of every page.
Create a GPU NodePool for AI/ML inference
Tip
Register
This section defines an EKS Auto Mode or Karpenter NodePool in your Amazon EKS cluster that dynamically provisions Amazon EC2 G-family GPU instances, using Spot capacity with On-Demand fallback. You can optionally add reserved capacity through an Amazon EC2 On-Demand Capacity Reservation (ODCR). The GPU instances are automatically created in EKS Auto Mode or Karpenter as workloads are run on the cluster.
This section builds on the cluster you created in the Set up Amazon EKS cluster for AI/ML workloads section. Complete that section first, then choose the same path (EKS Auto Mode or self-managed Karpenter) that you used to create your cluster.
-
For step-by-step instructions using the AWS CLI, see Create a GPU NodePool using CLIs.
-
For step-by-step instructions using Terraform, see Create a GPU NodePool using Terraform.
After your GPU NodePool is ready, continue to Load & Serve Models on Amazon EKS to deploy a large language model and serve inference requests.
