View a markdown version of this page

How resource optimization works with autoscalers and workload types - Amazon EKS

Help improve this page

To contribute to this user guide, choose the Edit this page on GitHub link that is located in the right pane of every page.

How resource optimization works with autoscalers and workload types

EKS Auto Mode resource optimization recommends CPU and memory requests for your workloads based on the usage it observes for each container, and in Apply mode resizes Pods to match. If you already run autoscalers, Java services, or Pods with sidecars, use this topic to learn what resource optimization changes for those workloads and what it leaves alone.

This topic covers the following sections.

For each workload that a policy selects, resource optimization creates a ResourceOptimization object (short name ro) in the workload’s namespace. Its name is the workload name followed by -deploy for a Deployment or -sts for a StatefulSet. The behavior in this topic appears as conditions and status fields on that object.

Run the following kubectl command to view the object, replacing web-deploy with the name of your workload’s ResourceOptimization object and shop with its namespace:

kubectl get ro web-deploy -n shop -o yaml

Supported workloads

Resource optimization manages Deployments and StatefulSets. It doesn’t manage DaemonSets, Jobs, CronJobs, ReplicaSets that you create directly, Pods without a controller, or custom workload controllers.

Resource optimization never manages workloads in the kube-system, kube-public, or kube-node-lease namespaces, even when a policy selects them.

If a workload’s Pod template sets the isolated-sandbox or confidential-sandbox RuntimeClass, resource optimization recommends for it. It never changes those Pods. A sandboxed Pod’s CPU and memory are fixed when it starts, so it can’t be resized in place. For these workloads, resource optimization doesn’t do any of the following:

  • Resize running Pods

  • Size new Pods

  • Raise memory after an out-of-memory kill

  • Hand back startup CPU

The ResourceOptimization shows the InPlaceResizeUnsupported condition as True with the reason SandboxRuntime. For more information, see How resource optimization applies changes to Pods.

To exclude a single workload, add the optimization.eks.amazonaws.com/enabled: "false" annotation to the Deployment or StatefulSet, not to its Pod template. When you opt out a workload, resource optimization deletes its ResourceOptimization object. It stops making recommendations and cancels any resize that the kubelet hasn’t applied yet. Resizes that the kubelet already applied stay in place, and new Pods start at the size in the Pod template. For the procedure, see Opt out a single workload.

Horizontal Pod Autoscaler and KEDA

Resource optimization doesn’t change the CPU or memory that a Horizontal Pod Autoscaler (HPA) scales on. If resource optimization lowered a request that an HPA measures utilization against, the HPA would read higher utilization and add replicas. The two would then work against each other.

Resource optimization stops managing a resource only when an HPA has a Resource or ContainerResource metric for cpu or memory with a target of type Utilization. It keeps managing the workload’s other resources. The following table shows what resource optimization does for each type of HPA metric.

HPA metric What resource optimization does

Resource metric on cpu with a Utilization target

Leaves CPU alone for every container in the Pod. Keeps sizing memory.

Resource metric on memory with a Utilization target

Leaves memory alone for every container in the Pod, including out-of-memory recovery. Keeps sizing CPU.

ContainerResource metric with a Utilization target

Leaves that resource alone for the named container only.

Resource or ContainerResource metric with an AverageValue target

Keeps sizing the resource, but keeps the container’s limit above the HPA target so that the Pods can still reach it and the HPA can scale out.

External, Object, or Pods metric, such as queue depth or requests per second

Keeps sizing both CPU and memory.

KEDA scales workloads through an HPA that it creates for each ScaledObject, so the same rules apply to KEDA. A ScaledObject that scales on an external event source, such as queue length or consumer lag, leaves CPU and memory to resource optimization. A KEDA cpu or memory trigger with the Utilization metric type doesn’t.

When an HPA owns a resource, the ResourceOptimization shows the HPAResourceConflict condition as True, and status.resourceOwnership lists each resource, container, and HPA. The reason is PartialOwnership when resource optimization still manages some of the workload’s resources, and NoEligibleObjectives when the HPA owns all of them. The Ready condition stays True. Resource optimization still shows recommendations for the HPA-owned resource so that you can review them, but it never applies them.

Run the following kubectl command to view which resources an HPA owns, replacing web-deploy and shop with your values:

kubectl get ro web-deploy -n shop -o jsonpath='{.status.conditions[?(@.type=="HPAResourceConflict")].message}'

An example output is as follows.

cpu (all containers) is scaled by HorizontalPodAutoscaler/web

When you delete the HPA or change it to scale on something else, resource optimization waits for a short stabilization period before it manages the resource again. If KEDA deletes and recreates its HPA, the wait prevents a resize in between.

If resource optimization can’t read the HPAs for a workload, it doesn’t change that workload until it can.

Vertical Pod Autoscaler

Resource optimization detects Vertical Pod Autoscaler (VPA) objects automatically. You don’t need to exclude VPA-managed workloads yourself. How resource optimization responds depends on the VPA’s update mode:

  • Off : The VPA only makes recommendations, so resource optimization manages the workload as usual.

  • Initial : Resource optimization doesn’t apply usage-based resizes or startup boost to the workload. The ResourceOptimization shows the Ready condition as False with the reason VPAConflict. Because this VPA mode doesn’t change running Pods, resource optimization still raises memory after an out-of-memory kill.

  • Any other mode: Resource optimization doesn’t apply usage-based resizes or startup boost, and shows VPAConflict as for Initial. It also leaves out-of-memory kills to the VPA for the containers whose memory the VPA controls.

To stop resource optimization from tracking a VPA-managed workload at all, add the optimization.eks.amazonaws.com/enabled: "false" annotation to the workload, or leave the workload out of your policy’s selectors. For more information, see How resource optimization policies select and combine.

Out-of-memory recovery

When Kubernetes kills a container because it ran out of memory, the container status shows the reason OOMKilled, and resource optimization raises that container’s memory. In Apply mode, resource optimization applies the raise to the running Pods. In Recommend mode, it only records the raise as a recommendation.

  • Size of the raise: The first kill raises the memory request by 20%. A second kill within 24 hours raises it by 50%, and each further kill within 24 hours doubles it. Resource optimization counts each kill once.

  • Limits: When the container has a memory limit, resource optimization raises the limit by the same ratio, so the container keeps its request-to-limit ratio and its quality of service class. Resource optimization doesn’t add a limit to a container that doesn’t have one.

  • Caps: A raise never goes above the Maximum that the Balanced profile sets, which is twice the largest memory limit in the Pod template. It also never goes above what the largest node in your NodePools can fit. If any container in the Pod has a memory request but no limit, there’s no Maximum, and only the node cap applies. When memory reaches either cap, the ResourceOptimization shows the OOMAtMaximum condition with the reason MemoryCapReached. If a workload keeps running out of memory at the Maximum, raise its memory limit in the Pod template or reduce its memory use. If it’s at the node cap, allow larger instance types in a NodePool that the Pod can use, or raise that NodePool’s limits.

  • Node memory pressure: Sometimes the kernel kills a container that is below its own limit because the node ran low on memory. If that’s the only kill in 24 hours, resource optimization doesn’t raise the memory.

  • Your guardrails: If one of your own guardrails rejects the raise, the ResourceOptimization shows the OOMRaiseRejected condition with the reason RejectedByPolicy. A guardrail can be a ResourceQuota, a LimitRange, an admission webhook, or a ValidatingAdmissionPolicy. Rejections from webhooks and policies with names that end in .amazonaws.com or .k8s.aws, or that start with eks- or vpc-resource-, don’t set this condition. EKS and AWS use these names for the webhooks and policies that they install, so don’t use them for your own.

Resource optimization needs a memory request or limit on the container to raise it. It doesn’t raise memory that an HPA scales on, or for Pods in a sandbox RuntimeClass. For how raises reach running Pods, see How resource optimization applies changes to Pods.

Memory decreases

You can’t lower a container’s memory limit below the memory that the container is using. The kubelet refuses the resize. A recommendation applies to every Pod of a workload. A busy Pod can use more memory than the new limit. To avoid a resize that the kubelet would refuse, resource optimization doesn’t lower a Pod’s memory if the new limit would be less than 10% above that Pod’s recent peak usage. It still applies the CPU change, and it tries the memory decrease again on a later check. If the kubelet refuses a decrease anyway, resource optimization restores the memory that the Pod is running with.

When any Pod keeps its memory this way, the ResourceOptimization shows the MemoryDecreaseHeld condition as True, with the reason MemoryInUse or MemoryDecreaseRefused. A held Pod can run above the recommendation until its usage drops or it’s replaced.

Java workloads

A Java application reserves heap memory that its measured usage might not show. If resource optimization sized memory from usage alone, the JVM could fail after resource optimization lowered its memory. To prevent this, resource optimization keeps each Java container’s memory request at or above a floor based on the maximum heap. The floor is the larger of 1.25 times the maximum heap, or the maximum heap plus 256 MiB. The extra memory covers metaspace, code cache, and thread stacks. The floor applies to recommendations and to out-of-memory raises.

Resource optimization reads the maximum heap from the container’s spec, in the following order:

  1. The JAVA_TOOL_OPTIONS and JDK_JAVA_OPTIONS environment variables

  2. The options of a java command in the container’s command and args

  3. The _JAVA_OPTIONS environment variable

The last -Xmx or -XX:MaxHeapSize setting wins, and it takes precedence over -XX:MaxRAMPercentage.

The following example Deployment manifest sets a 1 GiB maximum heap, so resource optimization never sets the memory request below 1.25 GiB. Replace public.ecr.aws/amazoncorretto/amazoncorretto:21 with your application’s image, and /app/orders.jar with the path to your application’s JAR file:

apiVersion: apps/v1 kind: Deployment metadata: name: orders namespace: shop spec: replicas: 2 selector: matchLabels: app: orders template: metadata: labels: app: orders spec: containers: - name: orders image: public.ecr.aws/amazoncorretto/amazoncorretto:21 command: - java - -Xmx1g - -jar - /app/orders.jar resources: requests: cpu: "1" memory: 2Gi limits: memory: 2Gi

In this preview, heap floors have the following limits:

  • Java only: Resource optimization doesn’t detect heap settings for other runtimes, such as Node.js, Go, or .NET.

  • Literal values only: Resource optimization reads only the values that are written in the Pod template. It can’t see a heap set in any of the following ways:

    • A ConfigMap or Secret (valueFrom or envFrom)

    • A $(VAR) reference

    • An @argfile or a -XX:VMOptionsFile

    • A java command that a shell script starts

      In those cases, resource optimization falls back to any -Xmx that it can see in the container’s environment variables, command, or arguments. This includes JAVA_OPTS, which the JVM doesn’t read but many startup scripts pass to java. If it finds none, it sizes the container from usage only, and the ResourceOptimization shows the HeapUndetected condition with the reason NoUnambiguousMaximumHeap. To get a floor, set -Xmx directly in the container’s environment variables or arguments.

  • Memory limit for percentage heaps: -XX:MaxRAMPercentage sets the heap as a share of the container’s memory limit. Without a limit, resource optimization can’t know the heap size and applies no floor. With a limit, resource optimization keeps the limit unchanged so that the heap size stays the same.

  • Floors above a bound: If the floor is above the Maximum (twice the largest memory limit in the Pod template), the container’s memory limit, or what the largest node can fit, the ResourceOptimization shows the HeapFloorInfeasible condition. The reason names the bound: MaximumBelowFloor, MemoryLimitBelowFloor, NodeCapacityBelowFloor, or MemoryLimitAboveMaximum. To fix it, lower the heap or raise the bound.

How resource settings affect sizing

How you set requests and limits in your Pod template changes what resource optimization can do. The following table shows how resource optimization handles each combination of request and limit settings for a resource.

Container settings What resource optimization does

Request and limit

Sizes the request, and the limit by the same ratio.

Request equal to limit (Guaranteed)

Keeps the request and limit equal, so the Pod stays Guaranteed.

Limit only

Kubernetes sets the request equal to the limit when it creates the Pod. Resource optimization sizes the container the same way and keeps the request and limit equal. Your template keeps only the limit, and running Pods show both.

Request only

Sizes the request and doesn’t add a limit. The request can grow up to what the largest node in your NodePools can fit.

Neither

Recommends a size from usage but never applies it, in either mode, because a resize can’t change a Pod’s quality of service class. The recommendation’s reason field says that it’s a recommendation only. A container with no memory request or limit also gets no out-of-memory raise. This includes BestEffort containers, which have no requests at all. In Apply mode, if no container in the workload declares CPU or memory, the ResourceOptimization shows the AdmissionSizing condition as False with the reason RecommendOnly. To have resource optimization apply its recommendations, set CPU and memory requests.

The following workload shapes also affect sizing:

  • Multiple containers: Resource optimization recommends a size for each container from that container’s own usage. The Balanced profile’s floor and ceiling are shared across the Pod, from half the smallest container request to twice the largest container limit. If any container has a request but no limit for a resource, the resource has no ceiling.

  • Native sidecars: Resource optimization sizes init containers with restartPolicy: Always, such as service mesh proxies and log shippers, the same way as regular containers, including out-of-memory recovery.

  • Init containers: Init containers that run once and exit can’t be resized, so resource optimization leaves them alone.

  • Pod-level resources: If the Pod template sets spec.resources, resource optimization sizes the Pod as a whole from the Pod’s total usage. An HPA that scales on a resource owns that resource for the whole Pod.

For how resource optimization chooses which workloads to manage, see How resource optimization policies select and combine. For how it applies changes to running and new Pods, see How resource optimization applies changes to Pods. For conditions and what to do about them, see Troubleshoot EKS Auto Mode resource optimization.