Set the default instance warmup for an Auto Scaling group
CloudWatch collects and aggregates usage data, such as CPU and network I/O, across your Auto Scaling instances. You use these metrics to create scaling policies that adjust the number of instances in your Auto Scaling group as the selected metric's value increases and decreases.
The default instance warmup lets you specify how long after an instance
reaches the InService
state it waits before contributing usage data
to the aggregated metrics. This keeps dynamic scaling from being affected by
metrics for individual instances that aren't handling application traffic yet
and that might be experiencing temporarily high usage of compute resources.
Default instance warmup is not configured and is not enabled by default. To optimize the performance of your target tracking and step scaling policies, we strongly recommend that you enable the default instance warmup.
In addition to affecting scaling performance, the default instance warmup influences the overall time required to replace an instance with features such as instance refresh and maximum instance lifetime.
Contents
Scaling performance considerations
Most applications will benefit from having one default instance warm-up time that applies to all features rather than different warm-up times for different features. For example, if you don't set a default instance warmup, the instance refresh feature uses the health check grace period as the default warm-up time. If you have any target tracking and step scaling policies, they use the value set for the default cooldown as the default warm-up time. If you have any predictive scaling policies, they have no default warm-up time.
While instances are warming up, your dynamic scaling policies only scale out if the metric value from instances that are not warming up is greater than the policy's alarm high threshold (or the target utilization of a target tracking scaling policy). If demand decreases, the intention is to scale in conservatively to protect your application's availability. This blocks scale in for dynamic scaling until the instances finish warming up.
While scaling out, we consider instances that are warming up as part of the theoretical capacity of the group when deciding how many instances to add to the group. Therefore, multiple alarm breaches that fall in the range of the same step adjustment result in a single scaling activity. The intention is to continuously (but not excessively) scale out. For more information, see the Instance warm-up section in the step scaling topic.
If default instance warmup is not enabled, the amount of time an instance waits before sending metrics to CloudWatch and counting it towards the current capacity will vary from instance to instance. In which case, there is the potential for your scaling policies to perform unpredictably compared to the actual workload that is occurring.
For example, consider an application with a recurring on-and-off workload pattern. A predictive scaling policy is used to make recurring decisions about whether to increase the number of instances. Because there is no default warm-up time for predictive scaling policies, the instances start contributing to the aggregated metrics immediately. If these instances have higher resource usage on startup, then adding instances could cause the aggregated metrics to spike. Depending on how long it takes for usage to stabilize, this could impact any dynamic scaling policies that use these metrics. If a dynamic scaling policy's alarm high threshold is breached, then the group increases in size again. While the new instances are warming up, scale-in activities will be blocked.
Choose the default instance warm-up time
The key to setting the default instance warmup is determining how long
your instances need to finish initializing and for resource consumption to
stabilize after they reach the InService
state. When choosing
the instance warm-up time, aim for an optimal balance between collecting
usage data for legitimate traffic, but minimizing data collection associated
with temporary usage spikes on startup.
Suppose you have an Auto Scaling group attached to an Elastic Load Balancing load balancer. When
new instances finish launching, they're registered to the load balancer
before they enter the InService
state. After the instances
enter the InService
state, resource consumption can still
experience temporary spikes and need time to stabilize. For example,
resource consumption for an application server that must download and cache
large assets takes longer to stabilize than a lightweight web server with no
large assets to download. The instance warmup provides the time delay
necessary for resource consumption to stabilize.
Important
If you're not sure how much time you need, you could start with 300
seconds and gradually decrease or increase it until you get the best
scaling performance for your application. You might need to experiment
to get it right. Alternatively, if you have any scaling policies that
have their own warm-up time (EstimatedInstanceWarmup
), you
could use this value to start. For more information, see Find
scaling policies with a previously set instance warm-up time.
Depending on whether your group uses any lifecycle hooks, you might be able to use a shorter warm-up time.
If | Then |
---|---|
Your Auto Scaling group has a lifecycle hook that delays instances from being put in service until they have finished initializing. | You can set the default instance warmup to a shorter duration. |
Your Auto Scaling group does not have a lifecycle hook that initializes instances before they're put in service. | You can specify a longer duration for the default instance warmup that includes the time your instances need to finish initializing. |
Enable the default instance warmup for a group
You can enable the default instance warmup when you create an Auto Scaling group. You can also enable it for existing groups.
By enabling the default instance warmup feature, you no longer have to specify values for warm-up parameters for the following features:
Verify the default instance warmup for a group
To verify the default instance warmup for an Auto Scaling group (AWS CLI)
Use the describe-auto-scaling-groups command.
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name
my-asg
The following is an example response.
{
"AutoScalingGroups": [
{
"AutoScalingGroupName": "my-asg",
"AutoScalingGroupARN": "arn",
...
"DefaultInstanceWarmup": 120
}
]
}
Find scaling policies with a previously set instance warm-up time
To identify whether you have policies that have their own warm-up time for
EstimatedInstanceWarmup
, run the following describe-policies command using the AWS CLI. Replace
my-asg
with the name of your Auto Scaling
group.
aws autoscaling describe-policies --auto-scaling-group-name
my-asg
--query 'ScalingPolicies[?EstimatedInstanceWarmup!=`null`]'
The following is example output.
[
{
"AutoScalingGroupName":"my-asg",
"PolicyName":"cpu40-target-tracking-scaling-policy",
"PolicyARN":"arn
",
"PolicyType":"TargetTrackingScaling",
"StepAdjustments":[],
"EstimatedInstanceWarmup":120,
"Alarms":[{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e",
"AlarmName": "TargetTracking-my-asg-AlarmHigh-fc0e4183-23ac-497e-9992-691c9980c38e"
},
{
"AlarmARN": "arn:aws:cloudwatch:us-west-2:123456789012:alarm:TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2",
"AlarmName": "TargetTracking-my-asg-AlarmLow-61a39305-ed0c-47af-bd9e-471a352ee1a2"
}],
"TargetTrackingConfiguration":{
"PredefinedMetricSpecification":{
"PredefinedMetricType":"ASGAverageCPUUtilization"
},
"TargetValue":40.0,
"DisableScaleIn":false
},
"Enabled":true
},
... additional policies ...
]
Clear the previously set instance warmup for a scaling policy
After enabling the default instance warmup, update any scaling policies that have still their own warm-up time to clear the previously set value. Otherwise, it will override the default instance warmup.
You can update scaling policies using the console, AWS CLI, or AWS SDKs.
This section covers the steps for the console. If you use the AWS CLI or AWS
SDKs, make sure you preserve the existing policy configuration, but remove
the EstimatedInstanceWarmup
property. When you update an
existing scaling policy, the policy will be replaced with what you specify
when you programmatically call PutScalingPolicy. The original values are not kept.
To clear the previously set instance warmup for a scaling policy (console)
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/
, and choose Auto Scaling Groups from the navigation pane. -
Select the check box next to the Auto Scaling group.
A split pane opens up in the bottom of the page.
-
On the Automatic scaling tab, in Dynamic scaling policies, choose the policy you're interested in, and then choose Actions, Edit.
-
For Instances need, clear the instance warm-up value to use the default instance warm-up value instead.
-
Choose Update.