Tutorial: Using cluster auto scaling with the AWS Management Console and the classic Amazon ECS console

This tutorial walks you through creating the resources for cluster auto scaling using the AWS Management Console and the classic Amazon ECS console. Where resources require a name, we will use the prefix ConsoleTutorial to ensure they all have unique names and to make them easy to locate.

Prerequisites

This tutorial assumes that the following prerequisites have been completed:

• The steps in Set up to use Amazon ECS have been completed.

• Your AWS user has the required permissions specified in the Amazon ECS first-run wizard permissions IAM policy example.

• The Amazon ECS container instance IAM role is created. For more information, see Amazon ECS container instance IAM role.

• The Amazon ECS service-linked IAM role is created. For more information, see Using service-linked roles for Amazon ECS.

• The Auto Scaling service-linked IAM role is created. For more information, see Service-Linked Roles for Amazon EC2 Auto Scaling in the Amazon EC2 Auto Scaling User Guide.

• You have a VPC and security group created to use. For more information, see Create a virtual private cloud.

Step 1: Create an Amazon ECS cluster

Use the following steps to create an Amazon ECS cluster. This tutorial uses an empty cluster so that we can manually create the Auto Scaling resources. When you use the AWS Management Console to create a non-empty cluster, Amazon ECS creates an AWS CloudFormation stack along with Auto Scaling resources. We want to avoid creating this AWS CloudFormation stack when using the cluster auto scaling feature.

To create an empty cluster

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Clusters.

4. On the Clusters page, choose Create Cluster.

5. For Select cluster compatibility, choose EC2 Linux + Networking and then choose Next step.

6. For Cluster name, enter ConsoleTutorial-cluster for the cluster name.

7. Select Create an empty cluster and then choose Create.

Step 2: Create the Auto Scaling resources

Use the following steps to create an Amazon EC2 launch template and Auto Scaling group.

To create an Amazon EC2 launch template

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. On the navigation pane, under Instances, choose Launch Templates.

4. On the next page, choose Create launch template.

5. On the Create launch template page, complete the following steps.

   1. For Launch template name, enter ConsoleTutorial-LaunchTemplate.

   2. For Template version description, provide a description for the launch template.

   3. For Amazon Machine Image (AMI), search for the latest Amazon ECS-optimized AMI. The AMI ID can be retrieved using the following link: View AMI ID. For more information, see Retrieving Amazon ECS-Optimized AMI metadata.

   4. For Instance type, select an instance type. For the purposes of this tutorial, the t2.micro instance type works.

   5. For Security groups, select one or more security groups.

   6. Expand the Advanced Details section to specify the IAM instance profile and user data for your Amazon ECS container instances.

      1. For IAM instance profile, select your container instance IAM role. For more information, see Amazon ECS container instance IAM role.

      2. For User data, paste the following script into the field. The ConsoleTutorial-cluster cluster was created in the first step.

         #!/bin/bash
         echo ECS_CLUSTER=ConsoleTutorial-cluster >> /etc/ecs/ecs.config

6. Choose Create launch template.

Next, create an Auto Scaling group using that launch template.

To create an Auto Scaling group

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. On the navigation pane, under Auto Scaling, choose Auto Scaling Groups, Create Auto Scaling group.

4. On the Create Auto Scaling group page, complete the following steps.

   1. For Auto Scaling group name, enter ConsoleTutorial-ASG for the Auto Scaling group name.

   2. For Launch template, select the ConsoleTutorial-LaunchTemplate launch template.

   3. Review the contents of the launch template, then choose Next.

   4. On the Configure settings step, under Network, select a VPC and one or more Subnets to use and then choose Next.

   5. On the Configure advanced options step, choose Next.

   6. On the Configure group size and scaling policies step, for Desired capacity, enter 0. This tutorial uses Amazon ECS managed scaling so there is no need to have the Auto Scaling group launch any initial instances. For Minimum capacity, enter 0. For Maximum capacity, enter 2.

   7. For Instance scale-in protection, select Enable instance scale-in protection and then choose Next. This enables you to use managed termination protection for the instances in the Auto Scaling group, which prevents your container instances that contain tasks from being terminated during scale-in actions.

   8. On the Add notifications and Add tags steps, choose Next.

   9. On the Review step, review the Auto Scaling group settings and then choose Create Auto Scaling group.

5. Repeat steps 3 and 4 to create a second Auto Scaling group. For Auto Scaling group name use ConsoleTutorial-ASG-burst and when setting the group size, set the Desired capacity and Minimum capacity to 0 and the Maximum capacity to 20.

Step 3: Create a capacity provider

Use the following steps to create an Amazon ECS capacity provider. See for more information.

To create a capacity provider

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Clusters.

4. On the Clusters page, select your ConsoleTutorial-cluster cluster.

5. On the Capacity Providers tab, choose Create.

6. On the Create Capacity Provider page, complete the following steps.

   1. For Capacity provider name, enter ConsoleTutorial-capacityprovider for the name.

   2. For Auto Scaling group, select the ConsoleTutorial-ASG Auto Scaling group you created.

   3. For Managed scaling, choose Enabled. This enables Amazon ECS to manage the scale-in and scale-out actions for the capacity provider.

   4. For Target capacity %, enter 100.

   5. For Managed termination protection, choose Enabled. This prevents your container instances that contain tasks and that are in the Auto Scaling group from being terminated during a scale-in action.

   6. Choose Create.

   7. Choose View in cluster to see your new capacity provider.

   8. Repeat steps 4 to 6, creating a second capacity provider with name ConsoleTutorial-capacityprovider-burst with your ConsoleTutorial-ASG-burst Auto Scaling group.

Step 4: Set a default capacity provider strategy for the cluster

When running a task or creating a service, the Amazon ECS console uses the default capacity provider strategy for the cluster. The default capacity provider strategy can be defined by updating the cluster.

To define a default capacity provider strategy

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Clusters.

4. On the Clusters page, select your ConsoleTutorial-cluster cluster.

5. On the Cluster : ConsoleTutorial-cluster page, choose Update Cluster.

6. For Default capacity provider strategy choose, Add provider.

7. Select your ConsoleTutorial-capacityprovider capacity provider.

8. Choose Add provider, select your ConsoleTutorial-capacityprovider-burst capacity provider.

9. For Provider 1, leave the Base value at 0 and leave the Weight value at 1.

10. Choose Update. This will add the capacity providers to the default capacity provider strategy for the cluster.

11. Choose View cluster.

Step 5: Register a task definition

Before you can run a task on your cluster, you must register a task definition. Task definitions are lists of containers grouped together. The following example is a simple task definition that uses an amazonlinux image from Docker Hub and simply sleeps. For more information about the available task definition parameters, see Amazon ECS task definitions.

To register a task definition

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Task Definitions, Create new Task Definition.

4. On the Create new Task Definition page, select EC2, Next step.

5. Choose Configure via JSON and copy and paste the following contents and then choose Save, Create.

   {
       "family": "ConsoleTutorial-taskdef",
       "containerDefinitions": [
           {
               "name": "sleep",
               "image": "amazonlinux:2",
               "memory": 20,
               "essential": true,
               "command": [
                   "sh",
                   "-c",
                   "sleep infinity"
               ]
           }
       ],
       "requiresCompatibilities": [
           "EC2"
       ]
   }

Step 6: Run a task

After you have registered a task definition for your account, you can run a task in the cluster. For this tutorial, you run five instances of the ConsoleTutorial-taskdef task definition in your ConsoleTutorial-cluster cluster.

To run a task

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Task Definitions.

4. Select your ConsoleTutorial-taskdef task definition.

5. From the Actions menu, choose Run Task.

6. Use the following steps to complete the run task workflow.

   1. For Capacity provider strategy, the default capacity provider strategy for the cluster must be selected.

   2. For Cluster, select your ConsoleTutorial-cluster cluster.

   3. For Number of tasks, enter 5.

   4. For Placement Templates, choose BinPack.

   5. Choose Run Task.

Step 7: Verify

At this point in the tutorial, you should have two Auto Scaling groups with one capacity provider for each of them. The capacity providers have Amazon ECS managed scaling enabled. A cluster was created and five tasks are running.

We can verify that everything is working properly by viewing the CloudWatch metrics, the Auto Scaling group settings, and finally the Amazon ECS cluster task count.

To view the CloudWatch metrics for your cluster

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. On the navigation pane, choose Metrics.

4. On the All metrics tab, choose AWS/ECS/ManagedScaling.

5. Choose CapacityProviderName, ClusterName.

6. Choose the metric that corresponds to the ConsoleTutorial-capacityprovider capacity provider.

7. On the Graphed metrics tab, change Period to 30 seconds and Statistic to Maximum.

   The value displayed in the graph shows the target capacity value for the capacity provider. It should begin at 100, which was the target capacity percent we set. You should see it scale up to 200, which will trigger an alarm for the target tracking scaling policy. The alarm will then trigger the Auto Scaling group to scale out.

   

8. Steps 5 to 6 can be repeated for your ConsoleTutorial-capacityprovider-burst metric.

Use the following steps to view your Auto Scaling group details to confirm that the scale-out action occurred.

To verify the Auto Scaling group scaled out

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. On the navigation pane, under Auto Scaling, choose Auto Scaling Groups.

4. For each of your Auto Scaling groups, view the values in the Instances and Desired columns to confirm your group scaled out to two instances for each group.

Use the following steps to view your Amazon ECS cluster to confirm that the Amazon EC2 instances were registered with the cluster and your tasks transitioned to a RUNNING status.

To verify the instances in the Auto Scaling group

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Clusters.

4. On the Clusters page, select your ConsoleTutorial-cluster cluster.

5. On the ECS Instances tab, confirm you see four instances registered, which matches your Auto Scaling group values.

6. On the Tasks tab, confirm you see five tasks in RUNNING status.

Step 8: Clean up

When you have finished this tutorial, clean up the resources associated with it to avoid incurring charges for resources that you aren't using. Deleting capacity providers and task definitions are not supported, but there is no cost associated with these resources.

To clean up the tutorial resources

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

3. In the navigation pane, choose Clusters.

4. On the Clusters page, select your ConsoleTutorial-cluster cluster.

5. From the Tasks tab, choose Stop All. Enter the verification and choose Stop all again.

6. Delete the Auto Scaling groups using the following steps.

   1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

   2. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

   3. On the navigation pane, under Auto Scaling, choose Auto Scaling Groups.

   4. Select your ConsoleTutorial-ASG Auto Scaling group, then from the Actions menu choose Delete.

   5. Select your ConsoleTutorial-ASG-burst Auto Scaling group, then from the Actions menu choose Delete.

7. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/.

8. On the navigation bar at the top of the screen, select the US West (Oregon) Region.

9. In the navigation pane, choose Clusters.

10. On the Clusters page, select your ConsoleTutorial-cluster cluster.

11. Choose Delete Cluster, enter the confirmation phrase, and choose Delete.