Appendix J: AWS Fargate (Amazon ECS) implementation - Ops Automator

Appendix J: AWS Fargate (Amazon ECS) implementation

Ops Automator is a developer framework that runs AWS Lambda tasks by default. In cases where parts of tasks take longer than the 15 minute time limit for Lambda functions, the framework can be configured to run these steps as Amazon Elastic Container Service (Amazon ECS) tasks using an AWS Fargate cluster. Fargate is a serverless compute engine that removes the need to provision and manage servers.

This appendix describes how to set up an Fargate cluster that is used by the Ops Automator framework for long-running tasks. Beginning with Ops Automator v2.2.0 you can either deploy with Fargate, or add it later. The following procedure assumes you deployed the Ops Automator using the AWS Management Console and an AWS CloudFormation template.

Note

If you previously deployed Ops Automator from the GitHub repository, refer to the README file to update Ops Automator in your environment.

Setup

Overview

Use the following procedure to deploy or update Ops Automator using the AWS Fargate on Amazon ECS option.

Deploy Ops Automator with AWS Fargate

You will deploy this solution’s template following the procedure in Automated deployment. When you review the parameters, choose the ECS/Fargate option by changing the default value to True. Refer to Step 1 for details.

You can choose to run Fargate in an existing Amazon Virtual Private Cloud (Amazon VPC), or create a new one.

  • To use an existing Amazon VPC, you must provide the VPC ID and subnet IDs for at least two subnets. The Fargate cluster deploys in the Amazon VPC and subnets that you specify. Note that public subnets are required.

  • If you do not provide a VPC ID, the solution creates a new VPC and public subnets for the Fargate cluster

Set up the build environment

The following resources are required to set up the build environment:

  • A workstation with Docker, Python with Boto3, and the AWS Command Line Interface (AWS CLI) installed.

  • An AWS Identity and Access Management (IAM) role that allows AWS CloudFormation describe-stacks, sts get-caller-identity, and ecr get-login parameters.

Download and install the following Amazon ECS image tools and files:

Build and deploy the Docker container

In your terminal, navigate to the folder where you downloaded the Amazon ECS files and run the following command.

./build-and-deploy-image.sh -s <stack-name> -r <region>

Replace <stack-name> with the name you entered for your Ops Automator CloudFormation stack. Replace <region> with the Region where your stack is currently running.

This script pulls down the required files to build a Docker image based on the AWS Linux Docker optimized Amazon Machine Images (AMI). It installs the ops-automator-ecs-runner.py script on the image, then uploads the image to the ops-automator repository created by the Ops Automator stack.

Deploy Ops Automator actions using Amazon ECS

The Amazon ECS option is now available for Actions. Continue with Step 2 in Automated deployment.