Common Use Cases in Amazon ECS
This topic provides guidance for two common use cases in Amazon ECS: microservices and batch jobs. Here you can find considerations and external resources that may be useful for getting your application running on Amazon ECS, and the common aspects of each solution.
Microservices are built with a software architectural method that decomposes complex applications into smaller, independent services. Containers are optimal for running small, decoupled services, and they offer the following advantages:
Containers make services easy to model in an immutable image with all of your dependencies.
Containers can use any application and any programming language.
The container image is a versioned artifact, so you can track your container images to the source they came from.
You can test your containers locally, and deploy the same artifact to scale.
The following sections cover some of the aspects and challenges that you must consider when designing a microservices architecture to run on Amazon ECS. You can also view the microservices reference architecture on GitHub. For more information, see Deploying Microservices with Amazon ECS, AWS CloudFormation, and an Application Load Balancer.
The application load for your microservice architecture can change over time. A responsive application can scale out or in, depending on actual or anticipated load. Amazon ECS provides you with several tools to scale not only your services that are running in your clusters, but the actual clusters themselves.
For example, Amazon ECS provides CloudWatch metrics for your clusters and services. For more information, see Amazon ECS CloudWatch Metrics. You can monitor the memory and CPU utilization for your clusters and services. Then, use those metrics to trigger CloudWatch alarms that can automatically scale out your cluster when its resources are running low, and scale them back in when you don't need as many resources. For more information, see Tutorial: Scaling Container Instances with CloudWatch Alarms.
In addition to scaling your cluster size, your Amazon ECS service can optionally be configured to use Service Auto Scaling to adjust its desired count up or down in response to CloudWatch alarms. Service Auto Scaling is available in all regions that support Amazon ECS. For more information, see Service Auto Scaling.
Service discovery is a key component of most distributed systems and service-oriented architectures. With service discovery, your microservice components are automatically discovered as they get created and terminated on a given infrastructure. There are several approaches that you can use to make your services discoverable. The following resources describe a few examples:
Run Containerized Microservices with Amazon EC2 Container Service and Application Load Balancer: This post describes how to use the dynamic port mapping and path-based routing features of Elastic Load Balancing Application Load Balancers to provide service discovery for a microservice architecture.
Amazon EC2 Container Service - Reference Architecture: Service Discovery: This Amazon ECS reference architecture provides service discovery to containers using CloudWatch Events, Lambda, and Amazon Route 53 private hosted zones.
Service Discovery via Consul with Amazon ECS: This post shows how a third party tool called Consul by HashiCorp can augment the capabilities of Amazon ECS by providing service discovery for an ECS cluster (complete with an example application).
Authorization and Secrets Management
Managing secrets, such as database credentials for an application, has always been a challenging issue. The Managing Secrets for Amazon ECS Applications Using Parameter Store and IAM Roles for Tasks post focuses on how to integrate the IAM roles for tasks functionality of Amazon ECS with the Amazon EC2 Systems Manager parameter store. Parameter store provides a centralized store to manage your configuration data, whether it's plaintext data such as database strings or secrets such as passwords, encrypted through AWS Key Management Service.
You can configure your container instances to send log information to CloudWatch Logs. This enables you to view different logs from your container instances in one convenient location. For more information about getting started using CloudWatch Logs on your container instances that were launched with the Amazon ECS-optimized AMI, see Using CloudWatch Logs with Container Instances.
You can configure the containers in your tasks to send log information to CloudWatch Logs.
This enables you to view different logs from your containers in one convenient
location, and it prevents your container logs from taking up disk space on your
container instances. For more information about getting started using the
awslogs log driver in your task definitions, see Using the awslogs Log Driver.
Continuous Integration and Continuous Deployment
Continuous integration and continuous deployment (CICD) is a common process for microservice architectures that are based on Docker containers. You can create a pipeline that takes the following actions:
Monitors changes to a source code repository
Builds a new Docker image from that source
Pushes the image to an image repository such as Amazon ECR or Docker Hub
Updates your Amazon ECS services to use the new image in your application
ECS Reference Architecture: Continuous Deployment: This reference architecture demonstrates how to achieve continuous deployment of an application to Amazon ECS using AWS CodePipeline, AWS CodeBuild, and AWS CloudFormation.
Continuous Delivery Pipeline for Amazon ECS Using Jenkins, GitHub, and Amazon ECR: This AWS labs repository helps you set up and configure a continuous delivery pipeline for Amazon ECS using Jenkins, GitHub, and Amazon ECR.
Docker containers are particularly suited for batch job workloads. Batch jobs are often short-lived and embarrassingly parallel. You can package your batch processing application into a Docker image so that you can deploy it anywhere, such as in an Amazon ECS task. If you are interested in running batch job workloads, consider the following resources:
AWS Batch: For fully managed batch processing at any scale, you should consider using AWS Batch. AWS Batch enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. AWS Batch dynamically provisions the optimal quantity and type of compute resources (for example, CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. For more information, see the AWS Batch product detail pages.
Amazon ECS Reference Architecture: Batch Processing: This reference architecture illustrates how to use AWS CloudFormation, Amazon S3, Amazon SQS, and CloudWatch alarms to handle batch processing on Amazon ECS.