Access container applications privately on Amazon ECS by using AWS Fargate, AWS PrivateLink, and a Network Load Balancer

Created by Kirankumar Chandrashekar (AWS)

Environment: Production	Technologies: Containers & microservices; Networking; Security, identity, compliance; Web & mobile apps	Workload: All other workloads
AWS services: Amazon EC2 Container Registry; Amazon ECS; Amazon EFS; Amazon RDS; Amazon VPC; Elastic Load Balancing (ELB); AWS Lambda

Summary

This pattern describes how to privately host a Docker container application on the Amazon Web Services (AWS) Cloud by using Amazon Elastic Container Service (Amazon ECS) with an AWS Fargate launch type, behind a Network Load Balancer, and access the application by using AWS PrivateLink. Amazon Relational Database Service (Amazon RDS) hosts the relational database for the application running on Amazon ECS with high availability (HA). You can use Amazon Elastic File System (Amazon EFS) if the application requires persistent storage.

This pattern uses a Fargate launch type for the Amazon ECS service running the Docker applications, with a Network Load Balancer at the front end. It can then be associated with a virtual private cloud (VPC) endpoint for access through AWS PrivateLink. This VPC endpoint service can then be shared with other VPCs by using their VPC endpoints.

You can use Fargate with Amazon ECS to run containers without having to manage servers or clusters of Amazon Elastic Compute Cloud (Amazon EC2) instances. You can also use an Amazon EC2 Auto Scaling group instead of Fargate. For more information, see Access container applications privately on Amazon ECS by using AWS PrivateLink and a Network Load Balancer.

Prerequisites and limitations

Prerequisites

• An active AWS account

• AWS Command Line Interface (AWS CLI) version 2, installed and configured on Linux, macOS, or Windows

• Docker, installed and configured on Linux, macOS, or Windows

• An application running on Docker

Architecture

Technology stack

• Amazon CloudWatch

• Amazon Elastic Container Registry (Amazon ECR)

• Amazon ECS

• Amazon EFS

• Amazon RDS

• Amazon Simple Storage Service (Amazon S3)

• AWS Fargate

• AWS Lambda

• AWS PrivateLink

• AWS Secrets Manager

• Application Load Balancer

• Network Load Balancer

• VPC

Automation and scale

• You can use AWS CloudFormation to create this pattern by using Infrastructure as Code.

Tools

• Amazon ECS – Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage containers on a cluster.

• Amazon ECR – Amazon Elastic Container Registry (Amazon ECR) is a managed AWS container image registry service that is secure, scalable, and reliable.

• Amazon EFS – Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources.

• AWS Fargate – AWS Fargate is a technology that you can use with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances.

• AWS Lambda – Lambda is a compute service that lets you run code without provisioning or managing servers.

• Amazon RDS – Amazon Relational Database Service (Amazon RDS) is a web service that makes it easier to set up, operate, and scale a relational database in the AWS Cloud.

• Amazon S3 – Amazon Simple Storage Service (Amazon S3) is storage for the internet. It is designed to make web-scale computing easier for developers.

• AWS Secrets Manager – Secrets Manager helps you replace hardcoded credentials in your code, including passwords, with an API call to Secrets Manager to retrieve the secret programmatically.

• Amazon VPC – Amazon Virtual Private Cloud (Amazon VPC) helps you launch AWS resources into a virtual network that you've defined.

• Elastic Load Balancing – Elastic Load Balancing (ELB) distributes incoming application or network traffic across multiple targets, such as EC2 instances, containers, and IP addresses, in multiple Availability Zones.

• Docker– Docker helps developers to easily pack, ship, and run any application as a lightweight, portable, and self-sufficient container.

Epics

Create networking components

Task	Description	Skills required
Create a VPC.	Sign in to the AWS Management Console, and open the Amazon VPC console. Choose Create VPC, and choose VPC and more.  Enter a name for your VPC, and choose an appropriate CIDR block range. Specify two Availability Zones, two public subnets, four private subnets. Two private subnets are for Amazon ECS tasks, and two private subnets are for Amazon RDS databases. Specify one NAT gateway for each Availability Zone. Choose Create VPC.	Cloud administrator

Create the load balancers

Task	Description	Skills required
Create a Network Load Balancer.	Open the Amazon EC2 console, and choose the AWS Region that contains your VPC.  Under Load balancing, choose Load balancers, and choose Create load balancer.  Choose Network Load Balancer, and choose Create.  On the Configure load balancer page, configure your Network Load Balancer and listener. Important: Make sure you choose your Network Load Balancer's scheme as Internal.  Choose the applicable security settings, configure a security group and a target group. Choose IP as the Target type in the Configure routing section. Make sure you do not register a target.  When you have configured all the settings, choose Next: Review, and then choose Create. For help with this and other stories, see the Related resources section.	Cloud administrator
Create an Application Load Balancer.	On the Amazon EC2 console, choose the same Region that contains your VPC.  Under Load balancing, choose Load balancers, and choose Create load balancer.  Choose Application Load Balancer, and choose Create.  Configure your Application Load Balancer and its listener. Important: Make sure you choose your Application Load Balancer's scheme as Internal.  Choose the applicable security settings, configure a security group and a target group. Choose IP as the Target type in the Configure routing section. Make sure you do not register a target.  When you have configured all the settings, choose Next: Review, and then choose Create.	Cloud administrator

Create an Amazon EFS file system

Task	Description	Skills required
Create an Amazon EFS file system.	Open the Amazon EFS console, and choose Create file system.  In the Create file system dialog box, enter a name for your file system, and choose your VPC.  Choose Create to create the file system.  Set up and configure your Amazon EFS file system.	Cloud administrator
Mount targets for the subnets.	Return to the Amazon EFS console, and choose File systems. The File systems page shows the Amazon EFS file systems in your account.  Choose the file system that you created, and choose Manage to display the Availability Zone.  To add a mount target, choose Add mount target, and add the four private subnets that you created.	Cloud administrator
Verify that the subnets are mounted as targets.	On the Amazon EFS console, choose File systems.  Choose Network to display the list of existing mount targets. Make sure that these include the four subnets that you created.	Cloud administrator

Create an S3 bucket

Task	Description	Skills required
Create an S3 bucket.	Open the Amazon S3 console and create an S3 bucket to store your application’s static assets, if required.	Cloud administrator

Create a Secrets Manager secret

Task	Description	Skills required
Create an AWS KMS key to encrypt the Secrets Manager secret.	Open the AWS Key Management Service (AWS KMS) console and create a KMS key.	Cloud administrator
Create a Secrets Manager secret to store the Amazon RDS password.	Open the AWS Secrets Manager console, and create a new secret by choosing Store a new secret.  Choose the KMS key that you created, and store your new secret.	Cloud administrator

Create an Amazon RDS instance

Task	Description	Skills required
Create a DB subnet group.	Open the Amazon RDS console, and choose Subnet groups.  Choose Create DB subnet group, and enter a name and description for your DB subnet group.  Choose the VPC that you created earlier, and choose the Availability Zones and subnets. Then choose Create.	Cloud administrator
Create an Amazon RDS instance.	Create and configure an Amazon RDS instance within the private subnets. Make sure that Multi-AZ is turned on for high availability (HA).	Cloud administrator
Load data to the Amazon RDS instance.	Load the relational data required by your application into your Amazon RDS instance. This process will vary depending on your application's needs, as well as how your database schema is defined and designed.	DBA

Create the Amazon ECS components

Task	Description	Skills required
Create an ECS cluster.	Open the Amazon ECS console, and choose Clusters.  Choose Create clusters, and set up an ECS cluster according to your required specifications.	Cloud administrator
Create the Docker images.	Create the Docker images by following the instructions in the Related resources section.	Cloud administrator
Create an Amazon ECR repository.	Open the Amazon ECR console, and choose Repositories.  Choose Create repository, and enter a unique name for your repository.  Configure the repository according to your specifications, including AWS KMS encryption if required.	Cloud administrator, DevOps engineer
Push the Docker images to the Amazon ECR repository.	Identify the Docker image you want to push, and run the docker images command in AWS CLI.  Tag your image with the Amazon ECR registry, repository, and optional image tag name combination.  Push the Docker image by running the docker push command.  Repeat these steps for all required images.	Cloud administrator
Create an Amazon ECS task definition.	A task definition is required to run Docker containers in Amazon ECS.  Return to the Amazon ECS console, choose Task definitions, and then choose Create new task definition.  On the Select compatibilities page, select the launch type that your task should use, and choose Next step. For help with setting up your task definition, see “Creating a task definition” in the Related resources section. Important: Make sure you provide the Docker images that you pushed to Amazon ECR.	Cloud administrator
Create an ECS service and choose Fargate as the launch type.	Create an Amazon ECS service by using the ECS cluster you created earlier. Make sure you choose Fargate as the launch type. Choose the task definition created in the previous step, and choose the target group of the Application Load Balancer.	Cloud administrator

Set up AWS PrivateLink

Task	Description	Skills required
Set up the AWS PrivateLink endpoint.	Open the Amazon VPC console, and create an AWS PrivateLink endpoint.  Associate this endpoint with the Network Load Balancer, which makes the application hosted on Amazon ECS available privately to customers.  For more information, see the Related resources section.	Cloud administrator

Create a VPC endpoint

Task	Description	Skills required
Create a VPC endpoint.	Create a VPC endpoint for the AWS PrivateLink endpoint that you created earlier. The VPC endpoint Fully Qualified Domain Name (FQDN) will point to the AWS PrivateLink endpoint FQDN. This creates an elastic network interface to the VPC endpoint service that the Domain Name Service endpoints can access.	Cloud administrator

Create the Lambda function

Task	Description	Skills required
Create the Lambda function.	Open the Lambda console and create a Lambda function to update the Application Load Balancer IP addresses as targets for the Network Load Balancer. For more information on this, see the "Using static IP addresses for Application Load Balancers" blog post in the Related resources section.	App developer

Related resources

Create the load balancers:

• Create a Network Load Balancer

• Create an Application Load Balancer

Create an Amazon EFS file system:

• Create an Amazon EFS file system

• Create mount targets in Amazon EFS

Create an S3 bucket:

• Create an S3 bucket

Create a Secrets Manager secret:

• Create keys in AWS KMS

• Create a secret in AWS Secrets Manager 

Create an Amazon RDS instance:

• Create an Amazon RDS DB instance

Create the Amazon ECS components:

• Create an Amazon ECS cluster 

• Create a Docker image 

• Create an Amazon ECR repository 

• Authenticate Docker with Amazon ECR repository

• Push an image to an Amazon ECR repository

• Create Amazon ECS task definition 

• Create an Amazon ECS service

Set up AWS PrivateLink:

• VPC endpoint services (AWS PrivateLink)

Create a VPC endpoint:

• Interface VPC endpoints (AWS PrivateLink)

Create the Lambda function:

• Create a Lambda function

Other resources:

• Using static IP addresses for Application Load Balancers

• Securely accessing services over AWS PrivateLink