Create a CI/CD pipeline to deploy microservices with AWS Fargate and Amazon API Gateway - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resourcesAttachments

Create a CI/CD pipeline to deploy microservices with AWS Fargate and Amazon API Gateway

Created by Jonathan Cardoso (AWS)

Environment: PoC or pilot

Technologies: DevOps; Containers & microservices

AWS services: AWS CodeBuild; AWS CodeCommit; AWS CodeDeploy; AWS CodePipeline; AWS Fargate

Summary

This pattern describes how to create a continuous integration and continuous deployment (CI/CD) pipeline to deploy microservices in the Amazon Web Services (AWS) Cloud. Traditionally, building a modern application involves decoupling and optimizing business logic into a microservices architecture to reduce complexity and to improve speed and alignment. APIs, which abstract communications among services, are the foundation of microservices. Having a CI/CD pipeline for deploying microservices and APIs can accelerate the development of modern applications and help maintain system stability and security.

This pattern uses AWS Fargate to provision compute resources for microservices and Amazon API Gateway to manage the APIs that handle communications among the microservices efficiently. The CI/CD pipeline is set up by implementing three AWS services: AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy. The pattern provides AWS CloudFormation templates and Python code to automate most of the steps, and includes a sample application.

Prerequisites and limitations

Prerequisites

Architecture

The pattern uses the following AWS services, as illustrated in the following diagram:

  • In the private subnets, AWS Fargate manages the compute resources for microservices, to remove the infrastructure burden.

  • API Gateway handles API calls and makes it easier to create, maintain, and secure APIs at any scale.

  • AWS KMS creates and controls the keys for encrypting content such as artifacts and reports.

  • Amazon Simple Storage Service (Amazon S3) is used as a repository to store artifacts and reports that are generated by the pipeline.

The following diagram illustrates the CI/CD workflow.

 

Tools

AWS services

  • API Gateway – Amazon API Gateway helps create, publish, maintain, monitor, and secure REST, HTTP, and WebSocket APIs for use in your applications and microservices.

  • AWS CLI – AWS Command Line Interface (AWS CLI) is an open-source tool that you can use to interact with AWS services from the command line.

  • CodeCommit – AWS CodeCommit is a fully managed source control service that stores and manages documents, source code, and binary files in the AWS Cloud.

  • CodePipeline – AWS CodePipeline is a continuous delivery service that helps you model and configure the different stages of a software release process.

  • CodeBuild – AWS CodeBuild is a build service that compiles source code, runs unit tests, and produces software packages that are ready to deploy.

  • CodeDeploy – AWS CodeDeploy is a software deployment service that automates deployments to Amazon Elastic Compute Cloud (Amazon EC2) instances, on-premises instances, serverless AWS Lambda functions, or Amazon Elastic Container Service (Amazon ECS) services.

  • Fargate – AWS Fargate is a serverless compute engine that works with Amazon ECS and provides on-demand, right-sized compute capacity for containers.

  • Amazon S3 – Amazon Simple Storage Service (Amazon S3) is an object storage service that lets you store and retrieve any amount of data at any time, from anywhere on the web.

Code

The code for this pattern is attached. You can download and unzip the project on your computer to use the files that are referenced in the Epics section.

Epics

TaskDescriptionSkills required
Create an IAM user.

In your AWS account, create an AWS Identity and Access Management (IAM) user. For instructions, see the IAM documentation.

Developer, Systems administrator, DevOps engineer
Configure your credentials.

Assign the IAM user programmatic access, so they can use the AWS CLI. For instructions, see the AWS CLI documentation.

Developer, Systems administrator, DevOps engineer
Test your access.

To validate that you configured the IAM user correctly, use the following AWS CLI command:

aws sts get-caller-identity --query Account --output text --profile <profile>

where <profile> specifies a profile from your credential file. The command outputs your account ID.

Developer, Systems administrator, DevOps engineer
TaskDescriptionSkills required
Run the CloudFormation template to deploy the infrastructure.

In your AWS account, create a virtual private cloud (VPC) that is configured with public and private subnets across two Availability Zones and an Application Load Balancer. To deploy this infrastructure, use the following AWS CLI command:

aws cloudformation deploy \
--stack-name foundation \
--template-file foundation/full.yaml \
--capabilities CAPABILITY_IAM \
--region <region> \
--profile <profile>

where foundation/full.yaml refers to the template that's included in the code attachment, <region> is the AWS Region to use, and  <profile> specifies a profile from your credential file.

Systems administrator, DevOps engineer
TaskDescriptionSkills required
Create a pipeline.

Open the Cloudformation console and create a new stack by launching the pipeline.yaml template, which is in the attachment. Configure the following parameters:

  • Stack name represents the pipeline for the application (default is myapp).

  • ServiceName is the application name, and should match the service name in myapp/templates/service.yaml (default is myapp).

  • BranchName specifies a branch in your code repository (default is main).

  • Email is the email address to use for notifications when manual approval is enabled.

  • ManualApproval indicates whether manual approval is required for pipeline operations (default is true).

This stack creates the following resources:

  • A CodeCommit repository, using the ServiceName parameter

  • An Amazon Elastic Container Registry (Amazon ECR) repository to store your Docker images

  • A CodePipeline pipeline for your application

  • If ManualApproval is set to true, an Amazon Simple Notification Service (Amazon SNS) topic that notifies the email address when there's a commit operation in the pipeline (you receive a confirmation email to confirm your subscription to the SNS topic)

The created pipeline contains these stages:

  • Source: Monitors any changes in the CodeCommit repository.

  • Continuous Integration: Tests your application by using the tools git-secrets, unittest, cfn-nag, hadolint, Dependency-Check, Bandit, Safety, and Trivy.

  • Publish: Builds and pushes the Docker image to the Docker image repository.

  • Deploy: After manual approval (if enabled), launches the CloudFormation template to deploy this new service version.

Systems administrator, DevOps engineer
Check the deployment.

You can check each step of your pipeline that was created, from source to deployment, in the CodePipeline console.

The pipeline won't run yet because the repository is empty. Next, you set up your sample application and start the pipeline.

Systems administrator, DevOps engineer
TaskDescriptionSkills required
Clone your CodeCommit repository.

Open the AWS CodeCommit console, select the repository created by the pipeline, and then choose Clone URL and HTTPS (as the protocol) to copy the URL. 

At the terminal or command prompt, clone the repository by using the command:

git clone <repoURL>

where <repoURL> refers to the clone URL you obtained from CodeCommit. For more information, see the CodeCommit documentation.

Developer
Copy the sample application files.

After you clone your repository, copy the content in the attachment, and place it in your new repository. You can use the following command on Linux or macOS:

cp -rpf ../aws-cicd-microservices-api/myapp/* <reponame>/

where <reponame> refers to the name of your CodeCommit repository.

Developer
Configure your application parameters.

Change the following two files to create your microservice.

In the templates/service.yaml file:

  • ServiceName: myapp (same as in pipeline.yaml)

  • ServicePath: /myapp (same as in app/main.py)

  • BranchName: main

  • AlbRulePriority: 2 (each microservice should have a unique AlbRulePriority number; increase this number by one for each microservice)

In the app/main.py file:

mypath="myapp"
Developer
Commit your changes.

Publish the updated files to the repository:

git add . 
git commit -m "first commit"
git push origin main
Developer
Check the deployment.

Now that you have completed your first commit operation, you can check each step of your pipeline, from source to deployment, in the CodePipeline console.

Systems administrator, DevOps engineer

Related resources

Attachments

To access additional content that is associated with this document, unzip the following file: attachment.zip