Build a loosely coupled architecture with microservices using DevOps practices and AWS Cloud9

Created by Alexandre Nardi (AWS)

Environment: PoC or pilot

Technologies: DevOps; Serverless; Web & mobile apps; Databases

AWS services: AWS Cloud9; AWS CloudFormation; AWS CodePipeline; Amazon DynamoDB; AWS CodeCommit

Summary

Notice: AWS Cloud9 is no longer available to new customers. Existing customers of AWS Cloud9 can continue to use the service as normal. Learn more

Notice: AWS CodeCommit is no longer available to new customers. Existing customers of AWS CodeCommit can continue to use the service as normal. Learn more

This pattern demonstrates how to develop a typical web application in a serverless architecture, for developers and development leads who are beginning to test DevOps practices on Amazon Web Services (AWS). It builds a sample application that creates a storefront and backend for browsing and purchasing books, and provides a microservice that can be developed independently. The pattern uses AWS Cloud9 as a development environment, an Amazon DynamoDB database as a data store, and AWS services such as AWS CodePipeline and AWS CodeBuild for continuous integration and continuous deployment (CI/CD) functionality.

The pattern guides you through the following development activities: 

• Creating a standard AWS Cloud9 development environment

• Using AWS CloudFormation templates to create a web application and a microservice for books

• Using AWS Cloud9 to modify the front-end, commit changes, and test changes

• Creating and testing a CI/CD pipeline to the microservice

• Automating unit tests

The code for this pattern is provided in GitHub, in the AWS DevOps End-to-End Workshop repository.

Prerequisites and limitations

Prerequisites 

• An active AWS account

• Files from the AWS DevOps End-to-End Workshop downloaded to your computer

Important: Building this demo application in your AWS account creates and consumes AWS resources. You are responsible for the cost of the AWS services and resources used to create and run the application. After you finish your work, be sure to remove all resources to avoid ongoing charges. For cleanup instructions, see the Epics section. 

Limitations 

This walkthrough is intended for demonstration and development purposes only. To use it in a production environment, see Security best practices in the AWS Identity and Access Management (IAM) documentation, and make the necessary changes to IAM roles, Amazon DynamoDB, and other services used. The web application is derived from the AWS Bookstore Demo App; for additional considerations, see the Known limitations section of the README file.

Architecture

The architecture of the bookstore application is illustrated in the Architecture section of the README file for the AWS Bookstore Demo App. 

From a deployment perspective, the Bookstore Demo App uses a single CloudFormation template to deploy all services and objects in one stack. This pattern makes a few changes to demonstrate how a particular developer or team could work in a specific product (Books), and update it independently from the rest of the application. For this reason, the code for this pattern separates the AWS Lambda functions and related objects for the Books microservice into a second CloudFormation template, which creates a Books stack. That makes it possible to see the microservice being updated by using CI/CD practices. In the following diagram, the dashed border identifies the Books microservice.

Tools

Tools

• Jest framework for JavaScript testing

• Python 3.9

Code

The source code and templates for this pattern are available on GitHub, in the AWS DevOps End-to-End Workshop repository. Before you follow the steps in the Epics section, download all the files from the repository to your computer.

Note: The Epics section provides the high-level steps for this walkthrough, to give you general information about the process. To complete each step, see the README file in the AWS DevOps End-to-End Workshop repository for detailed instructions.

The AWS DevOps End-to-End Workshop repository extends the AWS Bookstore Demo App repository and uses a modified version of the AWS Cloud9 Bootstrapping code to create the AWS Cloud9 IDE.

Best practices

Using the Bookstore application is straightforward. Here are some recommended best practices:

• When you install the application, you can use a project name of your choice or use the default name (demobookstore) for convenience.

• After you have the application up and running, it’s a good practice to shut down the Amazon Neptune database if you want to keep testing for another day, because the database instance could result in additional charges. However, be aware that the database will automatically be started after seven days.

• For code details, see the documentation for the AWS Bookstore Demo App repository. It describes each microservice and table.

• For additional best practices, see the Some challenges if you have time... section of the README file in the AWS DevOps End-to-End Workshop repository. We recommend that you review the information to dive deep into additional features for security and to practice decoupling services.

Epics

Download the source code

Task	Description	Skills required
Download the source code from GitHub.	The source code and templates for this pattern are available in GitHub, in the AWS DevOps End-to-End Workshop repository. Before you follow the next steps in the Epics section, download all the files from the repository to your computer. Note: The Epics section provides the high-level steps for this walkthrough, to give you general information about the process. To complete each step, see the README file in the AWS DevOps End-to-End Workshop repository for detailed instructions. The AWS DevOps End-to-End Workshop repository extends the AWS Bookstore Demo App repository and uses a modified version of the AWS Cloud9 Bootstrapping code to create the AWS Cloud9 IDE.	App developer

Build the Bookstore web application and the Books microservice

Task	Description	Skills required
Create the front-end and Lambda functions for the Bookstore app.	Log in to the CloudFormation console, and deploy the DemoBookstoreMainTemplate.yml template to create the DemoBookStoreStack stack. This creates the front-end and Lambda functions that are outside the Books microservice. In the Outputs tab of the stack, note the website URL under the WebApplication label.	Developer
Create the Books microservice.	On the CloudFormation console, deploy the DemoBookstoreBooksServiceTemplate.yml template to create the DemoBooksServiceStack stack.	Developer
Test your application.	Use the website URL from the DemoBookStoreStack stack to access the Bookstore application.	Developer

Use the Cloud9 environment to maintain your application

Task	Description	Skills required
Create an AWS Cloud9 IDE.	On the CloudFormation console, deploy the C9EnvironmentTemplate.yml template to create an AWS Cloud9 environment.	Developer, Developer lead
Create CodeCommit repositories.	Log in to the AWS CodeCommit console, and verify that you have a demobookstore-WebAssets repository, which contains the code for the front-end application. Create a repository for the Books microservice called demobookstore-BooksService. Clone the two repositories in AWS Cloud9 (demobookstore-WebAssets and demobookstore-BooksService) by using the git clone command.	Developer
Change the code in the frontend and check the pipeline.	Use AWS Cloud9 to make some code changes on a webpage. This will update the demobookstore-WebAssets repository. On the AWS CodePipeline console, verify that demobookstore-Assets-Pipeline is running.  Test your web application by refreshing it from the browser (Ctrl+F5 on Firefox).	Developer

Implement a CI/CD pipeline for the Books microservice

Task	Description	Skills required
Add the YAML files for the build and service update.	In AWS Cloud9, upload the buildspec.yml and DemoBookstoreBooksServiceUpdateTemplate.yml files. buildspec.yml has building instructions, and also includes testing instructions for automated tests. They are commented at this point, and will be used later. DemoBookstoreBooksServiceUpdateTemplate.yml is an updated version of DemoBookstoreBooksServiceTemplate.yml, to be used in the deployment stage of the pipeline. Commit and push the files.	Developer
Create an S3 bucket for the build pipeline.	To create an S3 bucket, follow the instructions in the Amazon S3 documentation. The bucket name must be globally unique; for example, demobookstore-books-service-pipeline-bucket-<YYYYMMDDHHMM>.  Clear the Block all public access check box, and select the I acknowledge… check box.	Developer
Use IAM to create a role for CloudFormation deployment.	Create a demobookstore-CloudFormation-role role and attach the AdministratorAccess policy. In the next epic, you can reconfigure this role for minimum permissions.	Developer
Create a new pipeline to automate building and deploying the Books microservice.	Create a pipeline (for example, demobookstore-BooksService-Pipeline) with Commit, Build, and Deploy stages, as described in the README file.	Developer
Test your microservice in AWS Cloud9.	Make a change in the ListBooks function and see the pipeline working.	Developer
Automate the unit test for the ListBooks Lambda function.	In the AWS Cloud9 IDE, enable the build to run unit tests, and check the test results. For instructions, see the README file.	Developer

(Optional) Implement additional functionality

Task	Description	Skills required
Make your solution secure.	Configure demobookstore-CloudFormation-role to have minimum permissions, and check other used roles as well.	Developer
Eliminate dependencies in the CloudFormation templates.	The method for exchanging information between the DemoBookstoreMainTemplate.yml  template and the DemoBookstoreBooksServiceTemplate.yml template is based on outputs and imports. Passing values between these two templates adds dependencies. To eliminate the dependencies, consider using AWS Systems Manager Parameter Store.	Developer
Create a Cart microservice.	Use the Books microservice as an example for taking shopping cart-related functions out of the DemoBookstoreMainTemplate.yml template and creating a Cart microservice.	Developer

Clean up

Task	Description	Skills required
Delete the S3 buckets.	On the Amazon S3 console, delete the following buckets that are associated with the sample web application: Two buckets created for the AWS Bookstore Demo App. The buckets names start with the stack name you provided for AWS CloudFormation when you created the frontend;  for example, DemoBookStoreStack. One bucket for the build pipeline; for example, demobookstore-books-service-pipeline-bucket-<YYYYMMDDHHMM>.	Developer
Delete the stacks.	On the CloudFormation console, delete the stacks associated with the sample web applicaton: DemoBooksServiceStack DemoBookStoreStack The removal could take more than 90 minutes. If the removal fails, delete them again, and also delete any manual resources (for example, the VPC or network interfaces) based on notifications.	Developer
Delete the IAM roles.	On the IAM console, delete the following roles: demobookstore-Cloudformation-role demobookstore-BooksService-BuildProject-service-role For step-by-step instructions, see the IAM documentation.	Developer

Related resources

• AWS Bookstore Demo App 

• AWS Cloud9 Bootstrapping example 

• Creating a stack on the AWS CloudFormation console (AWS CloudFormation documentation)

• Creating a bucket (Amazon S3 documentation)

Additional information

For detailed, step-by-step instructions, see the README file in the AWS DevOps End-to-End Workshop GitHub repository.

About the May 2023 update: This pattern was updated to use newer versions of Node and Python. We updated many of the packages in the source code and removed Glyphicon because it’s no longer free. We also removed all dependencies on the AWS Bookstore Demo App repository, so the two repositories can now evolve independently.