Automatically detect changes and initiate different CodePipeline pipelines for a monorepo in CodeCommit

Created by Helton Ribeiro (AWS), Petrus Batalha (AWS), and Ricardo Morais (AWS)

Code repository: AWS CodeCommit monorepo multi-pipeline triggers	Environment: PoC or pilot	Technologies: DevOps; Infrastructure; Serverless
AWS services: AWS CodeCommit; AWS CodePipeline; AWS Lambda

Summary

This pattern helps you automatically detect changes to the source code of a monorepo-based application in AWS CodeCommit and then initiate a pipeline in AWS CodePipeline that runs the continuous integration and continuous delivery (CI/CD) automation for each microservice. This approach means that each microservice in your monorepo-based application can have a dedicated CI/CD pipeline, which ensures better visibility, easier sharing of code, and improved collaboration, standardization, and discoverability.

The solution described in this pattern doesn't perform any dependency analysis among the microservices inside the monorepo. It only detects changes in the source code and initiates the matching CI/CD pipeline.

The pattern uses AWS Cloud9 as the integrated development environment (IDE) and AWS Cloud Development Kit (AWS CDK) to define an infrastructure by using two AWS CloudFormation stacks: MonoRepoStack and PipelinesStack. The MonoRepoStack stack creates the monorepo in AWS CodeCommit and the AWS Lambda function that initiates the CI/CD pipelines. The PipelinesStack stack defines your pipeline infrastructure.

Important: This pattern’s workflow is a proof of concept (PoC). We recommend that you use it only in a test environment. If you want to use this pattern’s approach in a production environment, see Security best practices in IAM in the AWS Identity and Access Management (IAM) documentation and make the required changes to your IAM roles and AWS services. 

Prerequisites and limitations

Prerequisites 

• An active AWS account.

• AWS Command Line Interface (AWS CLI), installed and configured. For more information, see Installing, updating, and uninstalling the AWS CLI in the AWS CLI documentation.  

• Python 3 and pip, installed on your local machine. For more information, see the Python documentation. 

• AWS CDK, installed and configured. For more information, see Getting started with the AWS CDK in the AWS CDK documentation. 

• An AWS Cloud9 IDE, installed and configured. For more information, see Setting up AWS Cloud9 in the AWS Cloud9 documentation. 

• The GitHub AWS CodeCommit monorepo multi-pipeline triggers repository, cloned on your local machine. 

• An existing directory containing application code that you want to build and deploy with CodePipeline.

• Familiarity and experience with DevOps best practices on the AWS Cloud. To increase your familiarity with DevOps, you can use the pattern Build a loosely coupled architecture with microservices using DevOps practices and AWS Cloud9 on the AWS Prescriptive Guidance website.  

Architecture

The following diagram shows how to use the AWS CDK to define an infrastructure with two AWS CloudFormation stacks: MonoRepoStack and PipelinesStack.

The diagram shows the following workflow:

1. The bootstrap process uses the AWS CDK to create the AWS CloudFormation stacks MonoRepoStack and PipelinesStack.

2. The MonoRepoStack stack creates the CodeCommit repository for your application and the monorepo-event-handler Lambda function that is initiated after each commit.

3. The PipelinesStack stack creates the pipelines in CodePipeline that are initiated by the Lambda function. Each microservice must have a defined infrastructure pipeline.

4. The pipeline for microservice-n is initiated by the Lambda function and starts its isolated CI/CD stages that are based on the source code in CodeCommit.

5. The pipeline for microservice-1 is initiated by the Lambda function and starts its isolated CI/CD stages that are based on the source code in CodeCommit.

The following diagram shows the deployment of the AWS CloudFormation stacks MonoRepoStack and PipelinesStack in an account.

1. A user changes code in one of the application’s microservices.

2. The user pushes the changes from a local repository to a CodeCommit repository.

3. The push activity initiates the Lambda function that receives all pushes to the CodeCommit repository.

4. The Lambda function reads a parameter in Parameter Store, a capability of AWS Systems Manager, to retrieve the most recent commit ID. The parameter has the naming format: /MonoRepoTrigger/{repository}/{branch_name}/LastCommit. If the parameter isn’t found, the Lambda function reads the last commit ID from the CodeCommit repository and saves the returned value in Parameter Store.

5. After identifying the commit ID and the changed files, the Lambda function identifies the pipelines for each microservice directory and initiates the required CodePipeline pipeline.

Tools

• AWS Cloud Development Kit (AWS CDK) is a software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation.

• Python is a programming language that lets you work quickly and integrate systems more effectively.

Code 

The source code and templates for this pattern are available in the GitHub AWS CodeCommit monorepo multi-pipeline triggers repository.

Best practices

• This sample architecture doesn't include a monitoring solution for the deployed infrastructure. If you want to deploy this solution in a production environment, we recommend that you enable monitoring. For more information, see Monitor your serverless applications with CloudWatch Application Insights in the AWS Serverless Application Model (AWS SAM) documentation.

• When you edit the sample code provided by this pattern, follow the best practices for developing and deploying cloud infrastructure in the AWS CDK documentation.

• When you define your microservice pipelines, review the security best practices in the AWS CodePipeline documentation.

• You can also check your AWS CDK code for best practices by using the cdk-nag utility. This tool uses a set of rules, grouped by packs, to evaluate your code. The available packs are:

  • AWS Solutions Library

  • Health Insurance Portability and Accountability Act (HIPAA) security

  • National Institute of Standards and Technology (NIST) 800-53 rev 4

  • NIST 800-53 rev 5

  • Payment Card Industry Data Security Standard (PCI DSS) 3.2.1

Epics

Set up the environment

Task	Description	Skills required
Create a virtual Python envionment.	In your AWS Cloud9 IDE, create a virtual Python environment and install the required dependencies by running the following command: make install	Developer
Bootstrap the AWS account and AWS Region for the AWS CDK.	Bootstrap the required AWS account and Region by running the following command: make bootstrap account-id=<your-AWS-account-ID> region=<required-region>	Developer

Add a new pipeline for a microservice

Task	Description	Skills required
Add your sample code to your application directory.	Add the directory that contains your sample application code to the monorepo-sample directory in the cloned GitHub AWS CodeCommit monorepo multi-pipeline triggers repository.	Developer
Edit the monorepo-main.json file.	Add the directory name of your application’s code and the pipeline's name to the monorepo-main.json file in the cloned repository .	Developer
Create the pipeline.	In the Pipelines directory for the repository, add the pipeline class for your application. The directory contains two sample files, pipeline_hotsite.py and pipeline_demo.py. Each file has three stages: source, build, and deploy. You can copy one of the files and makes changes to it according to your application’s requirements.	Developer
Edit the monorepo_config.py file.	In service_map, add the directory name for your application and the class that you created for the pipeline. For example, the following code shows a pipeline definition in the Pipelines directory that uses a file named pipeline_mysample.py  with a MySamplePipeline class: ... # Pipeline definition imports from pipelines.pipeline_demo import DemoPipeline from pipelines.pipeline_hotsite import HotsitePipeline from pipelines.pipeline_mysample import MySamplePipeline ### Add your pipeline configuration here service_map: Dict[str, ServicePipeline] = { # folder-name -> pipeline-class 'demo': DemoPipeline(), 'hotsite': HotsitePipeline(), 'mysample': MySamplePipeline() }	Developer

Deploy the MonoRepoStack stack

Task	Description	Skills required
Deploy the AWS CloudFormation stack.	Deploy the AWS CloudFormation MonoRepoStack stack with default parameter values in the root directory of the cloned repository by running the make deploy-core command. You can change the repository’s name by running the make deploy-core monorepo-name=<repo_name> command. Note: You can simultaneously deploy both pipelines by using the make deploy monorepo-name=<repo_name> command.	Developer
Validate the CodeCommit repository.	Validate that your resources were created by running the aws codecommit get-repository --repository-name <repo_name> command. Important: Because the AWS CloudFormation stack creates the CodeCommit repository where the monorepo is stored, don’t run the cdk destroy MonoRepoStack command if you have started to push modifications into it.	Developer
Validate the AWS CloudFormation stack results.	Validate that the AWS CloudFormation MonoRepoStack stack is correctly created and configured by running the following command: aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE --query 'StackSummaries[?StackName == 'MonoRepoStack']'	Developer

Deploy the PipelinesStack stack

Task	Description	Skills required
Deploy the AWS CloudFormation stack.	The AWS CloudFormation PipelinesStack stack must be deployed after you deploy the MonoRepoStack stack. The stack increases in size when new microservices are added to the monorepo’s code base and is redeployed when a new microservice is onboarded. Deploy the PipelinesStack stack by running the make deploy-pipelines command. Note: You can also deploy simultaneously deploy both pipelines by running the make deploy monorepo-name=<repo_name> command. The following sample output shows how the PipelinesStacks deployment prints the URLs for the microservices at the end of the implementation: Outputs: PipelinesStack.demourl = .cloudfront.net PipelinesStack.hotsiteurl = .cloudfront.net	Developer
Validate the AWS CloudFormation stack results.	Validate that the AWS CloudFormation PipelinesStacks stack is correctly created and configured by running the following command: aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE UPDATE_COMPLETE --query 'StackSummaries[?StackName == 'PipelinesStack']'	Developer

Clean up resources

Task	Description	Skills required
Delete your AWS CloudFormation stacks.	Run the make destroy command.	Developer
Delete the S3 buckets for your pipelines.	Sign in to the AWS Management Console and open the Amazon Simple Storage Service (Amazon S3) console. Delete the S3 buckets that are associated with your pipelines and use the following name: pipelinesstack-codepipeline*	Developer

Troubleshooting

Issue	Solution
I encountered AWS CDK issues.	See Troubleshooting common AWS CDK issues in the AWS CDK documentation.
I pushed my microservice code, but the microservice pipeline didn't run.	Setup validation Verify branch configuration: Make sure that you are pushing your code to the correct branch. This pipeline is configured to run only when changes are made to the main branch. Pushes to other branches don’t initiate the pipeline unless they are configured specifically. After you push your code, check if the commit is visible in AWS CodeCommit to make sure that the push was successful and that the connection between your local environment and the repository is intact. Refresh your credentials if there are issues pushing code. Validate configuration files: Confirm that the service_map variable in monorepo_config.py accurately reflects the current directory structure of your microservices. This variable plays a crucial role in mapping your code push to the respective pipeline. Make sure that monorepo-main.json is updated to include the new mapping for your microservice. This file is essential for the pipeline to recognize and correctly handle changes to your microservice. Troubleshooting on the console AWS CodePipeline checks: On the AWS Management Console, confirm that you're in the AWS Region where your pipeline is hosted. Open the CodePipeline console and check if the pipeline that corresponds to your microservice has been initiated. Error analysis: If the pipeline was initiated but failed, review any error messages or logs provided by CodePipeline to understand what went wrong. AWS Lambda troubleshooting: On the AWS Lambda console, open the monorepo-event-handler Lambda function. Verify that the function was initiated in response to the code push. Log analysis: Examine the Lambda function's logs for any issues. The logs can provide detailed insights into what happened when the function ran and help identify whether the function processed the event as expected.
I need to redeploy all my microservices.	There are two approaches to force the redeployment of all microservices. Choose the option that fits your requirements. Approach 1: Delete a parameter in Parameter Store This method involves deleting a specific parameter within Systems Manager Parameter Store that tracks the last commit ID used for deployment. When you remove this parameter, the system is forced to redeploy all microservices upon the next trigger, because it perceives it as a fresh state. Steps: Locate the specific Parameter Store entry that holds the commit ID or a related deployment marker for your monorepo. The parameter name follows the format: "/MonoRepoTrigger/{repository}/{branch_name}/LastCommit" Consider backing up the parameter value if it's critical or if you wish to maintain a record of the deployment state before resetting it. Use the AWS Management Console, AWS CLI, or SDKs to delete the identified parameter. This action resets the deployment marker. After deletion, the next push to the repository should cause the system to deploy all microservices, because it looks for the latest commit to consider for deployment. Pros: Simple and quick to implement with minimal steps. Doesn’t require making arbitrary code changes to initiate deployments. Cons: Less granular control over the deployment process. Potentially risky if the Parameter Store is used for managing other critical configurations. Approach 2: Push a commit in each monorepo subfolder This method involves making a minor change and pushing it in each microservice subfolder within the monorepo to initiate their individual pipelines. Steps: List all the microservices within the monorepo that need redeployment. For each microservice, make a minimal, non-impactful change in its subfolder. This might be updating a README file, adding a comment in a configuration file, or any change that doesn’t affect the service's functionality. Commit these changes with a clear message (such as "Initiate redeployment of microservices") and push them to the repository. Make sure that you push the changes to the branch that initiates the deployment. Monitor the pipelines for each microservice to confirm that they are initiated and complete successfully. Pros: Provides granular control over which microservices are redeployed. Safer because it doesn’t involve deleting configuration parameters that might be used for other purposes. Cons: More time-consuming, especially with a large number of microservices. Requires making unnecessary code changes that could clutter the commit history.

Related resources

• Continuous integration and delivery (CI/CD) using CDK Pipelines (AWS CDK documentation)

• aws-cdk/pipelines module (AWS CDK API reference)