Deploy Lambda functions with container images

Created by Ram Kandaswamy (AWS)

Environment: Production	Technologies: Containers & microservices; CloudNative; DevelopmentAndTesting; Serverless	Workload: All other workloads
AWS services: Amazon EC2 Container Registry; AWS Lambda

Summary

AWS Lambda supports containers images as a deployment model. This pattern shows how to deploy Lambda functions through container images. 

Lambda is a serverless, event-driven compute service that you can use to run code for virtually any type of application or backend service without provisioning or managing servers. With container image support for Lambda functions, you get the benefits of up to 10 GB of storage for your application artifact and the ability to use familiar container image development tools.

The example in this pattern uses Python as the underlying programming language, but you can use other languages, such as Java, Node.js, or Go. The pattern uses AWS CodeCommit as the source, but you could also use GitHub, Bitbucket, or Amazon Simple Storage Service (Amazon S3).

Prerequisites and limitations

Prerequisites 

• Amazon Elastic Container Registry (Amazon ECR) activated

• Application code

• Docker images with the runtime interface client and the latest version of Python

Limitations 

• Maximum image size supported is 10 GB.

• Maximum runtime for a Lambda based container deployment is 15 minutes.

Architecture

Target technology stack  

• Python programming language

• AWS CodeBuild

• AWS CodeCommit

• Docker image

• Amazon ECR

• AWS Identity and Access Management (IAM)

• AWS Lambda

• Amazon CloudWatch Logs

Target architecture 

1. You create a repository and commit the application code using CodeCommit.

2. The CodeBuild project is initiated when a change is made to CodeCommit, which is used as the source provider.

3. The CodeBuild project creates the Docker image and publishes the image to Amazon ECR.

4. You create the Lambda function by using the image in Amazon ECR.

Automation and scale

This pattern can be automated by using AWS CloudFormation, AWS Cloud Development Kit (AWS CDK), or API operations from an SDK. Lambda can automatically scale based on the number of requests, and you can tune it by using the concurrency parameters. For more information, see the Lambda documentation.

Tools

AWS services

• AWS CloudFormation Designer provides an integrated JSON and YAML editor that helps you view and edit CloudFormation templates.

• AWS CodeBuild is a fully managed build service that helps you compile source code, run unit tests, and produce artifacts that are ready to deploy.

• AWS CodeCommit is a version control service that helps you privately store and manage Git repositories, without needing to manage your own source control system.

• AWS CodeStar is a cloud-based service for creating, managing, and working with software development projects on AWS. For this pattern, you can use AWS CodeStar or another development environment.

• Amazon Elastic Container Registry (Amazon ECR) is a managed container image registry service that’s secure, scalable, and reliable.

• AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use.

Other tools

• Docker is a set of platform as a service (PaaS) products that use virtualization at the operating-system level to deliver software in containers.

Best practices

• Make your function as efficient and small as possible to avoid loading unnecessary files.

• Strive to have static layers higher up in your Docker file list, and place layers that change more often lower down. This improves caching, which improves performance.

• The image owner is responsible for updating and patching the image. Add that update cadence to your operational processes. For more information, see the AWS Lambda documentation.

Epics

Create a project in CodeBuild

Task	Description	Skills required
Create a CodeCommit repository.	Create a CodeCommit repository that will contain the Dockerfile, the buildspec.yaml file, and the application source code. For more information, see the AWS CodeCommit documentation.	Developer
Create a CodeBuild project.	On the CodeBuild console, create a new project that uses the CodeCommit repo and the buildspec.yaml file. You will use the CodeBuild project to create the image. Confirm that privileged mode is enabled. To build Docker images, this is necessary. Otherwise, the image will not build successfully. Provide values for project name and description. For the source provider, choose CodeCommit. For more information, see the AWS documentation.	Developer
Edit the Dockerfile.	The Dockerfile should be located in the top-level directory where you're developing the application. The Python code should be in the src folder. When you create the image, use the official Lambda supported images. Otherwise, a bootstrap error will occur, making the packing process more difficult. For details, see the Additional information section.	Developer
Create a repo in Amazon ECR.	Create a container repository in Amazon ECR. In the following example command, the name of the repository created is cf-demo. The repository will be reused in the buildspec.yaml file. aws ecr create-repository --cf-demo	AWS administrator, Developer
Push the image to Amazon ECR.	You can use CodeBuild to perform the image-build process. CodeBuild needs permission to interact with Amazon ECR and to work with S3. As part of the process, the Docker image is built and pushed to the Amazon ECR registry. For details on the template and the code, see the Additional information section.	Developer
Verify that the image is in the repository.	To verify that the image is in the repository, on the Amazon ECR console, choose Repositories. The image should be listed, with tags and with the results of a vulnerability scan report if that feature was turned on in the Amazon ECR settings.  For more information, see the AWS documentation.	Developer

Create the Lambda function to run the image

Task	Description	Skills required
Create the Lambda function.	On the Lambda console, choose Create function, and then choose Container image. Enter the function name and the URI for the image that is in the Amazon ECR repository, and then choose Create function. For more information, see the AWS Lambda documentation.	App developer
Test the Lambda function.	To invoke and test the function, choose Test. For more information, see the AWS Lambda documentation.	App developer

Troubleshooting

Issue	Solution
Build is not succeeding.	Check if the privileged mode is turned on for the CodeBuild project. Ensure that the Docker related commands have the necessary permissions. Trying adding sudo to the commands. Verify that the IAM role associated with CodeBuild has a policy with appropriate actions to interact with Amazon ECR, Amazon S3, and CloudWatch logs.

Related resources

• Base images for Lambda

• Docker sample for CodeBuild

• Pass temporary credentials

Additional information

Edit the Dockerfile

The following code shows the commands that you edit in the Dockerfile.

FROM public.ecr.aws/lambda/python:3.11

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT} 
COPY requirements.txt  ${LAMBDA_TASK_ROOT} 

# install dependencies
RUN pip3 install --user -r requirements.txt

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.lambda_handler" ]

The FROM command value corresponds to the Python 3.11 base image that is using the Lambda function in the public Amazon ECR image repository. 

The COPY app.py ${LAMBDA_TASK_ROOT} command copies the code to the task root directory, which the Lambda function will use. This command uses the environment variable so we don’t have to worry about the actual path. The function to be run is passed as an argument to the CMD [ "app.lambda_handler" ] command.

The COPY requirements.txt command captures the dependencies necessary for the code. 

The RUN pip install --user -r requirements.txt command installs the dependencies to the local user directory. 

To build your image, run the following command.

docker build -t <image name> .

Add the image in Amazon ECR

In the following code, replace aws_account_id with the account number, and replace us-east-1 if you are using a different Region. The buildspec file uses the CodeBuild build number to uniquely identify image versions as a tag value. You can change this to fit your requirements.

The buildspec custom code

phases:
  install:
    runtime-versions:
       python: 3.11
  pre_build:
    commands:
      - python3 --version
      - pip3 install --upgrade pip
      - pip3 install --upgrade awscli
      - sudo docker info
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...
      - ls
      - cd app
      - docker build -t cf-demo:$CODEBUILD_BUILD_NUMBER .
      - docker container ls
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin aws_account_id.dkr.ecr.us-east-1.amazonaws.com
      - docker tag cf-demo:$CODEBUILD_BUILD_NUMBER aws_account_id.dkr.ecr.us-east-1.amazonaws.com/cf-demo:$CODEBUILD_BUILD_NUMBER
      - docker push aws_account_id.dkr.ecr.us-east-1.amazonaws.com/cf-demo:$CODEBUILD_BUILD_NUMBER