Using Lambda with infrastructure as code (IaC) - AWS Lambda

Using Lambda with infrastructure as code (IaC)

Lambda offers several ways to deploy your code and create functions. For instance, you can use the Lambda console or the AWS Command Line Interface (AWS CLI) to manually create or update Lambda functions. In addition to these manual options, AWS offers a number of solutions for deploying Lambda functions and serverless applications using infrastructure as code (IaC). With IaC, you can provision and maintain Lambda functions and other AWS resources using code instead of using manual processes and settings.

Most of the time, Lambda functions don’t run in isolation. Instead, they form part of a serverless application with other resources such as databases, queues, and storage. With IaC, you can automate your deployment processes to quickly and repeatably deploy and update whole serverless applications involving many separate AWS resources. This approach speeds up your development cycle, makes configuration management easier, and ensures that your resources are deployed the same way every time.

IaC tools for Lambda

To deploy Lambda functions and serverless applications using IaC, AWS offers a number of different tools and services.

AWS CloudFormation was the first service offered by AWS to create and configure cloud resources. With AWS CloudFormation, you create text templates to define infrastructure and code. As AWS introduced more new services and the complexity of creating AWS CloudFormation templates increased, two further tools were released. AWS SAM is another template-based framework for defining serverless applications. The AWS Cloud Development Kit (AWS CDK) is a code-first approach for defining and provisioning infrastructure using code constructs in many popular programming languages.

With both AWS SAM and the AWS CDK, AWS CloudFormation operates behind the scenes to build and deploy your infrastructure. The following diagram illustrates the relationship between these tools, and the paragraphs after the diagram explain their key features.

Diagram showing how AWS SAM and AWS CDK both deploy AWS resources and code using AWS CloudFormation, which creates the application stack.
  • AWS CloudFormation - With CloudFormation you model and set up your AWS resources using a YAML or JSON template that describes your resources and their properties. CloudFormation provisions your resources in a safe, repeatable manner, enabling you to frequently build your infrastructure and applications without manual steps. When you change the configuration, CloudFormation determines the right operations to perform to update your stack. CloudFormation can even roll back changes.

  • AWS Serverless Application Model (AWS SAM) - AWS SAM is an open-source framework for defining serverless applications. AWS SAM templates use a shorthand syntax to define functions, APIs, databases, and event source mappings with just a few lines of text (YAML) per resource. During deployment, AWS SAM transforms and expands the AWS SAM syntax into AWS CloudFormation syntax. Because of this, any CloudFormation syntax can be added to AWS SAM templates. This gives AWS SAM all the power of CloudFormation, but with fewer lines of configuration.

  • AWS Cloud Development Kit (AWS CDK) - With the AWS CDK, you define your infrastructure using code constructs and provision it through AWS CloudFormation. AWS CDK enables you to model application infrastructure with TypeScript, Python, Java, .NET, and Go (in Developer Preview) using your existing IDE, testing tools, and workflow patterns. You get all the benefits of AWS CloudFormation, including repeatable deployment, easy rollback, and drift detection.

AWS also provides a service called AWS Application Composer to develop IaC templates using a simple graphical interface. With Application Composer, you design an application architecture by dragging, grouping, and connecting AWS services in a visual canvas. Application Composer then creates an AWS SAM template or an AWS CloudFormation template from your design that you can use to deploy your application.

In the Getting started with IaC for Lambda section below, you use Application Composer to develop a template for a serverless application based on an existing Lambda function.

Getting started with IaC for Lambda

In this tutorial, you can get started using IaC with Lambda by creating an AWS SAM template from an existing Lambda function and then building out a serverless application in Application Composer by adding other AWS resources.

If you'd rather start by carrying out an AWS SAM or AWS CloudFormation tutorial to learn how to work with templates without using Application Composer, you'll find links to other resources in the Next steps section at the end of this page.

As you carry out this tutorial, you’ll learn some fundamental concepts, like how AWS resources are specified in AWS SAM. You’ll also learn how to use Application Composer to build a serverless application you can deploy using AWS SAM or AWS CloudFormation.

To complete this tutorial, you’ll carry out the following steps:

  • Create an example Lambda function

  • Use the Lambda console to view the AWS SAM template for the function

  • Export your function’s configuration to AWS Application Composer and design a simple serverless application based on your function’s configuration

  • Save an updated AWS SAM template you can use as a basis to deploy your serverless application

In the Next steps section, you’ll find resources you can use to learn more about AWS SAM and Application Composer. These resources include links to more advanced tutorials that teach you how to deploy a serverless application using AWS SAM.

Prerequisites

In this tutorial, you use Application Composer’s local sync feature to save your template and code files to your local build machine. To use this feature, you need a browser that supports the File System Access API, which allows web applications to read, write, and save files in your local file system . We recommend using either Google Chrome or Microsoft Edge. For more information about the File System Access API, see What is the File System Access API?

Create a Lambda function

In this first step, you create a Lambda function you can use to complete the rest of the tutorial. To keep things simple, you use the Lambda console to create a basic 'Hello world' function using the Python 3.11 runtime.

To create a 'Hello world' Lambda function using the console
  1. Open the Lambda console.

  2. Choose Create function.

  3. Leave Author from scratch selected, and under Basic information, enter LambdaIaCDemo for Function name.

  4. For Runtime, select Python 3.11.

  5. Choose Create function.

View the AWS SAM template for your function

Before you export your function configuration to Application Composer, use the Lambda console to view your function's current configuration as an AWS SAM template. By following the steps in this section, you'll learn about the anatomy of an AWS SAM template and how to define resources like Lambda functions to start specifying a serverless application.

To view the AWS SAM template for your function
  1. Open the Functions page of the Lambda console.

  2. Choose the function you just created (LambdaIaCDemo).

  3. In the Function overview pane, choose Template.

    In place of the diagram representing your function’s configuration, you’ll see an AWS SAM template for your function. The template should look like the following.

    # This AWS SAM template has been generated from your function's # configuration. If your function has one or more triggers, note # that the AWS resources associated with these triggers aren't fully # specified in this template and include placeholder values.Open this template # in AWS Application Composer or your favorite IDE and modify # it to specify a serverless application with other AWS resources. AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: An AWS Serverless Specification template describing your function. Resources: LambdaIaCDemo: Type: AWS::Serverless::Function Properties: CodeUri: . Description: '' MemorySize: 128 Timeout: 3 Handler: lambda_function.lambda_handler Runtime: python3.11 Architectures: - x86_64 EventInvokeConfig: MaximumEventAgeInSeconds: 21600 MaximumRetryAttempts: 2 EphemeralStorage: Size: 512 RuntimeManagementConfig: UpdateRuntimeOn: Auto SnapStart: ApplyOn: None PackageType: Zip Policies: Statement: - Effect: Allow Action: - logs:CreateLogGroup Resource: arn:aws:logs:us-east-1:123456789012:* - Effect: Allow Action: - logs:CreateLogStream - logs:PutLogEvents Resource: - >- arn:aws:logs:us-east-1:123456789012:log-group:/aws/lambda/LambdaIaCDemo:*

Let’s take a moment to look at the YAML template for your function and understand some key concepts.

The template starts with the declaration Transform: AWS::Serverless-2016-10-31. This declaration is required because behind the scenes, AWS SAM templates are deployed through AWS CloudFormation. Using the Transform statement identifies the template as an AWS SAM template file.

Following the Transform declaration comes the Resources section. This is where the AWS resources you want to deploy with your AWS SAM template are defined. AWS SAM templates can contain a combination of AWS SAM resources and AWS CloudFormation resources. This is because during deployment, AWS SAM templates expand to AWS CloudFormation templates, so any valid AWS CloudFormation syntax can be added to an AWS SAM template.

At the moment, there is just one resource defined in the Resources section of the template, your Lambda function LambdaIaCDemo. To add a Lambda function to an AWS SAM template, you use the AWS::Serverless::Function resource type. The Properties of a Lambda function resource define the function’s runtime, function handler, and other configuration options. The path to your function’s source code that AWS SAM should use to deploy the function is also defined here. To learn more about Lambda function resources in AWS SAM, see AWS::Serverless::Function in the AWS SAM Developer Guide.

As well as the function properties and configurations, the template also specifies an AWS Identity and Access Management (IAM) policy for your function. This policy gives your function permission to write logs to Amazon CloudWatch Logs. When you create a function in the Lambda console, Lambda automatically attaches this policy to your function. To learn more about specifying an IAM policy for a function in an AWS SAM template, see the policies property on the AWS::Serverless::Function page of the AWS SAM Developer Guide.

To learn more about the structure of AWS SAM templates, see AWS SAM template anatomy.

Use AWS Application Composer to design a serverless application

To start building out a simple serverless application using your function’s AWS SAM template as a starting point, you export your function configuration to Application Composer and activate Application Composer’s local sync mode. Local sync automatically saves your function’s code and your AWS SAM template to your local build machine and keeps your saved template synced as you add other AWS resources in Application Composer.

To export your function to Application Composer
  1. In the Function Overview pane, choose Export to Application Composer.

    To export your function's configuration and code to Application Composer, Lambda creates an Amazon S3 bucket in your account to temporarily store this data.

  2. In the dialog box, choose Confirm and create project to accept the default name for this bucket and export your function's configuration and code to Application Composer.

  3. (Optional) To choose another name for the Amazon S3 bucket that Lambda creates, enter a new name and choose Confirm and create project. Amazon S3 bucket names must be globally unique and follow the bucket naming rules.

    Selecting Confirm and create project opens the Application Composer console. On the canvas, you’ll see your Lambda function.

  4. From the Menu dropdown, choose Activate local sync.

  5. In the dialog box that opens, choose Select folder and select a folder on your local build machine.

  6. Choose Activate to activate local sync.

To export your function to Application Composer, you need permission to use certain API actions. If you're unable to export your function, see Required permissions and make sure you have the permissions you need.

Note

Standard Amazon S3 pricing applies for the bucket Lambda creates when you export a function to Application Composer. The objects that Lambda puts into the bucket are automatically deleted after 10 days, but Lambda doesn't delete the bucket itself.

To avoid additional charges being added to your AWS account, follow the instructions in Deleting a bucket after you have exported your function to Application Composer. For more information about the Amazon S3 bucket Lambda creates, see Using AWS Lambda with AWS Application Composer.

To design your serverless application in Application Composer

After activating local sync, changes you make in Application Composer will be reflected in the AWS SAM template saved on your local build machine. You can now drag and drop additional AWS resources onto the Application Composer canvas to build out your application. In this example, you add an Amazon SQS simple queue as a trigger for your Lambda function and a DynamoDB table for the function to write data to.

  1. Add an Amazon SQS trigger to your Lambda function by doing the following:

    1. In the search field in the Resources palette, enter SQS.

    2. Drag the SQS Queue resource onto your canvas and position it to the left of your Lambda function.

    3. Choose Details, and for Logical ID enter LambdaIaCQueue.

    4. Choose Save.

    5. Connect your Amazon SQS and Lambda resources by clicking on the Subscription port on the SQS queue card and dragging it to the left hand port on the Lambda function card. The appearance of a line between the two resources indicates a successful connection. Application Composer also displays a message at the bottom of the canvas indicating that the two resources are successfully connected.

  2. Add an Amazon DynamoDB table for your Lambda function to write data to by doing the following:

    1. In the search field in the Resources palette, enter DynamoDB.

    2. Drag the DynamoDB Table resource onto your canvas and position it to the right of your Lambda function.

    3. Choose Details, and for Logical ID enter LambdaIaCTable.

    4. Choose Save.

    5. Connect the DynamoDB table to your Lambda function by clicking on the right hand port of the Lambda function card and dragging it to the left hand port on the DynamoDB card.

Now that you’ve added these extra resources, let’s take a look at the updated AWS SAM template Application Composer has created.

To view your updated AWS SAM template
  • On the Application Composer canvas, choose Template to switch from the canvas view to the template view.

Your AWS SAM template should now contain the following additional resources and properties:

  • An Amazon SQS queue with the identifier LambdaIaCQueue

    LambdaIaCQueue: Type: AWS::SQS::Queue Properties: MessageRetentionPeriod: 345600

    When you add an Amazon SQS queue using Application Composer, Application Composer sets the MessageRetentionPeriod property. You can also set the FifoQueue property by selecting Details on the SQS Queue card and checking or unchecking Fifo queue.

    To set other properties for your queue, you can manually edit the template to add them. To learn more about the AWS::SQS::Queue resource and its available properties, see AWS::SQS::Queue in the AWS CloudFormation User Guide.

  • An Events property in your Lambda function definition that specifies the Amazon SQS queue as a trigger for the function

    Events: LambdaIaCQueue: Type: SQS Properties: Queue: !GetAtt LambdaIaCQueue.Arn BatchSize: 1

    The Events property consists of an event type and a set of properties that depend on the type. To learn about the different AWS services you can configure to trigger a Lambda function and the properties you can set, see EventSource in the AWS SAM Developer Guide.

  • A DynamoDB table with the identifier LambdaIaCTable

    LambdaIaCTable: Type: AWS::DynamoDB::Table Properties: AttributeDefinitions: - AttributeName: id AttributeType: S BillingMode: PAY_PER_REQUEST KeySchema: - AttributeName: id KeyType: HASH StreamSpecification: StreamViewType: NEW_AND_OLD_IMAGES

    When you add a DynamoDB table using Application Composer, you can set your table's keys by choosing Details on the DynamoDB table card and editing the key values. Application Composer also sets default values for a number of other properties including BillingMode and StreamViewType.

    To learn more about these properties and other properties you can add to your AWS SAM template, see AWS::DynamoDB::Table in the AWS CloudFormation User Guide.

  • A new IAM policy that gives your function permission to perform CRUD operations on the DynamoDB table you added.

    Policies: ... - DynamoDBCrudPolicy: TableName: !Ref LambdaIaCTable

The complete final AWS SAM template should look like the following.

AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: An AWS Serverless Specification template describing your function. Resources: LambdaIaCDemo: Type: AWS::Serverless::Function Properties: CodeUri: . Description: '' MemorySize: 128 Timeout: 3 Handler: lambda_function.lambda_handler Runtime: python3.11 Architectures: - x86_64 EventInvokeConfig: MaximumEventAgeInSeconds: 21600 MaximumRetryAttempts: 2 EphemeralStorage: Size: 512 RuntimeManagementConfig: UpdateRuntimeOn: Auto SnapStart: ApplyOn: None PackageType: Zip Policies: - Statement: - Effect: Allow Action: - logs:CreateLogGroup Resource: arn:aws:logs:us-east-1:594035263019:* - Effect: Allow Action: - logs:CreateLogStream - logs:PutLogEvents Resource: - arn:aws:logs:us-east-1:594035263019:log-group:/aws/lambda/LambdaIaCDemo:* - DynamoDBCrudPolicy: TableName: !Ref LambdaIaCTable Events: LambdaIaCQueue: Type: SQS Properties: Queue: !GetAtt LambdaIaCQueue.Arn BatchSize: 1 Environment: Variables: LAMBDAIACTABLE_TABLE_NAME: !Ref LambdaIaCTable LAMBDAIACTABLE_TABLE_ARN: !GetAtt LambdaIaCTable.Arn LambdaIaCQueue: Type: AWS::SQS::Queue Properties: MessageRetentionPeriod: 345600 LambdaIaCTable: Type: AWS::DynamoDB::Table Properties: AttributeDefinitions: - AttributeName: id AttributeType: S BillingMode: PAY_PER_REQUEST KeySchema: - AttributeName: id KeyType: HASH StreamSpecification: StreamViewType: NEW_AND_OLD_IMAGES

Deploy your serverless application using AWS SAM (optional)

If you want to use AWS SAM to deploy a serverless application using the template you just created in Application Composer, you first need to install the AWS SAM CLI. To do this, follow the instructions in Installing the AWS SAM CLI.

Before you deploy your application, you also need to update the function code that Application Composer saved along with your template. At the moment, the lambda_function.py file that Application Composer saved contains only the basic 'Hello world' code that Lambda provided when you created the function.

To update your function code, copy the following code and paste it into the lambda_function.py file Application Composer saved to your local build machine. You specified the directory for Application Composer to save this file to when you activated Local Sync mode.

This code accepts a key value pair in a message from the Amazon SQS queue you created in Application Composer. If both the key and value are strings, the code then uses them to write an item to the DynamoDB table defined in your template.

import boto3 import os import json # define the DynamoDB table that Lambda will connect to tablename = os.environ['LAMBDAIACTABLE_TABLE_NAME'] # create the DynamoDB resource dynamo = boto3.client('dynamodb') def lambda_handler(event, context): # get the message out of the SQS event message = event['Records'][0]['body'] data = json.loads(message) # write event data to DDB table if check_message_format(data): key = next(iter(data)) value = data[key] dynamo.put_item( TableName=tablename, Item={ 'id': {'S': key}, 'Value': {'S': value} } ) else: raise ValueError("Input data not in the correct format") # check that the event object contains a single key value # pair that can be written to the database def check_message_format(message): if len(message) != 1: return False key, value = next(iter(message.items())) if not (isinstance(key, str) and isinstance(value, str)): return False else: return True
To deploy your serverless application

To deploy your application using the AWS SAM CLI, carry out the following steps. For your function to build and deploy correctly, Python version 3.11 must be installed on your build machine and on your PATH.

  1. Run the following command from the directory in which Application Composer saved your template.yaml and lambda_function.py files.

    sam build

    This command gathers the build artifacts for your application and places them in the proper format and location to deploy them.

  2. To deploy your application and create the Lambda, Amazon SQS, and DynamoDB resources specified in your AWS SAM template, run the following command.

    sam deploy --guided

    Using the --guided flag means that AWS SAM will show you prompts to guide you through the deployment process. For this deployment, accept the default options by pressing Enter.

During the deployment process, AWS SAM creates the following resources in your AWS account:

  • An AWS CloudFormation stack named sam-app

  • A Lambda function with the name format sam-app-LambdaIaCDemo-99VXPpYQVv1M

  • An Amazon SQS queue with the name format sam-app-LambdaIaCQueue-xL87VeKsGiIo

  • A DynamoDB table with the name format sam-app-LambdaIaCTable-CN0S66C0VLNV

AWS SAM also creates the necessary IAM roles and policies so that your Lambda function can read messages from the Amazon SQS queue and perform CRUD operations on the DynamoDB table.

To learn more about using AWS SAM to deploy serverless applications, see the resources in the Next steps section.

Testing your deployed application (optional)

To confirm that your serverless application deployed correctly, send a message to your Amazon SQS queue containing a key value pair and check that Lambda writes an item into your DynamoDB table using these values.

To test your serverless application
  1. Open the Queues page of the Amazon SQS console and select the queue that AWS SAM created from your template. The name has the format sam-app-LambdaIaCQueue-xL87VeKsGiIo.

  2. Choose Send and receive messages and paste the following JSON into the Message body in the Send message section.

    { "myKey": "myValue" }
  3. Choose Send message.

    Sending your message to the queue causes Lambda to invoke your function through the event source mapping defined in your AWS SAM template. To confirm that Lambda has invoked your function as expected, confirm that an item has been added to your DynamoDB table.

  4. Open the Tables page of the DynamoDB console and select your table. The name has the format sam-app-LambdaIaCTable-CN0S66C0VLNV.

  5. Choose Explore table items. In the Items returned pane, you should see an item with the id myKey and the Value myValue.

Next steps

To learn more about using Application Composer with AWS SAM and AWS CloudFormation, start with Using Application Composer with AWS CloudFormation and AWS SAM.

For a guided tutorial that uses AWS SAM to deploy a serverless application designed in Application Composer, we also recommend you carry out the AWS Application Composer tutorial in the AWS Serverless Patterns Workshop.

AWS SAM provides a command line interface (CLI) that you can use with AWS SAM templates and supported third-party integrations to build and run your serverless applications. With the AWS SAM CLI, you can build and deploy your application, perform local testing and debugging, configure CI/CD pipelines, and more. To learn more about using the AWS SAM CLI, see Getting started with AWS SAM in the AWS Serverless Application Model Developer Guide.

To learn how to deploy a serverless application with an AWS SAM template using the AWS CloudFormation console, start with Using the AWS CloudFormation console in the AWS CloudFormation User Guide.

Supported regions for Lambda integration with Application Composer

Lambda integration with Application Composer is supported in the following AWS Regions:

  • US East (N. Virginia)

  • US East (Ohio)

  • US West (N. California)

  • US West (Oregon)

  • Africa (Cape Town)

  • Asia Pacific (Hong Kong)

  • Asia Pacific (Hyderabad)

  • Asia Pacific (Jakarta)

  • Asia Pacific (Melbourne)

  • Asia Pacific (Mumbai)

  • Asia Pacific (Osaka)

  • Asia Pacific (Seoul)

  • Asia Pacific (Singapore)

  • Asia Pacific (Sydney)

  • Asia Pacific (Tokyo)

  • Canada (Central)

  • Europe (Frankfurt)

  • Europe (Zurich)

  • Europe (Ireland)

  • Europe (London)

  • Europe (Stockholm)

  • Middle East (UAE)