Prerequisites Create buckets and upload a sample object Create the IAM Policy Create the execution role Create the function Test the Lambda function Configure Amazon S3 to publish events Test the setup Clean up your resources

Tutorial: Using AWS Lambda with Amazon S3

Suppose you want to create a thumbnail for each image file that is uploaded to a bucket. You can create a Lambda function (CreateThumbnail) that Amazon S3 can invoke when objects are created. Then, the Lambda function can read the image object from the source bucket and create a thumbnail image to save in the target bucket.

Upon completing this tutorial, you will have the following Amazon S3, Lambda, and IAM resources in your account:

Lambda resources

A Lambda function.
An access policy associated with your Lambda function that grants Amazon S3 permission to invoke the Lambda function.

IAM resources

An execution role that grants permissions that your Lambda function needs through the permissions policy associated with this role.

Amazon S3 resources

A source bucket with a notification configuration that invokes the Lambda function.
A target bucket where the function saves resized images.

Prerequisites

This tutorial assumes that you have some knowledge of basic Lambda operations and the Lambda console. If you haven't already, follow the instructions in Getting started with Lambda to create your first Lambda function.

To complete the following steps, you need a command line terminal or shell to run commands. Commands are shown in listings preceded by a prompt symbol ($) and the name of the current directory, when appropriate:


~/lambda-project$ this is a command
this is output

For long commands, an escape character (\) is used to split a command over multiple lines.

On Linux and macOS, use your preferred shell and package manager. On Windows 10, you can install the Windows Subsystem for Linux to get a Windows-integrated version of Ubuntu and Bash.

Install npm to manage the function's dependencies.

The tutorial uses AWS CLI commands to create and invoke the Lambda function. Install the AWS CLI and configure it with your AWS credentials

Create buckets and upload a sample object

Follow the steps to create buckets and upload an object.

Open the Amazon S3 console.
Create two buckets. The target bucket name must be source followed by -resized, where source is the name of the bucket you want to use for the source. For example, mybucket and mybucket-resized.
In the source bucket, upload a .jpg object, HappyFace.jpg.

When you invoke the Lambda function manually before you connect to Amazon S3, you pass sample event data to the function that specifies the source bucket and HappyFace.jpg as the newly created object so you need to create this sample object first.

Create the IAM Policy

Create an IAM policy that defines the permissions for the Lambda function. The required permissions include:

Get the object from the source S3 bucket.
Put the resized object into the target S3 bucket.
Permissions related to the CloudWatch Logs.

To create an IAM Policy

Open the Policies page in the IAM Console.
Choose Create policy

Under the JSON tab, copy the following policy. Make sure the source and target bucket names match the bucket names that you created previously.


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:PutLogEvents",
                "logs:CreateLogGroup",
                "logs:CreateLogStream"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::mybucket/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::mybucket-resized/*"
        }
    ]
}

Choose Review policy, specify the policy name as AWSLambdaS3Policy, and create the policy.

Create the execution role

Create the execution role that gives your function permission to access AWS resources.

To create an execution role

Open the roles page in the IAM console.
Choose Create role.
Create a role with the following properties.
- Trusted entity – AWS Lambda.
- Permissions – AWSLambdaS3Policy.
- Role name – lambda-s3-role.

The AWSLambdaS3Policy policy has the permissions that the function needs to manage objects in Amazon S3 and write logs to CloudWatch Logs.

Create the function

The following example code receives an Amazon S3 event input and processes the message that it contains. It resizes an image in the source bucket and saves the output to the target bucket.

Note

For sample code in other languages, see Sample Amazon S3 function code.

Create the function code

Example index.js


// dependencies
const AWS = require('aws-sdk');
const util = require('util');
const sharp = require('sharp');

// get reference to S3 client
const s3 = new AWS.S3();

exports.handler = async (event, context, callback) => {

    // Read options from the event parameter.
    console.log("Reading options from event:\n", util.inspect(event, {depth: 5}));
    const srcBucket = event.Records[0].s3.bucket.name;
    // Object key may have spaces or unicode non-ASCII characters.
    const srcKey    = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));
    const dstBucket = srcBucket + "-resized";
    const dstKey    = "resized-" + srcKey;

    // Infer the image type from the file suffix.
    const typeMatch = srcKey.match(/\.([^.]*)$/);
    if (!typeMatch) {
        console.log("Could not determine the image type.");
        return;
    }

    // Check that the image type is supported  
    const imageType = typeMatch[1].toLowerCase();
    if (imageType != "jpg" && imageType != "png") {
        console.log(`Unsupported image type: ${imageType}`);
        return;
    }

    // Download the image from the S3 source bucket. 

    try {
        const params = {
            Bucket: srcBucket,
            Key: srcKey
        };
        var origimage = await s3.getObject(params).promise();

    } catch (error) {
        console.log(error);
        return;
    }  

    // set thumbnail width. Resize will set the height automatically to maintain aspect ratio.
    const width  = 200;

    // Use the Sharp module to resize the image and save in a buffer.
    try { 
        var buffer = await sharp(origimage.Body).resize(width).toBuffer();
            
    } catch (error) {
        console.log(error);
        return;
    } 

    // Upload the thumbnail image to the destination bucket
    try {
        const destparams = {
            Bucket: dstBucket,
            Key: dstKey,
            Body: buffer,
            ContentType: "image"
        };

        const putResult = await s3.putObject(destparams).promise(); 
        
    } catch (error) {
        console.log(error);
        return;
    } 
        
    console.log('Successfully resized ' + srcBucket + '/' + srcKey +
        ' and uploaded to ' + dstBucket + '/' + dstKey); 
};

Review the preceding code and note the following:

The function knows the source bucket name and the key name of the object from the event data it receives as parameters. If the object is a .jpg or a .png, the code creates a thumbnail and saves it to the target bucket.
The code assumes that the destination bucket exists and its name is a concatenation of the source bucket name followed by the string -resized. For example, if the source bucket identified in the event data is examplebucket, the code assumes you have an examplebucket-resized destination bucket.
For the thumbnail it creates, the code derives its key name as the concatenation of the string resized- followed by the source object key name. For example, if the source object key is sample.jpg, the code creates a thumbnail object that has the key resized-sample.jpg.

Create the deployment package

The deployment package is a .zip file containing your Lambda function code and dependencies.

To create a deployment package

Open a command line terminal or shell in a Linux environment. Ensure that the Node.js version in your local environment matches the Node.js version of your function.
Save the function code as index.js in a folder named lambda-s3.
Install the Sharp library with npm. For Linux, use the following command.
```
lambda-s3$ npm install sharp
```
For macOS, use the following command.
```
lambda-s3$ npm install --arch=x64 --platform=linux --target=12.13.0  sharp
```
After you complete this step, you will have the following folder structure:
```
lambda-s3
|- index.js
|- /node_modules/sharp
└ /node_modules/...
```
Create a deployment package with the function code and dependencies.
```
lambda-s3$ zip -r function.zip .
```

Create the Lambda function

To create the function

Create a Lambda function with the create-function command.
```
$ aws lambda create-function --function-name CreateThumbnail \
--zip-file fileb://function.zip --handler index.handler --runtime nodejs12.x \
--timeout 10 --memory-size 1024 \
--role arn:aws:iam::123456789012:role/lambda-s3-role
```
Note
If you are using AWS CLI version 2, add the following command parameters:
```
--cli-binary-format raw-in-base64-out
```
The Lambda function in the last step uses a Node.js function handler of index.handler. This name reflects the function name as handler, and the file where the handler code is stored in index.js. For more information, see AWS Lambda function handler in Node.js. It also specifies a runtime of nodejs12.x. For more information, see Lambda runtimes.

For the role parameter, replace the number sequence with your AWS account ID. The preceding example command specifies a 10-second timeout value as the function configuration. Depending on the size of objects you upload, you might need to increase the timeout value using the following AWS CLI command.


$ aws lambda update-function-configuration --function-name CreateThumbnail --timeout 30

Test the Lambda function

In this step, you invoke the Lambda function manually using sample Amazon S3 event data.

To test the Lambda function

Save the following Amazon S3 sample event data in a file and save it as inputFile.txt. You need to update the JSON by providing your sourcebucket name and a .jpg object key.


{
  "Records":[
    {
      "eventVersion":"2.0",
      "eventSource":"aws:s3",
      "awsRegion":"us-west-2",
      "eventTime":"1970-01-01T00:00:00.000Z",
      "eventName":"ObjectCreated:Put",
      "userIdentity":{
        "principalId":"AIDAJDPLRKLG7UEXAMPLE"
      },
      "requestParameters":{
        "sourceIPAddress":"127.0.0.1"
      },
      "responseElements":{
        "x-amz-request-id":"C3D13FE58DE4C810",
        "x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
      },
      "s3":{
        "s3SchemaVersion":"1.0",
        "configurationId":"testConfigRule",
        "bucket":{
          "name":"sourcebucket",
          "ownerIdentity":{
            "principalId":"A3NL1KOZZKExample"
          },
          "arn":"arn:aws:s3:::sourcebucket"
        },
        "object":{
          "key":"HappyFace.jpg",
          "size":1024,
          "eTag":"d41d8cd98f00b204e9800998ecf8427e",
          "versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
        }
      }
    }
  ]
}

Run the following Lambda CLI invoke command to invoke the function. Note that the command requests asynchronous execution. You can optionally invoke it synchronously by specifying RequestResponse as the invocation-type parameter value.
```
$ aws lambda invoke --function-name CreateThumbnail --invocation-type Event \
--payload file://inputFile.txt outputfile.txt
```
Note
If you are using AWS CLI version 2, add the following command parameters:
```
--cli-binary-format raw-in-base64-out
```
Verify that the thumbnail was created in the target bucket.

Configure Amazon S3 to publish events

In this step, you add the remaining configuration so that Amazon S3 can publish object-created events to AWS Lambda and invoke your Lambda function. You do the following in this step:

Add permissions to the Lambda function access policy to allow Amazon S3 to invoke the function.
Add notification configuration to your source bucket. In the notification configuration, you provide the following:
- Event type for which you want Amazon S3 to publish events. For this tutorial, you specify the s3:ObjectCreated:* event type so that Amazon S3 publishes events when objects are created.
- Lambda function to invoke.

To add permissions to the function policy

Run the following Lambda CLI add-permission command to grant Amazon S3 service principal (s3.amazonaws.com) permissions to perform the lambda:InvokeFunction action. Note that permission is granted to Amazon S3 to invoke the function only if the following conditions are met:
- An object-created event is detected on a specific bucket.
- The bucket is owned by your account. If you delete a bucket, it is possible for another account to create a bucket with the same ARN.
```
$ aws lambda add-permission --function-name CreateThumbnail --principal s3.amazonaws.com \
--statement-id s3invoke --action "lambda:InvokeFunction" \
--source-arn arn:aws:s3:::sourcebucket \
--source-account account-id
```
Verify the function's access policy by running the AWS CLI get-policy command.
```
$ aws lambda get-policy --function-name CreateThumbnail
```

Add notification configuration on the source bucket to request Amazon S3 to publish object-created events to Lambda.

Important

This procedure configures the bucket to invoke your function every time an object is created in it. Ensure that you configure this option only on the source bucket and do not create objects in the source bucket from the function that is triggered. Otherwise, your function could cause itself to be invoked continuously in a loop.

To configure notifications

Open the Amazon S3 console.
Choose the source bucket.
Choose Properties.
Under Events, configure a notification with the following settings.
- Name – lambda-trigger.
- Events – All object create events.
- Send to – Lambda function.
- Lambda – CreateThumbnail.

For more information on event configuration, see Enabling event notifications in the Amazon Simple Storage Service Console User Guide.

Test the setup

Now you can test the setup as follows:

Upload .jpg or .png objects to the source bucket using the Amazon S3 console.
Verify that the thumbnail was created in the target bucket using the CreateThumbnail function.
View logs in the CloudWatch console.

Clean up your resources

You can now delete the resources that you created for this tutorial, unless you want to retain them. By deleting AWS resources that you are no longer using, you prevent unnecessary charges to your AWS account.

To delete the Lambda function

Open the Functions page of the Lambda console.
Select the function that you created.
Choose Actions, Delete.
Choose Delete.

To delete the policy that you created

Open the Policies page of the IAM console.
Select the policy that you created.
From Policy actions, choose Delete.
Choose Delete.

To delete the execution role

Open the Roles page of the IAM console.
Select the execution role that you created.
Choose Delete role.
Choose Yes, delete.

To delete the S3 buckets

Open the Amazon S3 console.
Select the source bucket you created.
Choose Delete.
Enter the name of the source bucket in the text box.
Choose Confirm.

Warning Javascript is disabled or is unavailable in your browser.

To use the AWS Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Sample code