Core Components: AWS Lambda Function and Event Sources
A Lambda function and an event source are the core components that you use when you work with AWS Lambda. Event sources publish events, and a Lambda function is the custom code that you write to process the events that automatically gets invoked in response to events. In addition, you can invoke your Lambda function directly over HTTPS or using AWS SDKs.
After you upload your custom code to AWS Lambda, we refer to it as a Lambda function. A Lambda function consists of code you provide, associated dependencies, and configuration.
The configuration information you provide includes the compute resources you want to allocate (for example, memory), execution timeout, and an IAM role that AWS Lambda can assume to execute your Lambda function on your behalf. You use IAM roles to grant AWS Lambda the necessary permissions to execute the code in your Lambda function successfully.
As part of the configuration, you also specify a handler (that is, a method/function in your code) where AWS Lambda can begin executing your code. AWS Lambda provides event data as input to this handler, which processes the event.
Currently, you can author your Lambda function code in Java, Node.js, or Python 2.7.
Compute Resources for Your Lambda Function
At the time you create your Lambda function, you only choose the amount of memory you want to allocate for your function. Then, AWS Lambda allocates CPU power proportional to the memory by using the same ratio as a general purpose Amazon EC2 instance type, such as an M3 type. For example, if you allocate 256 MB to your Lambda function, it will receive twice the CPU share than if you allocated 128 MB.
You can request additional memory in 64 MB increments up to 1536 MB. For a full list of AWS Lambda limits, see AWS Lambda Limits.
Automatic Lambda Function Invocation: Using Event Sources
Event sources publish events that cause the Lambda function to be invoked. Upon invocation, AWS Lambda executes your code by passing the event to the handler in your Lambda function code. You associate an event source with your Lambda function using an event source mapping. How you implement the event source mapping depends on the event source, as discussed in the following section.
AWS Lambda supports pull and push models for function invocation, which directs AWS Lambda to execute the function. For details about the pull and push models, see The Pull and Push Event Models: How a Lambda Function Is Invoked.
We can categorize event sources as follows:
AWS services (for example, Amazon S3)
Amazon services (for example, Amazon Echo)
AWS Services as Event Sources
Currently, AWS Lambda supports events from the services listed in the following table. To see an example of an event source for each supported service, go to AWS Lambda Event Sources .
You can configure notification on an Amazon S3 bucket to publish bucket events, such as when objects are created or deleted, to AWS Lambda and invoke a Lambda function to respond to these events. This allows you to process changes in your Amazon S3 data immediately.
When you create or delete an object from the bucket, Amazon S3 detects the event and publishes it to AWS Lambda and invokes your Lambda function. Then, AWS Lambda executes the function by passing the incoming event information (such as bucket name or object key) to your Lambda function handler. For more information, see Lambda Function. Lastly, your Lambda function processes the event.
Amazon DynamoDB Streams enables you to track item updates in a table. DynamoDB publishes any item updates to the stream associated with the table. You can create an event source mapping in AWS Lambda that identifies a DynamoDB stream as an event source for your Lambda function. Then, AWS Lambda polls the stream and invokes the specified function when records are posted to the stream. For an example DynamoDB event, see Step 2.3.2: Test the Lambda Function (Invoke Manually). For general format, see GetRecord in the Amazon DynamoDB API Reference.
You can create an Amazon Kinesis stream to continuously capture and store terabytes of data per hour from hundreds of thousands of sources such as website click streams, financial transactions, social media feeds, IT logs, and location-tracking events. For more information, see Amazon Kinesis.
You can create an event source mapping in AWS Lambda that identifies an Amazon Kinesis stream as an event source for your Lambda function. AWS Lambda then polls the Amazon Kinesis stream and invokes the specified function when records are posted to the stream. For an example event, see Step 2.3: Create the Lambda Function and Test It Manually.
|Amazon Simple Notification Service||
Amazon Simple Notification Service (Amazon SNS) is a fast, flexible, fully managed push notification service that lets you send individual messages or to fan-out messages to large numbers of recipients. For more information about Amazon SNS, see Amazon Simple Notification Service.
When you push a new message to an Amazon SNS topic, it can trigger a Lambda function. The Lambda function receives the event information including the Amazon SNS topic, the time the message was pushed, as well as the message itself. Your Lambda function can process the event.
|Amazon Simple Email Service||
Amazon Simple Email Service (Amazon SES) is a cost-effective email service. With Amazon SES, in addition to sending emails, you can also use the service to receive messages and call your custom code via an AWS Lambda function. For more information about Amazon SES, see Amazon Simple Email Service.
When you use Amazon SES to receive messages, you can configure Amazon SES to call your Lambda function when messages arrive. The Lambda function receives the incoming email event information as the event parameter. Your Lambda code can process the event. For information about the event format, see Lambda Action in the Amazon Simple Email Service Developer Guide.
The Amazon Cognito Events feature enables you to run Lambda function in response to events in Amazon Cognito. For more information, see Amazon Cognito.
You can now invoke actions with changes in user data, such as app preferences or game state – and then validate, audit, or modify that data inside your Lambda function. To learn more, see "Introducing Amazon Cognito Events: Sync Triggers" in the Mobile Development blog. For information about event structure, see Amazon Cognito Events in the Amazon Cognito Developer Guide (see the "Writing Lambda Function for Sync Triggers" section).
Amazon CloudWatch Logs
You can use AWS Lambda functions to perform custom analysis on Amazon CloudWatch Logs using CloudWatch Logs subscriptions. CloudWatch Logs subscriptions provide access to a real-time feed of log events from CloudWatch Logs and deliver it to your AWS Lambda function for custom processing, analysis, or loading to other systems. You can also define the filter pattern to use for filtering which log events get delivered to your Lambda function.
As part of deploying AWS CloudFormation stacks you can specify a Lambda function as a custom resource to execute any custom commands as part of creation of your stack.
Amazon CloudWatch Events
Amazon CloudWatch Events help you to respond to state changes in your AWS resources. When your resources change state, they automatically send events into an event stream. You can create rules that match selected events in the stream and route them to your AWS Lambda function to take action. For example, you can automatically invoke an AWS Lambda function to log the state of an EC2 instance or AutoScaling Group.
Scheduled Events (powered by Amazon CloudWatch Events)
You can also set up AWS Lambda to invoke your code on a regular, scheduled basis using the schedule event capability in Amazon CloudWatch Events. You can specify a fixed rate (number of hours, days, or weeks) or you can specify a cron expression. You schedule it using a CloudWatch Events rule that you can create or modify using the AWS Lambda console, the Amazon CloudWatch Events console, or the Amazon CloudWatch Events CLI (see Schedule Expression Syntax for Rules in the Amazon CloudWatch Developer Guide).
You can use AWS Lambda functions to evaluate whether your AWS resource configurations comply with your custom Config rules. As resources are created, deleted, or changed, AWS Config records these changes and sends the information to your Lambda functions. Your Lambda functions then evaluate the changes and report results to AWS Config. You can then use AWS Config to assess overall resource compliance: you can learn which resources are noncompliant and which configuration attributes are the cause of noncompliance.
For more information, see Evaluating Resources With AWS Config Rules. For example functions, see Example AWS Lambda Functions for AWS Config Rules (Node.js)
In addition, you can also use Lambda functions with other AWS services that publish data to one of the event sources listed in the preceding table. For example:
You can trigger Lambda functions in response to CloudTrail updates because it records all API access events to an Amazon S3 bucket.
You can trigger Lambda functions in response to CloudWatch alarms because it publishes alarm events to an Amazon SNS topic.
Each of these event sources have a pre-defined event data structure. For example, an Amazon S3 event will have a finite structure providing information such as bucket name and object key. For more information, see Event Message Structure in the Amazon Simple Storage Service Developer Guide. The pre-defined event data structure enables you to write your Lambda function code to read the event and process it.
You can also set up AWS Lambda to invoke your code on a regular, scheduled basis using the AWS Lambda console. You can specify a fixed rate (number of hours, days, or weeks) or you can specify a cron expression.
Currently this feature is available only in the AWS Lambda console.
The AWS Lambda console provides a sample blueprint (lambda-canary) with example code in Python that you can use to create a Lambda function and test the scheduled events feature. For step-by-step instructions, see Using AWS Lambda with Scheduled Events.
Other Amazon Services as Event Sources
Currently, you can use Lambda functions to build services that give new skills to Alexa, the Voice assistant on Amazon Echo. The Alexa Skills Kit provides the APIs, tools, and documentation to create these new skills, powered by your own services running as Lambda functions. Amazon Echo users can access these new skills by asking Alexa questions or making requests. For more information, see Getting Started with Alexa Skills Kit.
On-Demand Lambda Function Invocation: Over HTTPS
In addition to invoking Lambda functions using event
sources, you can
also invoke your Lambda function over HTTPS. You can do this by defining a custom
REST API and endpoint using Amazon API Gateway. You map individual API methods, such
PUT, to specific Lambda functions. When you send an HTTPS request to the
API endpoint, the Amazon API Gateway service invokes the corresponding Lambda
For more information, see http://aws.amazon.com/api-gateway/.
On-Demand Lambda Function Invocation: Build Your Own Event Sources
In addition to AWS services generating events, user applications can also generate events. User applications such as client, mobile, or web applications can publish events and invoke Lambda functions using the AWS SDKs or AWS Mobile SDKs such as the AWS Mobile SDK for Android.
For more information, see Tools for Amazon Web Services. For an example tutorial, see Using AWS Lambda with Amazon API Gateway (On-Demand Over HTTPS).
At the time of invoking a Lambda function, one of the following invocation types can be specified:
Event – This invocation type causes AWS Lambda to execute the Lambda function asynchronously. The event sources Amazon S3, Amazon SNS or Amazon DynamoDB use this invocation type. Your custom applications can also invoke your Lambda function using this invocation type.
Because of the asynchronous nature of execution your Lambda function does not need to send any response.
RequestResponse – This invocation type causes AWS Lambda to execute the function synchronously and returns the response immediately to the calling application. This invocation type is available for custom applications.
When you invoke a Lambda function via the AWS console or over HTTPS using Amazon API Gateway, Lambda always use the
For more information about the
Invoke API, see Invoke.
Concurrent executions refers to the number of executions of your function code that are happening at any given time. Depending on how your function is invoked, your concurrent execution count can be calculated in two ways:
For Lambda functions reading off Amazon Kinesis and DynamoDB streams, Lambda executes your function concurrently for each shard on the stream. For example, if your Amazon Kinesis stream has 100 shards, your function will have at most 100 concurrent executions at any given time. The function reads serially off each active shard multiple times per second, invoking your function with as many records as are available, up to the batch size you have configured as part of your event source definition. For example, suppose you have a Amazon Kinesis stream with 10 active shards. In this scenario, your Lambda function will have 10 concurrent executions. For more information about the pull model, see The Pull Event Model.
For all other event sources and invocation mechanisms, Lambda executes your function concurrently for each request or event. You can estimate your concurrent requests using this formula: requests per second * function duration. For example, if your function is subscribed to an Amazon S3 bucket with 10 writes per second, and each function execution takes 3 seconds on average, your function has 10 * 3 = 30 concurrent executions.
Concurrent executions is different from your function request rate. In the case of stream event sources, your request rate will be multiple times the number of active shards on the source stream. For all others, it’s the rate at which events are generated by the event source, such as Amazon S3 or Amazon SNS.
Concurrent Execution Safety Limit
To limit the impact of runaway or recursive functions during initial development and testing,
Lambda has a safety limit on concurrent requests per account. The limit is applied to the total concurrent executions
across all functions within a given region. To see the default concurrent execution limit for an account and how to increase it,
see AWS Lambda Limits. Any invocation that causes your function's concurrent
execution to exceed the safety limit gets throttled, and does not execute your function. Each throttled invocation
will increase the CloudWatch
Throttles metric for the function.
The throttled invocation is handled differently based on how the function is invoked:
If the function is invoked synchronously and gets throttled, the invoking application will see a 429 error, and will be responsible for retries. If you are invoking the function directly, such as through the AWS SDKs using the
RequestResponseinvocation mode or through API Gateway, your client will see the 429 error and can choose to retry. Alexa, API Gateway, CloudWatch Logs, AWS CloudFormation, and Amazon Cognito invoke Lambda functions synchronously, and may have additional retries built into the integration. For example, CloudWatch Logs retries the failed batch up to five times with delays between retries.
If you are invoking Lambda through API Gateway, you need to make sure you map Lambda response errors to API Gateway error codes.
If your Lambda function is invoked asynchronously and gets throttled, Lambda will make a best effort to invoke your function at least once by automatically retrying the throttled event up to six hours, with delays between retries. Amazon S3, Amazon SNS, AWS IoT, scheduled tasks, and Amazon SES invoke Lambda asynchronously and, therefore, their throttled events will be retried. Asynchronous events are queued before being used to invoke the Lambda function.
If the Lambda function is subscribed to an Amazon Kinesis or DynamoDB stream and gets throttled, Lambda will attempt to process the throttled batch of records until the time the data expires, which can be up to seven days for Amazon Kinesis. The throttled request is treated as blocking and Lambda will not read any new records from the stream until the throttled batch of records either expires or succeeds. This way Lambda maintains the order in which events are processed from the stream.
Retries on Errors
Your Lambda function consists of custom code, which could fail for the following reasons:
The function times out while trying to reach an endpoint.
The function fails to successfully parse input data.
The function experiences resource constraints, such as out-of-memory errors or other timeouts.
If any of these failures occur, your function will throw an exception. The following summary explains how the exception is handled, depending on how your function was invoked:
If the function is invoked synchronously and throws an exception, the invoking application will see a 400 error, and will be responsible for retries. If you are invoking the function directly, such as through the AWS SDKs using the
RequestResponseinvocation mode, or through API Gateway, your client will see the error and can choose to retry. Alexa, API Gateway, CloudWatch Logs, AWS CloudFormation and Amazon Cognito invoke Lambda functions synchronously, and may have additional retries built into the integration. For example, CloudWatch Logs logs retries the failed batch up to 5 times with delays between retries.
If you are invoking Lambda through API Gateway, you need to make sure you map Lambda response errors to API gateway error codes.
If the Lambda function is invoked asynchronously and throws an exception, Lambda will automatically retry the failed event up to three times with delays between retries. Amazon S3, Amazon SNS, IoT, Scheduled tasks and Amazon SES invoke Lambda asynchronously, and therefore will see their events retried.
If the Lambda function is subscribed to an Amazon Kinesis or DynamoDB stream and throws an exception, Lambda will attempt to process the erring batch of records until the time the data expires, which can be up to 7 days for Amazon Kinesis. For Amazon Kinesis, Lambda will back off for up to one minute between retries until the data expires. The exception is treated as blocking; Lambda will not read any new records from the stream until the failed batch of records either expires or succeeds. This way Lambda maintains the order in which events are processed from the stream.