Troubleshooting payloads - AWS Lambda

Troubleshooting payloads

Once an error has occurred and you have narrowed down the problem to a specific Lambda function, there are a number of likely causes. This section explains the most common causes, their symptoms, and corrective actions that you can take.

Unexpected event payloads

All Lambda functions receive an event payload in the first parameter of the handler. In many functions, this is the single greatest variable in the operation of the code. The event payload is a JSON structure that may contain arrays and nested elements.

Malformed JSON can occur when provided by upstream services that do not use a robust process for checking their JSON structures. This occurs when services concatenate text strings or embed user input that has not been sanitized. JSON is also frequently serialized for passing between services. Always parse JSON structures both as the producer and consumer of JSON to ensure that the structure is valid.

Similarly, failing to check for ranges of values in the event payload can result in errors. This example shows a function that calculates a tax withholding:

exports.handler = async (event) => {
    let pct = event.taxPct
    let salary = event.salary

    // Calculate % of paycheck for taxes
    return (salary * pct)

This function uses a salary and tax rate from the event payload to perform the calculation. However, the code fails to check if the attributes are present. It also fails to check data types, or ensure boundaries, such as ensuring that the tax percentage is between 0 and 1. As a result, values outside of these bounds produce nonsensical results. An incorrect type or missing attribute causes a runtime error.

Create tests to ensure that your function handles larger payload sizes. The maximum size for a Lambda event payload is 256 KB. Depending upon the content, larger payloads may mean more items passed to the function or more binary data embedded in a JSON attribute. In both cases, this can result in more processing for a Lambda function.

Larger payloads can also cause timeouts. For example, a Lambda function processes one record per 100 ms and has a timeout of 3 seconds. Processing is successful for 0-29 items in the payload. However, once the payload contains more than 30 items, the function times out and throws an error. To avoid this, ensure that timeouts are set to handle the additional processing time for the maximum number of items expected.

Unexpectedly large payload sizes

Many event payloads contain pointers to other resources. With S3 events, while the event payload is a JSON object, the S3 object that caused the event may be up to 5 terabytes in size. If a function performs processing on a referenced data item such as an S3 object, it should first check the size of the data.

For example, a Lambda function with 128 MB of memory may convert a JPG file stored as an object in S3, by using an image-processing library. The function works as expected with smaller image files. However, when a larger JPG file is provided as input, the Lambda function throws an error due to running out of memory. To avoid this, the test cases should include examples from the upper bounds of expected data sizes and the code should also validate payload sizes.

Incorrectly processing payload parameters

While JSON is a flexible notation, services that generate JSON may use fields in specific ways. For AWS services that generate events, the message structure is documented to show the format of data attributes and whether the attribute is required. Check the documentation from the producing service to ensure that you are processing the JSON attributes correctly. If you are producing JSON, applying a style guide and creating documentation can help developers who are consuming your events.

For example, for events generated by S3, the s3.object.key attribute contains a URL encoded object key name. Many functions process this attribute as text to load the referenced S3 object:

const originalText = await s3.getObject({ Bucket: event.Records[0], Key: event.Records[0].s3.object.key }).promise()

This code works with the key name james.jpg but throws a NoSuchKey error when the name is james beswick.jpg. Since URL encoding converts spaces and other characters in a key name, you must ensure that functions decode keys before using this data:

const originalText = await s3.getObject({ Bucket: event.Records[0], Key: decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " ")) }).promise()