Configuring advanced logging controls for Lambda functions - AWS Lambda
Configuring JSON and plain text log formatsLog-level filteringConfiguring CloudWatch log groups

Configuring advanced logging controls for Lambda functions

To give you more control over how your functions’ logs are captured, processed, and consumed, Lambda offers the following logging configuration options:

  • Log format - select between plain text and structured JSON format for your function’s logs

  • Log level - for JSON structured logs, choose the detail level of the logs Lambda sends to CloudWatch, such as ERROR, DEBUG, or INFO

  • Log group - choose the CloudWatch log group your function sends logs to

To learn more about configuring advanced logging controls, refer to the following sections:

Configuring JSON and plain text log formats

Capturing your log outputs as JSON key value pairs makes it easier to search and filter when debugging your functions. With JSON formatted logs, you can also add tags and contextual information to your logs. This can help you to perform automated analysis of large volumes of log data. Unless your development workflow relies on existing tooling that consumes Lambda logs in plain text, we recommend that you select JSON for your log format.

For all Lambda managed runtimes, you can choose whether your function's system logs are sent to CloudWatch Logs in unstructured plain text or JSON format. System logs are the logs that Lambda generates and are sometimes known as platform event logs.

For supported runtimes, when you use one of the supported built-in logging methods, Lambda can also output your function's application logs (the logs your function code generates) in structured JSON format. When you configure your function's log format for these runtimes, the configuration you choose applies to both system and application logs.

For supported runtimes, if your function uses a supported logging library or method, you don't need to make any changes to your existing code for Lambda to capture logs in structured JSON.

Note

Using JSON log formatting adds additional metadata and encodes log messages as JSON objects containing a series of key value pairs. Because of this, the size of your function's log messages can increase.

Supported runtimes and logging methods

Lambda currently supports the option to output JSON structured application logs for the following runtimes.

Runtime Supported versions
Java All Java runtimes except Java 8 on Amazon Linux 1
.NET .NET 6 and .NET 8
Node.js Node.js 16 and later
Python Python 3.8 and later

For Lambda to send your function's application logs to CloudWatch in structured JSON format, your function must use the following built-in logging tools to output logs:

  • Java - the LambdaLogger logger or Log4j2.

  • .NET - the ILambdaLogger instance on the context object.

  • Node.js - The console methods console.trace, console.debug, console.log, console.info, console.error, and console.warn

  • Python - the standard Python logging library

For more information about using advanced logging controls with supported runtimes, see Log and monitor Java Lambda functions, Log and monitor Node.js Lambda functions, and Log and monitor Python Lambda functions.

For other managed Lambda runtimes, Lambda currently only natively supports capturing system logs in structured JSON format. However, you can still capture application logs in structured JSON format in any runtime by using logging tools such as Powertools for AWS Lambda that output JSON formatted log outputs.

Default log formats

Currently, the default log format for all Lambda runtimes is plain text.

If you’re already using logging libraries like Powertools for AWS Lambda to generate your function logs in JSON structured format, you don’t need to change your code if you select JSON log formatting. Lambda doesn’t double-encode any logs that are already JSON encoded, so your function’s application logs will continue to be captured as before.

JSON format for system logs

When you configure your function's log format as JSON, each system log item (platform event) is captured as a JSON object that contains key value pairs with the following keys:

  • "time" - the time the log message was generated

  • "type" - the type of event being logged

  • "record" - the contents of the log output

The format of the "record" value varies according to the type of event being logged. For more information see Telemetry API Event object types. For more information about the log levels assigned to system log events, see System log level event mapping.

For comparison, the following two examples show the same log output in both plain text and structured JSON formats. Note that in most cases, system log events contain more information when output in JSON format than when output in plain text.

Example plain text:
2024-03-13 18:56:24.046000 fbe8c1   INIT_START  Runtime Version: python:3.12.v18  Runtime Version ARN: arn:aws:lambda:eu-west-1::runtime:edb5a058bfa782cb9cedc6d534ac8b8c193bc28e9a9879d9f5ebaaf619cd0fc0
Example structured JSON:
{
  "time": "2024-03-13T18:56:24.046Z",
  "type": "platform.initStart",
  "record": {
    "initializationType": "on-demand",
    "phase": "init",
    "runtimeVersion": "python:3.12.v18",
    "runtimeVersionArn": "arn:aws:lambda:eu-west-1::runtime:edb5a058bfa782cb9cedc6d534ac8b8c193bc28e9a9879d9f5ebaaf619cd0fc0"
  }
}
Note

The Accessing real-time telemetry data for extensions using the Telemetry API always emits platform events such as START and REPORT in JSON format. Configuring the format of the system logs Lambda sends to CloudWatch doesn’t affect Lambda Telemetry API behavior.

JSON format for application logs

When you configure your function's log format as JSON, application log outputs written using supported logging libraries and methods are captured as a JSON object that contains key value pairs with the following keys.

  • "timestamp" - the time the log message was generated

  • "level" - the log level assigned to the message

  • "message" - the contents of the log message

  • "requestId" (Python, .NET, and Node.js) or "AWSrequestId" (Java) - the unique request ID for the function invocation

Depending on the runtime and logging method that your function uses, this JSON object may also contain additional key pairs. For example, in Node.js, if your function uses console methods to log error objects using multiple arguments, The JSON object will contain extra key value pairs with the keys errorMessage, errorType, and stackTrace. To learn more about JSON formatted logs in different Lambda runtimes, see Log and monitor Python Lambda functions, Log and monitor Node.js Lambda functions, and Log and monitor Java Lambda functions.

Note

The key Lambda uses for the timestamp value is different for system logs and application logs. For system logs, Lambda uses the key "time" to maintain consistency with Telemetry API. For application logs, Lambda follows the conventions of the supported runtimes and uses "timestamp".

For comparison, the following two examples show the same log output in both plain text and structured JSON formats.

Example plain text:
2024-10-27T19:17:45.586Z 79b4f56e-95b1-4643-9700-2807f4e68189 INFO some log message
Example structured JSON:
{
    "timestamp":"2024-10-27T19:17:45.586Z",
    "level":"INFO",
    "message":"some log message",
    "requestId":"79b4f56e-95b1-4643-9700-2807f4e68189"
}

Setting your function's log format

To configure the log format for your function, you can use the Lambda console or the AWS Command Line Interface (AWS CLI). You can also configure a function’s log format using the CreateFunction and UpdateFunctionConfiguration Lambda API commands, the AWS Serverless Application Model (AWS SAM) AWS::Serverless::Function resource, and the AWS CloudFormation AWS::Lambda::Function resource.

Changing your function’s log format doesn’t affect existing logs stored in CloudWatch Logs. Only new logs will use the updated format.

If you change your function's log format to JSON and do not set log level, then Lambda automatically sets your function's application log level and system log level to INFO. This means that Lambda sends only log outputs of level INFO and lower to CloudWatch Logs. To learn more about application and system log-level filtering see Log-level filtering

Note

For Python runtimes, when your function's log format is set to plain text, the default log-level setting is WARN. This means that Lambda only sends log outputs of level WARN and lower to CloudWatch Logs. Changing your function's log format to JSON changes this default behavior. To learn more about logging in Python, see Log and monitor Python Lambda functions.

For Node.js functions that emit embedded metric format (EMF) logs, changing your function's log format to JSON could result in CloudWatch being unable to recognize your metrics.

Important

If your function uses Powertools for AWS Lambda (TypeScript) or the open-sourced EMF client libraries to emit EMF logs, update your Powertools and EMF libraries to the latest versions to ensure that CloudWatch can continue to parse your logs correctly. If you switch to the JSON log format, we also recommend that you carry out testing to ensure compatibility with your function's embedded metrics. For further advice about node.js functions that emit EMF logs, see Using embedded metric format (EMF) client libraries with structured JSON logs.

To configure a function’s log format (console)

  1. Open the Functions page of the Lambda console.

  2. Choose a function

  3. On the function configuration page, choose Monitoring and operations tools.

  4. In the Logging configuration pane, choose Edit.

  5. Under Log content, for Log format select either Text or JSON.

  6. Choose Save.

To change the log format of an existing function (AWS CLI)

  • To change the log format of an existing function, use the update-function-configuration command. Set the LogFormat option in LoggingConfig to either JSON or Text.

    aws lambda update-function-configuration \
  --function-name myFunction \
  --logging-config LogFormat=JSON
To set log format when you create a function (AWS CLI)

  • To configure log format when you create a new function, use the --logging-config option in the create-function command. Set LogFormat to either JSON or Text. The following example command creates a Node.js function that outputs logs in structured JSON.

    If you don’t specify a log format when you create a function, Lambda will use the default log format for the runtime version you select. For information about default logging formats, see Default log formats.

    aws lambda create-function \ 
  --function-name myFunction \ 
  --runtime nodejs20.x \
  --handler index.handler \
  --zip-file fileb://function.zip \
  --role arn:aws:iam::123456789012:role/LambdaRole \
  --logging-config LogFormat=JSON

Log-level filtering

Lambda can filter your function's logs so that only logs of a certain detail level or lower are sent to CloudWatch Logs. You can configure log-level filtering separately for your function's system logs (the logs that Lambda generates) and application logs (the logs that your function code generates).

For Supported runtimes and logging methods, you don't need to make any changes to your function code for Lambda to filter your function's application logs.

For all other runtimes and logging methods , your function code must output log events to stdout or stderr as JSON formatted objects that contain a key value pair with the key "level". For example, Lambda interprets the following output to stdout as a DEBUG level log.

print('{"level": "debug", "msg": "my debug log", "timestamp": "2024-11-02T16:51:31.587199Z"}')

If the "level" value field is invalid or missing, Lambda will assign the log output the level INFO. For Lambda to use the timestamp field, you must specify the time in valid RFC 3339 timestamp format. If you don't supply a valid timestamp, Lambda will assign the log the level INFO and add a timestamp for you.

When naming the timestamp key, follow the conventions of the runtime you are using. Lambda supports most common naming conventions used by the managed runtimes.

Note

To use log-level filtering, your function must be configured to use the JSON log format. The default log format for all Lambda managed runtimes is currently plain text. To learn how to configure your function's log format to JSON, see Setting your function's log format.

For application logs (the logs generated by your function code), you can choose between the following log levels.

Log level Standard usage
TRACE (most detail) The most fine-grained information used to trace the path of your code's execution
DEBUG Detailed information for system debugging
INFO Messages that record the normal operation of your function
WARN Messages about potential errors that may lead to unexpected behavior if unaddressed
ERROR Messages about problems that prevent the code from performing as expected
FATAL (least detail) Messages about serious errors that cause the application to stop functioning

When you select a log level, Lambda sends logs at that level and lower to CloudWatch Logs. For example, if you set a function’s application log level to WARN, Lambda doesn’t send log outputs at the INFO and DEBUG levels. The default application log level for log filtering is INFO.

When Lambda filters your function’s application logs, log messages with no level will be assigned the log level INFO.

For system logs (the logs generated by the Lambda service), you can choose between the following log levels.

Log level Usage
DEBUG (most detail) Detailed information for system debugging
INFO Messages that record the normal operation of your function
WARN (least detail) Messages about potential errors that may lead to unexpected behavior if unaddressed

When you select a log level, Lambda sends logs at that level and lower. For example, if you set a function’s system log level to INFO, Lambda doesn’t send log outputs at the DEBUG level.

By default, Lambda sets the system log level to INFO. With this setting, Lambda automatically sends "start" and "report" log messages to CloudWatch. To receive more or less detailed system logs, change the log level to DEBUG or WARN. To see a list of the log levels that Lambda maps different system log events to, see System log level event mapping.

Configuring log-level filtering

To configure application and system log-level filtering for your function, you can use the Lambda console or the AWS Command Line Interface (AWS CLI). You can also configure a function’s log level using the CreateFunction and UpdateFunctionConfiguration Lambda API commands, the AWS Serverless Application Model (AWS SAM) AWS::Serverless::Function resource, and the AWS CloudFormation AWS::Lambda::Function resource.

Note that if you set your function's log level in your code, this setting takes precedence over any other log level settings you configure. For example, if you use the Python logging setLevel() method to set your function's logging level to INFO, this setting takes precedence over a setting of WARN that you configure using the Lambda console.

To configure an existing function’s application or system log level (console)

  1. Open the Functions page of the Lambda console.

  2. Choose a function.

  3. On the function configuration page, choose Monitoring and operations tools.

  4. In the Logging configuration pane, choose Edit.

  5. Under Log content, for Log format ensure JSON is selected.

  6. Using the radio buttons, select your desired Application log level and System log level for your function.

  7. Choose Save.

To configure an existing function’s application or system log level (AWS CLI)

  • To change the application or system log level of an existing function, use the update-function-configuration command. Use --logging-config to set SystemLogLevel to one of DEBUG, INFO, or WARN. Set ApplicationLogLevel to one of DEBUG, INFO, WARN, ERROR, or FATAL. 

    aws lambda update-function-configuration \
  --function-name myFunction \
  --logging-config LogFormat=JSON,ApplicationLogLevel=ERROR,SystemLogLevel=WARN
To configure log-level filtering when you create a function

  • To configure log-level filtering when you create a new function, use --logging-config to set the SystemLogLevel and ApplicationLogLevel keys in the create-function command. Set SystemLogLevel to one of DEBUG, INFO, or WARN. Set ApplicationLogLevel to one of DEBUG, INFO, WARN, ERROR, or FATAL.

    aws lambda create-function \
  --function-name myFunction \
  --runtime nodejs20.x \
  --handler index.handler \
  --zip-file fileb://function.zip \
  --role arn:aws:iam::123456789012:role/LambdaRole \ 
  --logging-config LogFormat=JSON,ApplicationLogLevel=ERROR,SystemLogLevel=WARN

System log level event mapping

For system level log events generated by Lambda, the following table defines the log level assigned to each event. To learn more about the events listed in the table, see Lambda Telemetry API Event schema reference

Event name Condition Assigned log level
initStart runtimeVersion is set INFO
initStart runtimeVersion is not set DEBUG
initRuntimeDone status=success DEBUG
initRuntimeDone status!=success WARN
initReport initializationType=snapstart INFO
initReport initializationType!=snapstart DEBUG
initReport status!=success WARN
restoreStart runtimeVersion is set INFO
restoreStart runtimeVersion is not set DEBUG
restoreRuntimeDone status=success DEBUG
restoreRuntimeDone status!=success WARN
restoreReport status=success INFO
restoreReport status!=success WARN
start - INFO
runtimeDone status=success DEBUG
runtimeDone status!=success WARN
report status=success INFO
report status!=success WARN
extension state=success INFO
extension state!=success WARN
logSubscription - INFO
telemetrySubscription - INFO
logsDropped - WARN
Note

The Accessing real-time telemetry data for extensions using the Telemetry API always emits the complete set of platform events. Configuring the level of the system logs Lambda sends to CloudWatch doesn’t affect Lambda Telemetry API behavior.

Application log-level filtering with custom runtimes

When you configure application log-level filtering for your function, behind the scenes Lambda sets the application log level in the runtime using the AWS_LAMBDA_LOG_LEVEL environment variable. Lambda also sets your function's log format using the AWS_LAMBDA_LOG_FORMAT environment variable. You can use these variables to integrate Lambda advanced logging controls into a custom runtime.

For the ability to configure logging settings for a function using a custom runtime with the Lambda console, AWS CLI, and Lambda APIs, configure your custom runtime to check the value of these environment variables. You can then configure your runtime's loggers in accordance with the log format and log levels you select.

Configuring CloudWatch log groups

By default, CloudWatch automatically creates a log group named /aws/lambda/<function name> for your function when it's first invoked. To configure your function to send logs to an existing log group, or to create a new log group for your function, you can use the Lambda console or the AWS CLI. You can also configure custom log groups using the CreateFunction and UpdateFunctionConfiguration Lambda API commands and the AWS Serverless Application Model (AWS SAM) AWS::Serverless::Function resource.

You can configure multiple Lambda functions to send logs to the same CloudWatch log group. For example, you could use a single log group to store logs for all of the Lambda functions that make up a particular application. When you use a custom log group for a Lambda function, the log streams Lambda creates include the function name and function version. This ensures that the mapping between log messages and functions is preserved, even if you use the same log group for multiple functions.

The log stream naming format for custom log groups follows this convention:

YYYY/MM/DD/<function_name>[<function_version>][<execution_environment_GUID>]

Note that when configuring a custom log group, the name you select for your log group must follow the CloudWatch Logs naming rules. Additionally, custom log group names mustn't begin with the string aws/. If you create a custom log group beginning with aws/, Lambda won't be able to create the log group. As a result of this, your function's logs won't be sent to CloudWatch.

To change a function’s log group (console)

  1. Open the Functions page of the Lambda console.

  2. Choose a function.

  3. On the function configuration page, choose Monitoring and operations tools.

  4. In the Logging configuration pane, choose Edit.

  5. In the Logging group pane, for CloudWatch log group, choose Custom.

  6. Under Custom log group, enter the name of the CloudWatch log group you want your function to send logs to. If you enter the name of an existing log group, then your function will use that group. If no log group exists with the name that you enter, then Lambda will create a new log group for your function with that name.

To change a function's log group (AWS CLI)

  • To change the log group of an existing function, use the update-function-configuration command.

    aws lambda update-function-configuration \
  --function-name myFunction \
  --logging-config LogGroup=myLogGroup
To specify a custom log group when you create a function (AWS CLI)

  • To specify a custom log group when you create a new Lambda function using the AWS CLI, use the --logging-config option. The following example command creates a Node.js Lambda function that sends logs to a log group named myLogGroup.

    aws lambda create-function \
  --function-name myFunction \
  --runtime nodejs20.x \
  --handler index.handler \
  --zip-file fileb://function.zip \
  --role arn:aws:iam::123456789012:role/LambdaRole \
  --logging-config LogGroup=myLogGroup

Execution role permissions

For your function to send logs to CloudWatch Logs, it must have the logs:PutLogEvents permission. When you configure your function's log group using the Lambda console, if your function doesn't have this permission, Lambda adds it to the function's execution role by default. When Lambda adds this permission, it gives the function permission to send logs to any CloudWatch Logs log group.

To prevent Lambda from automatically updating the function's execution role and edit it manually instead, expand Permissions and uncheck Add required permissions.

When you configure your function's log group using the AWS CLI, Lambda won't automatically add the logs:PutLogEvents permission. Add the permission to your function's execution role if it doesn't already have it. This permission is included in the AWSLambdaBasicExecutionRole managed policy.