Using AWS Lambda with AWS X-Ray

You can use AWS X-Ray to visualize the components of your application, identify performance bottlenecks, and troubleshoot requests that resulted in an error. Your Lambda functions send trace data to X-Ray, and X-Ray processes the data to generate a service map and searchable trace summaries.

If you've enabled X-Ray tracing in a service that invokes your function, Lambda sends traces to X-Ray automatically. The upstream service, such as Amazon API Gateway, or an application hosted on Amazon EC2 that is instrumented with the X-Ray SDK, samples incoming requests and adds a tracing header that tells Lambda to send traces or not. Traces from upstream message producers, such as Amazon SQS, are automatically linked to traces from downstream Lambda functions, creating an end-to-end view of the entire application. For more information, see Tracing event-driven applications in the AWS X-Ray Developer Guide.

Note

X-Ray tracing is currently not supported for Lambda functions with Amazon Managed Streaming for Apache Kafka (Amazon MSK), self-managed Apache Kafka, Amazon MQ with ActiveMQ and RabbitMQ, or Amazon DocumentDB event source mappings.

To toggle active tracing on your Lambda function with the console, follow these steps:

To turn on active tracing

1. Open the Functions page of the Lambda console.

2. Choose a function.

3. Choose Configuration and then choose Monitoring and operations tools.

4. Choose Edit.

5. Under X-Ray, toggle on Active tracing.

6. Choose Save.

Pricing

You can use X-Ray tracing for free each month up to a certain limit as part of the AWS Free Tier. Beyond that threshold, X-Ray charges for trace storage and retrieval. For more information, see AWS X-Ray pricing.

Your function needs permission to upload trace data to X-Ray. When you activate tracing in the Lambda console, Lambda adds the required permissions to your function's execution role. Otherwise, add the AWSXRayDaemonWriteAccess policy to the execution role.

X-Ray doesn't trace all requests to your application. X-Ray applies a sampling algorithm to ensure that tracing is efficient, while still providing a representative sample of all requests. The sampling rate is 1 request per second and 5 percent of additional requests.

Note

You cannot configure the X-Ray sampling rate for your functions.

In X-Ray, a trace records information about a request that is processed by one or more services. Services record segments that contain layers of subsegments. Lambda records a segment for the Lambda service that handles the invocation request, and one for the work done by the function. The function segment comes with subsegments for Initialization, Invocation, Restore (Lambda SnapStart only), and Overhead. For more information, see Lambda execution environment lifecycle.

Note

X-Ray treats unhandled exceptions in your Lambda function as Error statuses. X-Ray records Fault statuses only when Lambda experiences internal server errors. For more information, see Errors, faults, and exceptions in the X-Ray Developer Guide.

The Initialization subsegment represents the init phase of the Lambda execution environment lifecycle. During this phase, Lambda creates or unfreezes an execution environment with the resources you have configured, downloads the function code and all layers, initializes extensions, initializes the runtime, and runs the function's initialization code.

The Invocation subsegment represents the invoke phase where Lambda invokes the function handler. This begins with runtime and extension registration and it ends when the runtime is ready to send the response.

(Lambda SnapStart only) The Restore subsegment shows the time it takes for Lambda to restore a snapshot, load the runtime (JVM), and run any afterRestore runtime hooks. The process of restoring snapshots can include time spent on activities outside the MicroVM. This time is not reported in the Restore subsegment, but is included in the AWS::Lambda segment in X-Ray traces.

The Overhead subsegment represents the phase that occurs between the time when the runtime sends the response and the signal for the next invoke. During this time, the runtime finishes all tasks related to an invoke and prepares to freeze the sandbox.

Note

Occasionally, you may notice a large gap between the function initialization and invocation phases in your X-Ray traces. For functions using provisioned concurrency, this is because Lambda initializes your function instances well in advance of invocation. For functions using unreserved (on-demand) concurrency, Lambda may proactively initialize a function instance, even if there's no invocation. Visually, both of these cases show up as a time gap between the initialization and invocation phases.

Important

In Lambda, you can use the X-Ray SDK to extend the Invocation subsegment with additional subsegments for downstream calls, annotations, and metadata. You can't access the function segment directly or record work done outside of the handler invocation scope.

See the following topics for a language-specific introduction to tracing in Lambda:

• Instrumenting Node.js code in AWS Lambda

• Instrumenting Python code in AWS Lambda

• Instrumenting Ruby code in AWS Lambda

• Instrumenting Java code in AWS Lambda

• Instrumenting Go code in AWS Lambda

• Instrumenting C# code in AWS Lambda

For a full list of services that support active instrumentation, see Supported AWS services in the AWS X-Ray Developer Guide.

Execution role permissions

Lambda needs the following permissions to send trace data to X-Ray. Add them to your function's execution role.

• xray:PutTraceSegments

• xray:PutTelemetryRecords

These permissions are included in the AWSXRayDaemonWriteAccess managed policy.

The AWS X-Ray daemon

Instead of sending trace data directly to the X-Ray API, the X-Ray SDK uses a daemon process. The AWS X-Ray daemon is an application that runs in the Lambda environment and listens for UDP traffic that contains segments and subsegments. It buffers incoming data and writes it to X-Ray in batches, reducing the processing and memory overhead required to trace invocations.

The Lambda runtime allows the daemon to up to 3 percent of your function's configured memory or 16 MB, whichever is greater. If your function runs out of memory during invocation, the runtime terminates the daemon process first to free up memory.

The daemon process is fully managed by Lambda and cannot be configured by the user. All segments generated by function invocations are recorded in the same account as the Lambda function. The daemon cannot be configured to redirect them to any other account.

For more information, see The X-Ray daemon in the X-Ray Developer Guide.

Enabling active tracing with the Lambda API

To manage tracing configuration with the AWS CLI or AWS SDK, use the following API operations:

• UpdateFunctionConfiguration

• GetFunctionConfiguration

• CreateFunction

The following example AWS CLI command enables active tracing on a function named my-function.

aws lambda update-function-configuration --function-name my-function \
--tracing-config Mode=Active

Tracing mode is part of the version-specific configuration when you publish a version of your function. You can't change the tracing mode on a published version.

Enabling active tracing with AWS CloudFormation

To activate tracing on an AWS::Lambda::Function resource in an AWS CloudFormation template, use the TracingConfig property.

Example function-inline.yml – Tracing configuration

Resources:
  function:
    Type: AWS::Lambda::Function
    Properties:
      TracingConfig:
        Mode: Active
      ...

For an AWS Serverless Application Model (AWS SAM) AWS::Serverless::Function resource, use the Tracing property.

Example template.yml – Tracing configuration

Resources:
  function:
    Type: AWS::Serverless::Function
    Properties:
      Tracing: Active
      ...