AWS X-Ray Concepts
AWS X-Ray receives data from services in the form of segments, groups segments with a common request into traces, and processes traces to generate a service graph and provide a visual representation of your application.
The compute resources running your application logic send data about the work that they do in the form of segments. A segment provides the name of the resource, details about the request, and details about the work done. For example, when an HTTP request reaches your application, it can record data about:
The host – host name, alias or IP address
The request – method, client address, path, user agent
The response – status, content
The work done – start and end times, subsegments
The X-Ray SDK gathers information from request and response headers, the code in your application, and metadata about the AWS resources on which it runs. You choose which data is collected by adding modifying your application configuration or code to instrument incoming requests, downstream requests, and AWS SDK clients.
If a request is forwarded to your application by a load balancer or other intermediary,
The client IP in the segment is taken from the
X-Forwarded-For header in the
request instead of the source IP in the IP packet. The client IP recorded for a forwarded
request can be forged so should not be trusted.
Data about the work done can be broken down into subsegments, which provide more granular timing information and details about downstream calls that your application made to fulfill the original request. A subsegment can contain additional details about a call to an AWS service, an external HTTP API, or an SQL database. You can even define arbitrary subsegments to instrument specific functions or lines of code in your application.
X-Ray uses the data that your application sends to generate a service graph. Each AWS resource that sends data to X-Ray appears as a service, with edges connecting the services that work together to serve requests.
name should match the domain name or logical name of the service
generates the segment, but this is not enforced. Any application with permission to
PutTraceSegments can send segments with any name.
For example, your application could use an application load balancer to distribute traffic to Amazon EC2 instances, which use the AWS SDK to contact DynamoDB to store data, and make HTTP calls to external web APIs. In the X-Ray service graph, the instances, DynamoDB tables, and downstream HTTP APIs all appear as separate services connected by edges.
The path of a request through your application is tracked with a trace ID. A trace collects all of the segments generated by a single request, typically an HTTP GET or POST request that travels through a load balancer, hits your application code, and generates downstream calls to other AWS services or external web APIs. A trace ID header is added to each HTTP request by the first supported service that it interacts with, and propagates downstream to track the latency, disposition, and other request data.
To ensure that tracing is efficient, while still providing a representative sample of the requests that your application serves, the first service that a request hits applies a sampling algorithm to determine which requests get traced. You can modify the default sampling rules and configure different sampling rates for different routes that your application serves with the X-Ray SDK.
All requests are traced up to a configurable minimum, after which a percentage of requests
are traced to avoid unnecessary cost. The sampling decision and trace ID, are added to HTTP
requests in tracing headers named
X-Amzn-Trace-Id. The tracing header is added to the request by the first
X-Ray-integrated service that it hits, read by the X-Ray SDK, and included in the
Example Tracing header with root trace ID and sampling decision
X-Amzn-Trace-Id: Root=1-5759e988-bd862e3fe1be46a994272793; Sampled=1
Tracing Header Security
A tracing header can originate from the X-Ray SDK, an AWS service, or the client
request. Your application can remove
X-Amzn-Trace-Id from incoming requests to
avoid issues caused by users adding trace IDs or sampling decisions to their
The tracing header can also contain a parent segment ID if the request originated from an instrumented application. For example, if your application calls a downstream HTTP web API with an instrumented HTTP client, the X-Ray SDK adds the ID of the segment for the original request to the tracing header of the downstream request. An instrumented application that serves the downstream request can record the parent segment ID to connect the two requests.
Example Tracing header with root trace ID, parent segment ID and sampling decision
X-Amzn-Trace-Id: Root=1-5759e988-bd862e3fe1be46a994272793; Parent=53995c3f42cd8ad8; Sampled=1
Even with sampling, a complex application generates a lot of data. The AWS X-Ray console provides an easy-to-navigate view of the service graph. It shows health and performance information that helps you identify issues and opportunities for optimization in your application. For advanced tracing, you can drill down to traces for individual requests, or use filter expressions to find traces related to specific paths or users.