Amazon CloudFront
Developer Guide (API Version 2014-10-21)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Access Logs

You can configure CloudFront to create log files that contain detailed information about every user request that CloudFront receives. These access logs are available for both web and RTMP distributions. If you enable logging, you can also specify the Amazon S3 bucket that you want CloudFront to save files in.

How Logging Works

The following diagram shows how CloudFront logs information about requests for your objects.

Basic flow for access logs

How CloudFront Logs Information About Requests for Your Objects

In this diagram, you have two websites, A and B, and two corresponding CloudFront distributions. Users request your objects using URLs that are associated with your distributions.

CloudFront routes each request to the appropriate edge location.

CloudFront writes data about each request to a log file specific to that distribution. In this example, information about requests related to Distribution A goes into a log file just for Distribution A, and information about requests related to Distribution B goes into a log file just for Distribution B.

CloudFront periodically saves the log file for a distribution in the Amazon S3 bucket that you specified when you enabled logging. CloudFront then starts saving information about subsequent requests in a new log file for the distribution.


Each entry in a log file gives details about a single request. For more information about log file format, see Log File Format.

Choosing an Amazon S3 Bucket for Your Access Logs

When you enable logging for a distribution, you specify the Amazon S3 bucket that you want CloudFront to store log files in. If you're using Amazon S3 as your origin, we recommend that you do not use the same bucket for your log files; using a separate bucket simplifies maintenance.

You can store the log files for multiple distributions in the same bucket. When you enable logging, you can specify an optional prefix for the file names, so you can keep track of which log files are associated with which distributions.

If no users access your content during a given hour, you don't receive any log files for that hour.

Amazon S3 Permissions Required to Access Your Log Files

You must have Amazon S3 FULL_CONTROL permission for the bucket that you specify for log files. You have this permission by default if you're the bucket owner. If you're not, the bucket owner must grant your AWS account FULL_CONTROL permission.

When you enable logging, CloudFront automatically updates the bucket policy for the bucket to give the AWS data feeds account read and write access. This account writes the log files to the bucket.

Each log file has an access control list (ACL) that is separate from the bucket policy. The bucket owner has FULL_CONTROL permission for the log files, the distribution owner (if different from the bucket owner) has no permission, and the data feeds account has read and write permissions.

Note

Removing the permissions for the data feeds account doesn't disable logging. If you remove permissions but you don't disable logging, CloudFront adds the permissions again the next time the data feeds account needs to write a log file to your log bucket.

If you disable logging, CloudFront doesn't remove the read/write permissions for the data feeds account on either the bucket or the log files. If you want, you can do that yourself.

File Name Format

The name of each log file that CloudFront saves in your Amazon S3 bucket uses the following file name format:

bucket-name.s3.amazonaws.com/optional-prefix/distribution-ID.YYYY-MM-DD-HH.unique-ID.gz

The date and time are in Coordinated Universal time (UTC).

For example, if your bucket name is mylogs, your prefix is myprefix/, and your distribution ID is EMLARXS9EXAMPLE, your file names look similar to this:

mylogs.s3.amazonaws.com/myprefix/EMLARXS9EXAMPLE.2014-11-14-20.RT4KCN4SGK9.gz

When you enable logging for a distribution, you can specify an optional prefix for the file names, so you can keep track of which log files are associated with which distributions. If you include a value for the log file prefix and your prefix doesn't include a /, CloudFront adds one automatically. If your value does include a /, CloudFront doesn't add another one.

The .gz at the end of the file name indicates that CloudFront has compressed the log file using gzip.

Timing of Log File Delivery

CloudFront delivers access logs for a distribution up to several times an hour. In general, a log file contains information about the requests that CloudFront received during a given time period. CloudFront usually delivers the log file for that time period to your Amazon S3 bucket within an hour of the events that appear in the log. Note, however, that some or all log file entries for a time period can sometimes be delayed by up to 24 hours. When log entries are delayed, CloudFront saves them in a log file for which the file name includes the date and time of the period in which the requests occurred, not the date and time when the file was delivered.

When creating a log file, CloudFront consolidates information for your distribution from all of the edge locations that received requests for your objects during the time period that the log file covers.

CloudFront can save more than one file for a time period depending on how many requests CloudFront receives for the objects associated with a distribution.

Note

If no users request your objects during the time period, you don't receive any log files for that period.

Analyzing Access Logs

Because you can receive multiple access logs an hour, we recommend that you combine all the log files you receive for a given period into one file. You can then analyze the data for that period more quickly and accurately.

Important

We recommend that you use the logs to understand the nature of the requests for your content, not as a complete accounting of all requests. CloudFront delivers access logs on a best-effort basis. The log entry for a particular request might be delivered long after the request was actually processed and, in rare cases, a log entry might not be delivered at all. When a log entry is omitted from access logs, the number of entries in the access logs won't match the usage that appears in the AWS usage and billing reports.

For more information about CloudFront access logs, including recommendations for tools that you can use to analyze access logs, see Using CloudFront Logging.

Editing Your Logging Settings

You can enable or disable logging, change the Amazon S3 bucket where your logs are stored, and change the prefix for log files by using the CloudFront console or the CloudFront API. Your changes to logging settings take effect within 12 hours.

For more information, see the following topics:

To use the CloudFront API to change access log settings for web distributions, you must use the 2009-04-02 or later version of the API. To use the CloudFront API to change access log settings for RTPM distributions, you must use the 2010-05-01 or later version of the API.

Deleting Log Files from an Amazon S3 Bucket

CloudFront does not automatically delete log files from your Amazon S3 bucket. For information about deleting log files from an Amazon S3 bucket, see the following topics:

  • Using the Amazon S3 console: Deleting an Object in the Amazon Simple Storage Service Console User Guide.

  • Using the REST API: DELETE Object in the Amazon Simple Storage Service API Reference.

  • Using the SOAP API: DeleteObject in the Amazon Simple Storage Service API Reference.

Log File Format

Each entry in a log file gives details about a single user request. The log files for web and for RTMP distributions are not identical, but they share the following characteristics:

  • Use the W3C extended log file format. (For more information, go to http://www.w3.org/TR/WD-logfile.html.)

  • Contain tab-separated values.

  • Contain records that are not necessarily in chronological order.

  • Contain two header lines: one with the file-format version, and another that lists the W3C fields included in each record.

  • Substitute URL-encoded equivalents for spaces and non-standard characters in field values.

    These non-standard characters consist of all ASCII codes below 32 and above 127, plus the characters in the following table. The URL encoding standard is RFC 1738. For more information, go to http://www.ietf.org/rfc/rfc1738.txt.

URL-Encoded Value

Character

%3C

<

%3E

>

%22

"

%23

#

%25

%

%7B

{

%7D

}

%7C

|

%5C

\

%5E

^

%7E

~

%5B

[

%5D

]

%60

`

%27

'

%20

space

Web Distribution Log File Format

The log file for a web distribution includes the following fields in the listed order.

FieldDescription
dateThe date on which the event occurred in the format yyyy-mm-dd, for example, 2014-05-23. The date and time are in Coordinated Universal Time (UTC).
timeThe time when the server finished processing the request (in UTC), for example, 01:42:39.
x-edge-location The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront.
sc-bytesServer to client bytes, including headers, for example, 1045619.
c-ip Client IP, for example, 192.0.2.183.
cs-method HTTP access method.
cs(Host) DNS name (the CloudFront distribution name specified in the request). If you made the request to a CNAME, the DNS name field will contain the underlying distribution DNS name, not the CNAME.
cs-uri-stemURI stem (for example, /images/daily-ad.jpg).
sc-status

One of the following values:

  • An HTTP status code (for example, 200). For more information, see HTTP 4xx and 5xx Status Codes that CloudFront Caches.

  • 000, which indicates that the viewer closed the connection (for example, closed the browser tab) before CloudFront could respond to a request.

cs(Referer) The referrer.
cs(User-Agent)The user agent.
cs-uri-queryThe query string portion of the URI that is included on the connect string. When a URI doesn't contain a query string, the log file contains a single hyphen (-) in the cs-uri-query field for that request. The encoding standard is RFC 1738. For more information, see Log File Format.
cs(Cookie)

The cookie header in the request, including name-value pairs and the associated attributes. If you enable cookie logging, CloudFront logs the cookies in all requests regardless of which cookies you choose to forward to the origin: none, all, or a whitelist of cookie names. When a request doesn't include a cookie header, the log file contains a single hyphen (-) in the cs(Cookie) field for that request.

For more information about cookies, see Configuring CloudFront to Cache Objects Based on Cookies.

x-edge-result-type

The result type of a request. Result types include:

  • Hit: CloudFront served the object to the viewer from the edge cache.

  • RefreshHit: CloudFront found the object in the edge cache but it had expired, so CloudFront contacted the origin to verify that the cache has the latest version of the object.

  • Miss: The request could not be satisfied by an object in the edge cache, so CloudFront forwarded the request to the origin server and returned the result to the viewer.

  • LimitExceeded: The request was denied because a CloudFront limit was exceeded.

  • CapacityExceeded: CloudFront returned a 503 error because the edge location didn't have enough capacity at the time of the request to serve the object.

  • Error: Typically, this means the request resulted in a client error (sc-status is 4xx) or a server error (sc-status is 5xx).

    If sc-status is 403 and you configured CloudFront to restrict the geographic distribution of your content, the request might have come from a restricted location. For more information about geo restriction, see Restricting the Geographic Distribution of Your Content.

    If sc-status is 2xx, the client probably disconnected before finishing the download.

x-edge-request-idAn encrypted string that uniquely identifies a request.
x-host-header

The value that the viewer included in the Host header for this request. This is the domain name in the request:

  • If you're using the CloudFront domain name (http://d111111abcdef8.cloudfront.net/logo.png), the x-host-header column contains the domain name that CloudFront assigned to your distribution, such as d111111abcdef8.cloudfront.net.

  • If you're using alternate domain names (http://example.com/logo.png), the x-host-header column contains the alternate domain name, such as example.com. To use alternate domain names, you must add them to your distribution. For more information, see Using Alternate Domain Names (CNAMEs).

cs-protocolThe protocol the viewer specified in the request, either http or https.
cs-bytesThe number of bytes of data that the viewer included in the request (client to server bytes), including headers.
time-takenThe number of seconds between the time a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering.

Note

Question marks (?) in URLs and query strings are not included in the log.

The following is an example log file for a web distribution:

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken 
2014-05-23 01:13:11 FRA2 182 192.0.2.10 GET d111111abcdef8.cloudfront.net /view/my/file.html 200 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC) - zip=98101 RefreshHit MRVMF7KydIvxMWfJIglgwHQwZsbG2IhRJ07sn9AkKUFSHS9EXAMPLE== d111111abcdef8.cloudfront.net http - 0.001
2014-05-23 01:13:12 LAX1 2390282 192.0.2.202 GET d111111abcdef8.cloudfront.net /soundtrack/happy.mp3 304 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1) a=b&c=d zip=50158 Hit xGN7KWpVEmB9Dp7ctcVFQC4E-nrcOcEKS3QyAez--06dV7TEXAMPLE== d111111abcdef8.cloudfront.net http - 0.002

RTMP Distribution Log File Format

Each record in an RTMP access log represents a playback event, for example, connect, play, pause, stop, disconnect, and so on. As a result, CloudFront generates multiple log records each time a viewer watches a video. To relate log records that stem from the same stream ID, use the x-sid field.

Note

Some fields have values for all events, and some have values only for Play, Stop, Pause, Unpause, and Seek events. Usually, when the log file contains a single hyphen (-) for a field, the field isn't relevant for the corresponding event.

The following table describes the fields that are present in each record in the RTMP distribution log file, regardless of the type of event. The fields appear in the log in the order listed.

FieldDescription
dateThe date on which the event occurred in the format yyyy-mm-dd, for example, 2014-05-23. The date and time are in Coordinated Universal Time (UTC).
timeThe time when the server received the request (in UTC), for example, 01:42:39.
x-edge-locationThe edge location where the playback event occurred. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations might change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront.
c-ipClient IP, for example, 192.0.2.183.
x-eventThe event type. This is a Connect, Disconnect, Play, Stop, Pause, Unpause, or Seek event.
sc-bytesThe running total number of bytes sent from the server to the client, up to the time of the event.
x-cf-statusA code indicating the status of the event. Currently, "OK" is the only value for this field. New functionality in the future could require new status codes.
x-cf-client-id

An opaque string identifier that can be used to differentiate clients.

This value is unique for each connection.

cs-uri-stemThe stem portion of the URI, including the application and the application instance. This is sometimes referred to as the FMS connect string. For example, rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st.
cs-uri-queryThe query string portion of the URI that is included on the connect string.
c-referrerThe URI of the referrer.
x-page-urlThe URL of the page from which the SWF is linked.
c-user-agentThe user agent.

The following fields usually have values only for Play, Stop, Pause, Unpause, and Seek events. For other events, they contain a single hyphen (-). These fields appear in the log after the fields in the preceding table and in the order listed.

FieldDescription
x-snameThe stream name.
x-sname-queryThe stream query string, if any.
x-file-extThe stream type, for example, FLV.
x-sidThe stream ID. This is a unique integer identifier for the connection.

Note

Question marks (?) in URLs and query strings are not included in the log.

The following is an example of a log file for an RTMP distribution:

#Version: 1.0
#Fields: date time x-edge-location c-ip x-event sc-bytes x-cf-status x-cf-client-id cs-uri-stem cs-uri-query c-referrer x-page-url​  c-user-agent x-sname x-sname-query x-file-ext x-sid
2010-03-12   23:51:20   SEA4   192.0.2.147   connect   2014   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   -   -   -   -
2010-03-12   23:51:21   SEA4   192.0.2.222   play   3914   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   myvideo   p=2&q=4   flv   1
2010-03-12   23:53:44   SEA4   192.0.2.4   stop   323914   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/other/myvideo   p=2&q=4   flv   1
2010-03-12   23:53:44   SEA4   192.0.2.103   play   8783724   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/favs/myothervideo   p=42&q=14   mp4   2
2010-03-12   23:56:21   SEA4   192.0.2.199   stop   429822014   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/favs/myothervideo   p=42&q=14   mp4   2
2010-03-12   23:59:44   SEA4   192.0.2.14   disconnect   429824092   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   -   -   -   -	

Charges for Access Logs

Access logging is an optional feature of CloudFront. There is no extra charge for enabling access logging. However, you accrue the usual Amazon S3 charges for storing and accessing the files on Amazon S3 (you can delete them at any time). For more information about charges for CloudFront, see CloudFront Billing and Usage Reports.