Amazon CloudFront
Developer Guide (API Version 2014-05-31)
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.Did this page help you?  Yes | No |  Tell us about it...

Access Logs

Amazon CloudFront provides optional log files with information about user requests. This section describes how to enable and disable logging, the content of log files, and how AWS charges you if you decide to use logging.

Note

If you use a custom origin, you will need to create an Amazon S3 bucket to store your log files in.

Overview

You can enable CloudFront to deliver access logs per distribution to an Amazon S3 bucket of your choice. The following figure and table describe the basic process for access logs.

Basic flow for access logs

Process for Access Logs

Your end users use your application or website.

In this graphic, you have two different websites, A and B, each using a different CloudFront distribution (Distribution A and Distribution B).

Your end users send requests, and CloudFront routes each request to the appropriate edge location.

CloudFront writes data about each request to a log file specific to that distribution. In this example, Information about requests related to Distribution A goes into a log file just for Distribution A, and information about requests related to Distribution B goes into a log file just for Distribution B.

CloudFront periodically puts the distribution's log file in an Amazon S3 bucket of your choice, and then starts writing a new log file for the distribution.


Each entry in a log file gives details about a single request. For more information about log file format, see Log File Format.

You can store log files for a distribution either in the same Amazon S3 bucket as your origin server or in a different bucket. (To simplify maintenance, we recommend that you use a separate bucket.) You can also store the log files for multiple distributions in the same bucket. When you enable logging for a particular distribution, you can specify an optional prefix for the names of your log files.

If no users access your content during a given hour, you don't receive any log files for that hour.

Analyzing Access Logs

Because logs for a single stream can get recorded in multiple files, we recommend that you combine all the log files you receive for a given period into one file. You can then analyze the data for that period more quickly and accurately.

Important

We recommend that you use the logs to understand the nature of the requests for your content, not as a complete accounting of all requests. CloudFront delivers access logs on a best-effort basis. The log record for a particular request might be delivered long after the request was actually processed, or not at all. In rare cases, usage that appears in the AWS usage tracking and billing systems might not appear in CloudFront access logs.

For more information about CloudFront access logs, including recommendations for tools that you can use to analyze access logs, see Using CloudFront Logging.

Bucket and File Ownership

You must have Amazon S3 FULL_CONTROL permission for the log file bucket. You have this permission by default if you're the bucket owner. If you're not, the bucket owner must grant your AWS account FULL_CONTROL permission.

When you enable logging, you do it with an API call to the CloudFront API. Making that API call also automatically calls the Amazon S3 API to update the bucket's ACL to allow read and write permissions for the AWS data feeds account. This account writes the log files to the bucket.

Each log file has its own ACL (separate from the bucket's ACL). The bucket owner has FULL_CONTROL permission for the log files, the distribution owner (if not the bucket owner) has no permission, and the data feeds account has read and write permission.

Note

Removing the permissions for the data feeds account does not disable logging. If you remove those permissions, but don't disable logging (which you do with the API), we reinstate those permissions the next time the data feeds account needs to write a log file to your log bucket.

If you disable logging, we don't remove the read/write permissions for the data feeds account on either the bucket or the log files. If you want, you can do that yourself.

How to Change Logging Settings

You can enable or disable logging, change the Amazon S3 bucket where your logs are stored, and change the prefix for log files using the CloudFront console or using the CloudFront API:

Your changes to logging settings take effect within 12 hours.

To use the CloudFront API to change access log settings for:

  • web distributions, you must use the 2009-04-02 or later version of the API.

  • RTMP distributions, you must use the 2010-05-01 or later version of the API.

  • cookies, you must use the 2012-07-01 or later version of the API.

Note

To enable easier listing of keys in a bucket, Amazon S3 users commonly use a prefix followed by a slash (/) as a delimiter.

How to Delete Log Files from an Amazon S3 Bucket

CloudFront does not automatically delete log files from the Amazon S3 bucket that you specified when you enabled logging. For information about deleting log files from an Amazon S3 bucket, see the applicable Amazon S3 documentation:

  • Using the Amazon S3 console: See Deleting an Object in the Amazon Simple Storage Service Console User Guide.

  • Using the REST API: See DELETE Object in the Amazon Simple Storage Service API Reference.

  • Using the SOAP API: See DeleteObject in the Amazon Simple Storage Service API Reference.

File Name Format and Timing of File Delivery

The filename follows this format (with the date and hour in UTC):

{bucket-name}.s3.amazonaws.com/{optional-prefix/}{distribution-ID}.{YYYY}-{MM}-{DD}-{HH}.{unique-ID}.gz

For example, if your bucket name is mylogs and your prefix is myprefix/, your filenames look similar to this:

mylogs.s3.amazonaws.com/myprefix/EMLARXS9EXAMPLE.2012-07-01-20.RT4KCN4SGK9.gz

If you include a value for {optional-prefix/}, and if your value doesn't include a /, CloudFront adds one automatically. If your value does include a /, CloudFront does not add another one.

CloudFront compresses each log file using gzip and saves it in the Amazon S3 bucket that you specify. Typically, CloudFront saves log files within 24 hours after receiving the corresponding requests. Depending on how many users submit requests, CloudFront can save several log files per hour.

Note

If no users submit requests during a given hour, you don't receive any log files for that hour.

Log File Format

Each entry in a log file gives details about a single end user request. The log files for web and for RTMP distributions are not identical, but both types of log files:

  • Use the W3C extended log file format. (For more information, go to http://www.w3.org/TR/WD-logfile.html.)

  • Contain tab-separated values.

  • Contain records that are not necessarily in chronological order.

  • Contain two header lines: one with the file-format version, and another that lists the W3C fields included in each record.

  • Substitute URL-encoded equivalents for spaces and non-standard characters in field values.

    These non-standard characters consist of all ASCII codes below 32 and above 127, plus the characters in the following table. The URL encoding standard is RFC 1738. For more information, go to http://www.ietf.org/rfc/rfc1738.txt.

URL-Encoded Value

Character

%3C

<

%3E

>

%22

"

%23

#

%25

%

%7B

{

%7D

}

%7C

|

%5C

\

%5E

^

%7E

~

%5B

[

%5D

]

%60

`

%27

'

%20

space

Web Distribution Log File Format

The log file for a web distribution includes the following fields in the listed order.

FieldDescription
dateThe date (UTC) on which the event occurred, for example, 2009-03-10.
timeTime when the server finished processing the request (UTC), for example, 01:42:39.
x-edge-location The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations may change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront.
sc-bytesServer to client bytes, including headers, for example, 1045619.
c-ip Client IP, for example, 192.0.2.183.
cs-method HTTP access method.
cs(Host) DNS name (the CloudFront distribution name specified in the request). If you made the request to a CNAME, the DNS name field will contain the underlying distribution DNS name, not the CNAME.
cs-uri-stemURI stem (for example, /images/daily-ad.jpg).
sc-status

One of the following values:

  • An HTTP status code (for example, 200). For more information, see HTTP 4xx and 5xx Status Codes that CloudFront Caches.

  • 000, which indicates that the viewer closed the connection (for example, closed the browser tab) before CloudFront could respond to a request.

cs(Referer) The referrer.
cs(User-Agent)The user agent.
cs-uri-queryThe query string portion of the URI that is included on the connect string. When a URI doesn't contain a query string, the log file contains a single hyphen (-) in the cs-uri-query field for that request. The encoding standard is RFC 1738. For more information, see Log File Format.
cs(Cookie)

The cookie header in the request, including name-value pairs and the associated attributes. If you enable cookie logging, CloudFront logs the cookies in all requests regardless of which cookies you choose to forward to the origin: none, all, or a whitelist of cookie names. When a request doesn't include a cookie header, the log file contains a single hyphen (-) in the cs(Cookie) field for that request.

For more information about cookies, see Configuring CloudFront to Cache Objects Based on Cookies.

x-edge-result-type

The result type of a request. Result types include:

  • Hit: CloudFront served the object to the viewer from the edge cache.

  • RefreshHit: CloudFront found the object in the edge cache but it had expired, so CloudFront contacted the origin to verify that the cache has the latest version of the object.

  • Miss: The request could not be satisfied by an object in the edge cache, so CloudFront forwarded the request to the origin server and returned the result to the viewer.

  • LimitExceeded: The request was denied because a CloudFront limit was exceeded.

  • CapacityExceeded: CloudFront returned a 503 error because the edge location didn't have enough capacity at the time of the request to serve the object.

  • Error: Typically, this means the request resulted in a client error (sc-status is 4xx) or a server error (sc-status is 5xx).

    If sc-status is 403 and you configured CloudFront to restrict the geographic distribution of your content, the request might have come from a restricted location. For more information about geo restriction, see Restricting the Geographic Distribution of Your Content.

    If sc-status is 2xx, the client probably disconnected before finishing the download.

x-edge-request-idAn encrypted string that uniquely identifies a request.
x-host-header

The value that the viewer included in the Host header for this request. This is the domain name in the request:

  • If you're using the CloudFront domain name (http://d111111abcdef8.cloudfront.net/logo.png), the x-host-header column contains the domain name that CloudFront assigned to your distribution, such as d111111abcdef8.cloudfront.net.

  • If you're using alternate domain names (http://example.com/logo.png), the x-host-header column contains the alternate domain name, such as example.com. To use alternate domain names, you must add them to your distribution. For more information, see Using Alternate Domain Names (CNAMEs).

cs-protocolThe protocol the viewer specified in the request, either http or https.
cs-bytesThe number of bytes of data that the viewer included in the request (client to server bytes), including headers.
time-takenThe number of seconds between the time a CloudFront edge server receives a viewer's request and the time that CloudFront writes the last byte of the response to the server's output queue as measured on the server. From the perspective of the viewer, the total time to get the full object will be longer than this value due to network latency and TCP buffering.

Note

Question marks (?) in URLs and query strings are not included in the log.

The following is an example log file for a web distribution.

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken 
05/01/2014 01:13:11 FRA2 182 192.0.2.10 GET d111111abcdef8.cloudfront.net /view/my/file.html 200 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC) - zip=98101 RefreshHit MRVMF7KydIvxMWfJIglgwHQwZsbG2IhRJ07sn9AkKUFSHS9EXAMPLE== d111111abcdef8.cloudfront.net http - 0.001
05/01/2014 01:13:12 LAX1 2390282 192.0.2.202 GET d111111abcdef8.cloudfront.net /soundtrack/happy.mp3 304 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1) a=b&c=d zip=50158 Hit xGN7KWpVEmB9Dp7ctcVFQC4E-nrcOcEKS3QyAez--06dV7TEXAMPLE== d111111abcdef8.cloudfront.net http - 0.002

RTMP Distribution Log File Format

Each record in an RTMP access log represents a playback event, for example, connect, play, pause, stop, disconnect, and so on. As a result, CloudFront generates multiple log records each time a viewer watches a video. To relate log records that stem from the same stream ID, use the x-sid field.

Note

Some fields have values for all events, and some have values only for Play, Stop, Pause, Unpause, and Seek events. Usually, when the log file contains a single hyphen (-) for a field, the field isn't relevant for the corresponding event.

The following table describes the fields that are present in each record in the RTMP distribution log file, regardless of the type of event. The fields appear in the log in the order listed.

FieldDescription
dateDate (UTC) on which the event occurred.
timeTime when the server received the request (UTC), for example, 01:42:39.
x-edge-locationThe edge location where the playback event occurred. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations may change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront.
c-ipClient IP, for example, 192.0.2.183.
x-eventThe event type. This is a Connect, Disconnect, Play, Stop, Pause, Unpause, or Seek event.
sc-bytesThe running total number of bytes sent from the server to the client, up to the time of the event.
x-cf-statusA code indicating the status of the event. Currently, "OK" is the only value for this field. New functionality in the future could require new status codes.
x-cf-client-id

An opaque string identifier that can be used to differentiate clients

This value is unique for each connection.

cs-uri-stemThe stem portion of the URI, including the application and the application instance. This is sometimes referred to as the FMS connect string. For example, rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st.
cs-uri-queryThe query string portion of the URI that is included on the connect string.
c-referrerThe URI of the referrer.
x-page-urlThe URL of the page from which the SWF is linked.
c-user-agentThe user agent.

The following fields usually have values only for Play, Stop, Pause, Unpause, and Seek events. For other events, they contain a single hyphen (-). These fields appear in the log after the fields in the previous table and in the order listed.

FieldDescription
x-snameThe stream name.
x-sname-queryThe stream query string, if any.
x-file-extThe stream type, for instance, FLV.
x-sidThe stream ID. This is a unique integer identifier for the connection.

Note

Question marks (?) in URLs and query strings are not included in the log.

The following is an example of a log file for an RTMP distribution.

#Version: 1.0
#Fields: date time x-edge-location c-ip x-event sc-bytes x-cf-status x-cf-client-id cs-uri-stem cs-uri-query c-referrer x-page-url​  c-user-agent x-sname x-sname-query x-file-ext x-sid
2010-03-12   23:51:20   SEA4   192.0.2.147   connect   2014   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   -   -   -   -
2010-03-12   23:51:21   SEA4   192.0.2.222   play   3914   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   myvideo   p=2&q=4   flv   1
2010-03-12   23:53:44   SEA4   192.0.2.4   stop   323914   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/other/myvideo   p=2&q=4   flv   1
2010-03-12   23:53:44   SEA4   192.0.2.103   play   8783724   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/favs/myothervideo   p=42&q=14   mp4   2
2010-03-12   23:56:21   SEA4   192.0.2.199   stop   429822014   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   dir/favs/myothervideo   p=42&q=14   mp4   2
2010-03-12   23:59:44   SEA4   192.0.2.14   disconnect   429824092   OK   bfd8a98bee0840d9b871b7f6ade9908f   rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st​  key=value   http://player.longtailvideo.com/player.swf   http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204   LNX%2010,0,32,18   -   -   -   -	

Charges for Access Logs

Access logging is an optional feature of CloudFront. There is no extra charge for enabling access logging. However, you accrue the usual Amazon S3 charges for storing and accessing the files on Amazon S3 (you can delete them at any time). For more information about charges for CloudFront see CloudFront Billing and Usage Reports.