| « PreviousNext » | |
   | Did this page help you? Yes | No | Tell us about it... |
Access Logs
Topics
Amazon CloudFront provides optional log files with information about end user access to your objects. This section describes how to enable and disable logging, the content of log files, and how AWS charges you if you decide to use logging.
Note
If you use a custom origin, you will need to create an Amazon S3 bucket to store your log files in.
Overview
You can enable CloudFront to deliver access logs per distribution to an Amazon S3 bucket of your choice. The following figure and table describe the basic process for access logs.
Process for Access Logs
 |
Your end users use your application or website. In this graphic, you have two different websites, A and B, each using a different CloudFront distribution (Distribution A and Distribution B). |
 |
Your end users send requests for content, and CloudFront routes each request to the appropriate edge location. |
 |
CloudFront writes data about each request to a log file specific to that distribution. In this graphic, CloudFront writes information about requests related to Distribution A in a log file just for Distribution A, and requests for Distribution B in a log file just for Distribution B. |
 |
CloudFront periodically puts the distribution's log file in an Amazon S3 bucket of your choice, and then starts writing a new log file for the distribution. |
Each entry in a log file gives details about a single end user request for an object. For more information about log file format, see Log File Format.
You can store log files for a distribution either in the same Amazon S3 bucket as your origin server or in a different bucket. (To simplify maintenance, we recommend that you use a separate bucket.) You can also store the log files for multiple distributions in the same bucket. When you enable logging for a particular distribution, you can specify an optional prefix for the names of your log files.
If no users access your content during a given hour, you don't receive any log files for that hour.
Analyzing Access Logs
Because logs for a single stream can get recorded in multiple files, we recommend that you combine all the log files you receive for a given period into one file. You can then analyze the data for that period more quickly and accurately.
Important
We recommend that you use the logs to understand the nature of the requests for your content, not as a complete accounting of all requests. CloudFront delivers access logs on a best-effort basis. The log record for a particular request might be delivered long after the request was actually processed, or not at all. In rare cases, usage that appears in the AWS usage tracking and billing systems might not appear in CloudFront access logs.
For more information about CloudFront access logs, including recommendations for tools that you can use to analyze access logs, see Using CloudFront Logging.
Bucket and File Ownership
You must have Amazon S3 FULL_CONTROL permission for the log file bucket.
You have this permission by default if you're the bucket owner. If you're not, the
bucket owner must grant your AWS account FULL_CONTROL permission.
When you enable logging, you do it with an API call to the CloudFront API. Making that API call also automatically calls the Amazon S3 API to update the bucket's ACL to allow read and write permissions for the AWS data feeds account. This account writes the log files to the bucket.
Each log file has its own ACL (separate from the bucket's ACL). The bucket owner has
FULL_CONTROL permission for the log files, the distribution owner (if
not the bucket owner) has no permission, and the data feeds account has read and write
permission.
Note
Removing the permissions for the data feeds account does not disable logging. If you remove those permissions, but don't disable logging (which you do with the API), we reinstate those permissions the next time the data feeds account needs to write a log file to your log bucket.
If you disable logging, we don't remove the read/write permissions for the data feeds account on either the bucket or the log files. If you want, you can do that yourself.
How to Change Logging Settings
You can enable or disable logging, change the Amazon S3 bucket where your logs are stored, and change the prefix for log files using the CloudFront console or using the CloudFront API:
For information about updating a download or a streaming distribution using the CloudFront console, see Listing, Viewing, and Updating CloudFront Distributions.
For information about updating a download distribution using the CloudFront API, go to PUT Distribution Config in the Amazon CloudFront API Reference.
For information about updating a streaming distribution using the CloudFront API, go to PUT Streaming Distribution Config in the Amazon CloudFront API Reference.
Your changes to logging settings take effect within 12 hours.
To use the CloudFront API to change access log settings for:
download distributions, you must use the 2009-04-02 or later version of the API.
streaming distributions, you must use the 2010-05-01 or later version of the API.
cookies, you must use the 2012-07-01 or later version of the API.
Note
To enable easier listing of keys in a bucket, Amazon S3 users commonly use a prefix followed by a slash (/) as a delimiter.
How to Delete Log Files from an Amazon S3 Bucket
CloudFront does not automatically delete log files from the Amazon S3 bucket that you specified when you enabled logging. For information about deleting log files from an Amazon S3 bucket, see the applicable Amazon S3 documentation:
Using the Amazon S3 console: See Deleting an Object in the Amazon Simple Storage Service Console User Guide.
Using the REST API: See DELETE Object in the Amazon Simple Storage Service API Reference.
Using the SOAP API: See DeleteObject in the Amazon Simple Storage Service API Reference.
File Name Format and Timing of File Delivery
The filename follows this format (with the date and hour in UTC):
{bucket-name}.s3.amazonaws.com/{optional-prefix}{distribution-ID}.{YYYY}-{MM}-{DD}-{HH}.{unique-ID}.gz
For example, if your bucket name is mylogs and your prefix is myprefix/,
your filenames look similar to this:
mylogs.s3.amazonaws.com/myprefix/EMLARXS9EXAMPLE.2012-07-01-20.RT4KCN4SGK9.gz
CloudFront compresses each log file using gzip and saves it in the Amazon S3 bucket that you specify. Typically, CloudFront saves log files within 24 hours after receiving the corresponding requests. Depending on how many users access your content, CloudFront can save several log files per hour.
Note
If no users access your content during a given hour, you don't receive any log files for that hour.
Log File Format
Each entry in a log file gives details about a single end user request for an object. The log files for download and for streaming distributions are not identical, but both types of log files:
Use the W3C extended log file format. (For more information, go to http://www.w3.org/TR/WD-logfile.html.)
Contain tab-separated values.
Contain records that are not necessarily in chronological order.
Contain two header lines: one with the file-format version, and another that lists the W3C fields included in each record.
Substitute URL-encoded equivalents for spaces and non-standard characters in field values.
These non-standard characters consist of all ASCII codes below 32 and above 127, plus the characters in the following table. The URL encoding standard is RFC 1738. For more information, go to http://www.ietf.org/rfc/rfc1738.txt.
URL-Encoded Value | Character |
|---|---|
%3C | < |
%3E | > |
%22 | " |
%23 | # |
%25 | % |
%7B | { |
%7D | } |
%7C | | |
%5C | \ |
%5E | ^ |
%7E | ~ |
%5B | [ |
%5D | ] |
%60 | ` |
%27 | ' |
%20 | space |
Download Distribution File Format
The log file for a download distribution includes the following fields in the listed order.
| Field | Description |
|---|---|
| date | The date (UTC) on which the event occurred, for example, 2009-03-10. |
| time | Time when the server finished processing the request (UTC), for example, 01:42:39. |
| x-edge-location | The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations may change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront. |
| sc-bytes | Server to client bytes, for example, 1045619. |
| c-ip | Client IP, for example, 192.0.2.183. |
| cs-method | HTTP access method. |
| cs(Host) | DNS name (the CloudFront distribution name specified in the request). If you made the request to a CNAME, the DNS name field will contain the underlying distribution DNS name, not the CNAME. |
| cs-uri-stem | URI stem (for example, /images/daily-ad.jpg). |
| sc-status | HTTP status code (for example, 200). For more information, see Supported HTTP Status Codes for Custom Origin Servers. |
| cs(Referer) | The referrer. |
| cs(User-Agent) | The user agent. |
| cs-uri-query | The query string portion of the URI that is included on the connect string. When a URI doesn't contain a
query string, the log file contains a single hyphen (-) in the cs-uri-query field for that request.
The encoding standard is RFC 1738. For more information, see
Log File Format. |
| cs(Cookie) | The cookie header in the request, including name-value pairs and the associated attributes.
If you enable cookie logging, CloudFront logs the cookies in all requests regardless of which cookies you choose to
forward to the origin: none, all, or a whitelist of cookie names. When a request doesn't include a cookie header,
the log file contains a single hyphen (-) in the For more information about cookies, see How CloudFront Forwards, Caches, and Logs Cookies. |
| x-edge-result-type | The result type of a request. Result types include:
|
| x-edge-request-id | An encrypted string that uniquely identifies a request. |
Note
Question marks (?) in URLs and query strings are not included in the log.
The following is an example log file for a download distribution.
#Version: 1.0 #Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id 07/01/2012 01:13:11 FRA2 182 192.0.2.10 GET d2819bc28.cloudfront.net /view/my/file.html 200 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC) - zip=98101 RefreshHit MRVMF7KydIvxMWfJIglgwHQwZsbG2IhRJ07sn9AkKUFSHS9EXAMPLE== 07/01/2012 01:13:12 LAX1 2390282 192.0.2.202 GET www.singalong.com /soundtrack/happy.mp3 304 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1) a=b&c=d zip=50158 Hit xGN7KWpVEmB9Dp7ctcVFQC4E-nrcOcEKS3QyAez--06dV7TEXAMPLE==
Streaming Distribution Log File Format
Each record in a streaming access log represents a playback event, for example, connect, play, pause, stop,
disconnect, and so on. As a result, CloudFront generates multiple log records each time a viewer watches a video.
To relate log records that stem from the same stream ID, use the x-sid field.
Note
Some fields are present for all events, whereas others appear only on Play, Stop, Pause, Unpause, and Seek events. When a field isn't relevant for a given event, the log file will contain a single hyphen (-).
The following table describes the fields that are present in each record in the streaming distribution log file, regardless of the type of event. The fields appear in the log in the order listed.
| Field | Description |
|---|---|
| date | Date (UTC) on which the event occurred. |
| time | Time when the server received the request (UTC), for example, 01:42:39. |
| x-edge-location | The edge location where the playback event occurred. Each edge location is identified by a three-letter code and an arbitrarily assigned number, for example, DFW3. The three-letter code typically corresponds with the International Air Transport Association airport code for an airport near the edge location. (These abbreviations may change in the future.) For a list of edge locations, see the Amazon CloudFront detail page, http://aws.amazon.com/cloudfront. |
| c-ip | Client IP, for example, 192.0.2.183. |
| x-event | The event type. This is a Connect, Disconnect, Play, Stop, Pause, Unpause, or Seek event. |
| sc-bytes | The running total number of bytes sent from the server to the client, up to the time of the event. |
| x-cf-status | A code indicating the status of the event. Currently, "OK" is the only value for this field. New functionality in the future could require new status codes. |
| x-cf-client-id | An opaque string identifier that can be used to differentiate clients This value is unique for each connection. |
| cs-uri-stem | The stem portion of the URI, including the application and the application instance. This is sometimes referred to as the FMS connect string. For example, rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st. |
| cs-uri-query | The query string portion of the URI that is included on the connect string. |
| c-referrer | The URI of the referrer. |
| x-page-url | The URL of the page from which the SWF is linked. |
| c-user-agent | The user agent. |
The following fields are present only on Play, Stop, Pause, Unpause, and Seek events. For other events, these fields will contain a single dash (-). These fields appear in the log after the field listed above, and they appear in the order listed.
| Field | Description |
|---|---|
| x-sname | The stream name. |
| x-sname-query | The stream query string, if any. |
| x-file-ext | The stream type, for instance, FLV. |
| x-sid | The stream ID. This is a unique integer identifier for the connection. |
Note
Question marks (?) in URLs and query strings are not included in the log.
The following is an example of a log file for a streaming distribution.
#Version: 1.0 #Fields: date time x-edge-location c-ip x-event sc-bytes x-cf-status x-cf-client-id cs-uri-stem cs-uri-query c-referrer x-page-url c-user-agent x-sname x-sname-query x-file-ext x-sid 2010-03-12 23:51:20 SEA4 192.0.2.147 connect 2014 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 - - - - 2010-03-12 23:51:21 SEA4 192.0.2.222 play 3914 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 myvideo p=2&q=4 flv 1 2010-03-12 23:53:44 SEA4 192.0.2.4 stop 323914 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 dir/other/myvideo p=2&q=4 flv 1 2010-03-12 23:53:44 SEA4 192.0.2.103 play 8783724 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 dir/favs/myothervideo p=42&q=14 mp4 2 2010-03-12 23:56:21 SEA4 192.0.2.199 stop 429822014 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 dir/favs/myothervideo p=42&q=14 mp4 2 2010-03-12 23:59:44 SEA4 192.0.2.14 disconnect 429824092 OK bfd8a98bee0840d9b871b7f6ade9908f rtmp://shqshne4jdp4b6.cloudfront.net/cfx/st key=value http://player.longtailvideo.com/player.swf http://www.longtailvideo.com/support/jw-player-setup-wizard?example=204 LNX%2010,0,32,18 - - - -
Charges for Access Logs
Access logging is an optional feature of CloudFront. There is no extra charge for enabling access logging. However, you accrue the usual Amazon S3 charges for storing and accessing the files on Amazon S3 (you can delete them at any time).
Related Topics
