Enabling Amazon S3 server access logging
Server access logging provides detailed records for the requests that are made to an Amazon S3 bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. It can also help you learn about your customer base and understand your Amazon S3 bill.
By default, Amazon S3 doesn't collect server access logs. When you enable logging, Amazon S3 delivers access logs for a source bucket to a target bucket that you choose. The target bucket must be in the same AWS Region as the source bucket and must not have a default retention period configuration.
An access log record contains details about the requests that are made to a bucket. This information can include the request type, the resources that are specified in the request, and the time and date that the request was processed. For more information about logging basics, see Logging requests using server access logging.
There is no extra charge for enabling server access logging on an Amazon S3 bucket. However, any log files that the system delivers to you will accrue the usual charges for storage. (You can delete the log files at any time.) We do not assess data transfer charges for log file delivery, but we do charge the normal data transfer rate for accessing the log files.
You can enable or disable server access logging by using the Amazon S3 console, Amazon S3 API, the AWS Command Line Interface (AWS CLI), or AWS SDKs.
Before you enable server access logging, consider the following:
-
In Amazon S3, you can grant permission to deliver access logs through bucket access control lists (ACLs), but not through bucket policy.
-
Adding deny conditions to a bucket policy might prevent Amazon S3 from delivering access logs.
-
You can use default bucket encryption on the target bucket only if AES256 (SSE-S3) is selected. SSE-KMS encryption is not supported.
-
You can't enable S3 Object Lock on the target bucket.
-
Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/
. -
In the Buckets list, choose the name of the bucket that you want to enable server access logging for.
-
Choose Properties.
-
In the Server access logging section, choose Edit.
-
Under Server access logging, select Enable.
-
For Target bucket, enter the name of the bucket that you want to receive the log record objects.
The target bucket must be in the same Region as the source bucket and must not have a default retention period configuration.
-
Choose Save changes.
When you enable logging on a bucket, the console both enables logging on the source bucket and adds a grant in the target bucket's access control list (ACL) granting write permission to the Log Delivery group.
You can view the logs in the target bucket. After you enable server access logging, it might take a few hours before the logs are delivered to the target bucket. For more information about how and when logs are delivered, see How are logs delivered?.
For more information, see Viewing the properties for an S3 bucket.
To enable logging, you submit a PUT Bucket logging request to add the logging configuration on the source bucket. The request specifies the target bucket and, optionally, the prefix to be used with all log object keys.
The following example identifies logbucket
as the target bucket and
logs/
as the prefix.
<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01"> <LoggingEnabled> <TargetBucket>logbucket</TargetBucket> <TargetPrefix>logs/</TargetPrefix> </LoggingEnabled> </BucketLoggingStatus>
The log objects are written and owned by the Log Delivery account, and the bucket owner is granted full permissions on the log objects. In addition, you can optionally grant permissions to other users so that they can access the logs. For more information, see PUT Bucket logging.
Amazon S3 also provides the GET Bucket
logging API to retrieve logging configuration on a bucket. To delete the logging
configuration, you send the PUT Bucket logging request with an empty
BucketLoggingStatus
.
<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01"> </BucketLoggingStatus>
You can use either the Amazon S3 API or the AWS SDK wrapper libraries to enable logging on a bucket.
using Amazon; using Amazon.S3; using Amazon.S3.Model; using System; using System.Threading.Tasks; namespace Amazon.DocSamples.S3 { class ServerAccesLoggingTest { private const string bucketName = "*** bucket name for which to enable logging ***"; private const string targetBucketName = "*** bucket name where you want access logs stored ***"; private const string logObjectKeyPrefix = "Logs"; // Specify your bucket region (an example region is shown). private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2; private static IAmazonS3 client; public static void Main() { client = new AmazonS3Client(bucketRegion); EnableLoggingAsync().Wait(); } private static async Task EnableLoggingAsync() { try { // Step 1 - Grant Log Delivery group permission to write log to the target bucket. await GrantPermissionsToWriteLogsAsync(); // Step 2 - Enable logging on the source bucket. await EnableDisableLoggingAsync(); } catch (AmazonS3Exception e) { Console.WriteLine("Error encountered on server. Message:'{0}' when writing an object", e.Message); } catch (Exception e) { Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message); } } private static async Task GrantPermissionsToWriteLogsAsync() { var bucketACL = new S3AccessControlList(); var aclResponse = client.GetACL(new GetACLRequest { BucketName = targetBucketName }); bucketACL = aclResponse.AccessControlList; bucketACL.AddGrant(new S3Grantee { URI = "http://acs.amazonaws.com/groups/s3/LogDelivery" }, S3Permission.WRITE); bucketACL.AddGrant(new S3Grantee { URI = "http://acs.amazonaws.com/groups/s3/LogDelivery" }, S3Permission.READ_ACP); var setACLRequest = new PutACLRequest { AccessControlList = bucketACL, BucketName = targetBucketName }; await client.PutACLAsync(setACLRequest); } private static async Task EnableDisableLoggingAsync() { var loggingConfig = new S3BucketLoggingConfig { TargetBucketName = targetBucketName, TargetPrefix = logObjectKeyPrefix }; // Send request. var putBucketLoggingRequest = new PutBucketLoggingRequest { BucketName = bucketName, LoggingConfig = loggingConfig }; await client.PutBucketLoggingAsync(putBucketLoggingRequest); } } }
We recommend that you create a dedicated logging bucket in each AWS Region that you
have S3 buckets in. Then have the Amazon S3 access log delivered to that S3 bucket.
For more
information and examples, see put-bucket-logging
Example — Enable access logs with five buckets across two Regions
In this example, you have the following five buckets:
-
1-awsexamplebucket1-us-east-1
-
2-awsexamplebucket1-us-east-1
-
3-awsexamplebucket1-us-east-1
-
1-awsexamplebucket1-us-west-2
-
2-awsexamplebucket1-us-west-2
-
Create two logging buckets in the following Regions:
-
awsexamplebucket1-logs-us-east-1
-
awsexamplebucket1-logs-us-west-2
-
-
Then enable the Amazon S3 access logs as follows:
-
1-awsexamplebucket1-us-east-1
logs to the S3 bucketawsexamplebucket1-logs-us-east-1
with prefix1-awsexamplebucket1-us-east-1
-
2-awsexamplebucket1-us-east-1
logs to the S3 bucketawsexamplebucket1-logs-us-east-1
with prefix2-awsexamplebucket1-us-east-1
-
3-awsexamplebucket1-us-east-1
logs to the S3 bucketawsexamplebucket1-logs-us-east-1
with prefix3-awsexamplebucket1-us-east-1
-
1-awsexamplebucket1-us-west-2
logs to the S3 bucketawsexamplebucket1-logs-us-west-2
with prefix1-awsexamplebucket1-us-west-2
-
2-awsexamplebucket1-us-west-2
logs to the S3 bucketawsexamplebucket1-logs-us-west-2
with prefix2-awsexamplebucket1-us-west-2
-
-
You can then enable the Amazon S3 access logs using the following methods:
-
Using the Enabling Amazon S3 server access logging or,
-
Using the AWS CLI put-bucket-logging command to programmatically enable access logs on a bucket using the following commands:
-
First, grant Amazon S3 permission using
put-bucket-acl
.aws s3api put-bucket-acl --bucket awsexamplebucket1-logs --grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDelivery
-
Then, apply the logging policy.
aws s3api put-bucket-logging --bucket awsexamplebucket1 --bucket-logging-status file://logging.json
Logging.json
is a JSON document in the current folder that contains the logging policy.{ "LoggingEnabled": { "TargetBucket": "awsexamplebucket1-logs", "TargetPrefix": "awsexamplebucket1/", "TargetGrants": [ { "Grantee": { "Type": "AmazonCustomerByEmail", "EmailAddress": "user@example.com" }, "Permission": "FULL_CONTROL" } ] } }
Note The
put-bucket-acl
command is required to grant the Amazon S3 log delivery system the necessary permissions (write and read-acp permissions). -
Use a bash script to add access logging for all the buckets in your account.
loggingBucket='awsexamplebucket1-logs' region='us-west-2' # Create Logging bucket aws s3 mb s3://$loggingBucket --region $region aws s3api put-bucket-acl --bucket $loggingBucket --grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDelivery # List buckets in this account buckets="$(aws s3 ls | awk '{print $3}')" # Put bucket logging on each bucket for bucket in $buckets do printf '{ "LoggingEnabled": { "TargetBucket": "%s", "TargetPrefix": "%s/" } }' "$loggingBucket" "$bucket" > logging.json aws s3api put-bucket-logging --bucket $bucket --bucket-logging-status file://logging.json echo "$bucket done" done rm logging.json echo "Complete"
Note This only works if all your buckets are in the same Region. If you have buckets in multiple Regions, you must adjust the script.
-
WRITE and READ_ACP permissions for the Amazon S3 Log Delivery group
Amazon S3 writes the log files to the target bucket as a member of the predefined Amazon S3 group Log Delivery. These writes are subject to the usual access control restrictions.
If you enable server access logging using the S3 console, S3 automatically updates your bucket access control list (ACL) to grant access to the S3 Log Delivery Group. You don't need to manually grant ACL permissions. For information about ACLs, see Access control list (ACL) overview.
The AWS CLI and AWS SDK examples on this page include steps for granting ACL permissions to the S3 Log Delivery Group. If you use these examples, you don't have to manually grant ACL permissions to the Log Delivery Group.
If you enable server access logging programmatically and don't specify ACL permissions,
you must grant s3:GetObjectAcl
and s3:PutObject
permissions to this
group by adding WRITE
and READ_ACP
grants to the access control list
(ACL) of the target bucket. The Log Delivery group is represented by the following
URL.
http://acs.amazonaws.com/groups/s3/LogDelivery
To grant WRITE
and READ_ACP
(ACL read) permissions, add the
following grants to the target bucket ACL.
<Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"> <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> </Grantee> <Permission>WRITE</Permission> </Grant> <Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"> <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> </Grantee> <Permission>READ_ACP</Permission> </Grant>
For examples of adding ACL grants programmatically, see Configuring ACLs.