WRITE and READ_ACP permissions for the Amazon S3 Log Delivery group

Enabling Amazon S3 server access logging

Server access logging provides detailed records for the requests that are made to an Amazon S3 bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. It can also help you learn about your customer base and understand your Amazon S3 bill.

By default, Amazon S3 doesn't collect server access logs. When you enable logging, Amazon S3 delivers access logs for a source bucket to a target bucket that you choose. The target bucket must be in the same AWS Region as the source bucket and must not have a default retention period configuration.

An access log record contains details about the requests that are made to a bucket. This information can include the request type, the resources that are specified in the request, and the time and date that the request was processed. For more information about logging basics, see Logging requests using server access logging.

Important

There is no extra charge for enabling server access logging on an Amazon S3 bucket. However, any log files that the system delivers to you will accrue the usual charges for storage. (You can delete the log files at any time.) We do not assess data transfer charges for log file delivery, but we do charge the normal data transfer rate for accessing the log files.

You can enable or disable server access logging by using the Amazon S3 console, Amazon S3 API, the AWS Command Line Interface (AWS CLI), or AWS SDKs.

Before you enable server access logging, consider the following:

In Amazon S3, you can grant permission to deliver access logs through bucket access control lists (ACLs), but not through bucket policy.
Adding deny conditions to a bucket policy might prevent Amazon S3 from delivering access logs.
You can use default bucket encryption on the target bucket only if AES256 (SSE-S3) is selected. SSE-KMS encryption is not supported.
You can't enable S3 Object Lock on the target bucket.

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
In the Buckets list, choose the name of the bucket that you want to enable server access logging for.
Choose Properties.
In the Server access logging section, choose Edit.
Under Server access logging, select Enable.
For Target bucket, enter the name of the bucket that you want to receive the log record objects.

The target bucket must be in the same Region as the source bucket and must not have a default retention period configuration.
Choose Save changes.

When you enable logging on a bucket, the console both enables logging on the source bucket and adds a grant in the target bucket's access control list (ACL) granting write permission to the Log Delivery group.

You can view the logs in the target bucket. After you enable server access logging, it might take a few hours before the logs are delivered to the target bucket. For more information about how and when logs are delivered, see How are logs delivered?.

For more information, see Viewing the properties for an S3 bucket.

To enable logging, you submit a PUT Bucket logging request to add the logging configuration on the source bucket. The request specifies the target bucket and, optionally, the prefix to be used with all log object keys.

The following example identifies logbucket as the target bucket and logs/ as the prefix.


<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
  <LoggingEnabled>
    <TargetBucket>logbucket</TargetBucket>
    <TargetPrefix>logs/</TargetPrefix>
  </LoggingEnabled>
</BucketLoggingStatus>

The log objects are written and owned by the Log Delivery account, and the bucket owner is granted full permissions on the log objects. In addition, you can optionally grant permissions to other users so that they can access the logs. For more information, see PUT Bucket logging.

Amazon S3 also provides the GET Bucket logging API to retrieve logging configuration on a bucket. To delete the logging configuration, you send the PUT Bucket logging request with an empty BucketLoggingStatus.


<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
</BucketLoggingStatus>

You can use either the Amazon S3 API or the AWS SDK wrapper libraries to enable logging on a bucket.


using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
using System;
using System.Threading.Tasks;

namespace Amazon.DocSamples.S3
{
    class ServerAccesLoggingTest
    {
        private const string bucketName = "*** bucket name for which to enable logging ***"; 
        private const string targetBucketName = "*** bucket name where you want access logs stored ***"; 
        private const string logObjectKeyPrefix = "Logs";
        // Specify your bucket region (an example region is shown).
        private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2;
        private static IAmazonS3 client;

        public static void Main()
        {
            client = new AmazonS3Client(bucketRegion);
            EnableLoggingAsync().Wait();
        }

        private static async Task EnableLoggingAsync()
        {
            try
            {
                // Step 1 - Grant Log Delivery group permission to write log to the target bucket.
                await GrantPermissionsToWriteLogsAsync();
                // Step 2 - Enable logging on the source bucket.
                await EnableDisableLoggingAsync();
            }
            catch (AmazonS3Exception e)
            {
                Console.WriteLine("Error encountered on server. Message:'{0}' when writing an object", e.Message);
            }
            catch (Exception e)
            {
                Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message);
            }
        }

        private static async Task GrantPermissionsToWriteLogsAsync()
        {
            var bucketACL = new S3AccessControlList();
            var aclResponse = client.GetACL(new GetACLRequest { BucketName = targetBucketName });
            bucketACL = aclResponse.AccessControlList;
            bucketACL.AddGrant(new S3Grantee { URI = "http://acs.amazonaws.com/groups/s3/LogDelivery" }, S3Permission.WRITE);
            bucketACL.AddGrant(new S3Grantee { URI = "http://acs.amazonaws.com/groups/s3/LogDelivery" }, S3Permission.READ_ACP);
            var setACLRequest = new PutACLRequest
            {
                AccessControlList = bucketACL,
                BucketName = targetBucketName
            };
            await client.PutACLAsync(setACLRequest);
        }

        private static async Task EnableDisableLoggingAsync()
        {
            var loggingConfig = new S3BucketLoggingConfig
            {
                TargetBucketName = targetBucketName,
                TargetPrefix = logObjectKeyPrefix
            };

            // Send request.
            var putBucketLoggingRequest = new PutBucketLoggingRequest
            {
                BucketName = bucketName,
                LoggingConfig = loggingConfig
            };
            await client.PutBucketLoggingAsync(putBucketLoggingRequest);
        }
    }
}

We recommend that you create a dedicated logging bucket in each AWS Region that you have S3 buckets in. Then have the Amazon S3 access log delivered to that S3 bucket. For more information and examples, see put-bucket-logging in the AWS CLI Reference.

Example — Enable access logs with five buckets across two Regions

In this example, you have the following five buckets:

1-awsexamplebucket1-us-east-1
2-awsexamplebucket1-us-east-1
3-awsexamplebucket1-us-east-1
1-awsexamplebucket1-us-west-2
2-awsexamplebucket1-us-west-2

Create two logging buckets in the following Regions:
- awsexamplebucket1-logs-us-east-1
- awsexamplebucket1-logs-us-west-2
Then enable the Amazon S3 access logs as follows:
- 1-awsexamplebucket1-us-east-1 logs to the S3 bucket awsexamplebucket1-logs-us-east-1 with prefix 1-awsexamplebucket1-us-east-1
- 2-awsexamplebucket1-us-east-1 logs to the S3 bucket awsexamplebucket1-logs-us-east-1 with prefix 2-awsexamplebucket1-us-east-1
- 3-awsexamplebucket1-us-east-1 logs to the S3 bucket awsexamplebucket1-logs-us-east-1 with prefix 3-awsexamplebucket1-us-east-1
- 1-awsexamplebucket1-us-west-2 logs to the S3 bucket awsexamplebucket1-logs-us-west-2 with prefix 1-awsexamplebucket1-us-west-2
- 2-awsexamplebucket1-us-west-2 logs to the S3 bucket awsexamplebucket1-logs-us-west-2 with prefix 2-awsexamplebucket1-us-west-2

You can then enable the Amazon S3 access logs using the following methods:

Using the Enabling Amazon S3 server access logging or,
Using the AWS CLI put-bucket-logging command to programmatically enable access logs on a bucket using the following commands:

First, grant Amazon S3 permission using put-bucket-acl.



 aws s3api put-bucket-acl --bucket awsexamplebucket1-logs  --grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDelivery

Then, apply the logging policy.



aws s3api put-bucket-logging --bucket awsexamplebucket1 --bucket-logging-status file://logging.json

Logging.json is a JSON document in the current folder that contains the logging policy.



  {
      "LoggingEnabled": {
          "TargetBucket": "awsexamplebucket1-logs",
          "TargetPrefix": "awsexamplebucket1/",
          "TargetGrants": [
               {
                  "Grantee": {
                      "Type": "AmazonCustomerByEmail",
                      "EmailAddress": "user@example.com"
                   },
                  "Permission": "FULL_CONTROL"
               }
           ]
       }
   }

Note

The put-bucket-acl command is required to grant the Amazon S3 log delivery system the necessary permissions (write and read-acp permissions).

Use a bash script to add access logging for all the buckets in your account.



  loggingBucket='awsexamplebucket1-logs'
  region='us-west-2'
  
  
  # Create Logging bucket
  aws s3 mb s3://$loggingBucket --region $region
  
  aws s3api put-bucket-acl --bucket $loggingBucket --grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDelivery
  
  # List buckets in this account
  buckets="$(aws s3 ls | awk '{print $3}')"
  
  # Put bucket logging on each bucket
  for bucket in $buckets
      do printf '{
     "LoggingEnabled": {
         "TargetBucket": "%s",
         "TargetPrefix": "%s/"
          }
      }' "$loggingBucket" "$bucket"  > logging.json
      aws s3api put-bucket-logging --bucket $bucket --bucket-logging-status file://logging.json
      echo "$bucket done"
  done
  
  rm logging.json
  
  echo "Complete"

Note

This only works if all your buckets are in the same Region. If you have buckets in multiple Regions, you must adjust the script.

WRITE and READ_ACP permissions for the Amazon S3 Log Delivery group

Amazon S3 writes the log files to the target bucket as a member of the predefined Amazon S3 group Log Delivery. These writes are subject to the usual access control restrictions.

If you enable server access logging using the S3 console, S3 automatically updates your bucket access control list (ACL) to grant access to the S3 Log Delivery Group. You don't need to manually grant ACL permissions. For information about ACLs, see Access control list (ACL) overview.

The AWS CLI and AWS SDK examples on this page include steps for granting ACL permissions to the S3 Log Delivery Group. If you use these examples, you don't have to manually grant ACL permissions to the Log Delivery Group.

If you enable server access logging programmatically and don't specify ACL permissions, you must grant s3:GetObjectAcl and s3:PutObject permissions to this group by adding WRITE and READ_ACP grants to the access control list (ACL) of the target bucket. The Log Delivery group is represented by the following URL.


http://acs.amazonaws.com/groups/s3/LogDelivery

To grant WRITE and READ_ACP (ACL read) permissions, add the following grants to the target bucket ACL.


<Grant>
    <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:type="Group">
        <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> 
    </Grantee>
    <Permission>WRITE</Permission>
</Grant>
<Grant>
    <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:type="Group">
        <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> 
    </Grantee>
    <Permission>READ_ACP</Permission>
</Grant>

For examples of adding ACL grants programmatically, see Configuring ACLs.

Warning Javascript is disabled or is unavailable in your browser.

To use the AWS Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Logging server access

Log format