Managing S3 Intelligent-Tiering

The S3 Intelligent-Tiering storage class delivers automatic storage cost savings in three low-latency and high-throughput access tiers. It also offers optional archive capabilities to help you get the lowest storage costs in the cloud for data that can be accessed in minutes to hours. The S3 Intelligent-Tiering storage class supports all Amazon S3 features, including the following:

• S3 Inventory, for verifying the access tier of objects

• S3 Replication, for replicating data to any AWS Region

• S3 Storage Lens, for viewing storage usage and activity metrics

• Server-side encryption, for protecting object data

• S3 Object Lock, for preventing accidental deletion of data

• AWS PrivateLink, for accessing Amazon S3 through a private endpoint in a virtual private cloud (VPC)

Identifying which S3 Intelligent-Tiering access tier objects are stored in

To get a list of your objects and their corresponding metadata, including their S3 Intelligent-Tiering access tier, you can use Amazon S3 Inventory. S3 Inventory provides CSV, ORC, or Parquet output files that list your objects and their corresponding metadata. You can receive these inventory reports on either a daily or weekly basis for an Amazon S3 bucket or a shared prefix. (Shared prefix refers to objects that have names that begin with a common string.)

Viewing the archive status of an object within S3 Intelligent-Tiering

To receive notice when an object within the S3 Intelligent-Tiering storage class has moved to either the Archive Access tier or the Deep Archive Access tier, you can set up S3 Event Notifications. For more information, see Enabling event notifications.

Amazon S3 can publish event notifications to an Amazon Simple Notification Service (Amazon SNS) topic, an Amazon Simple Queue Service (Amazon SQS) queue, or an AWS Lambda function. For more information, see Amazon S3 Event Notifications.

The following is an example of a message that Amazon S3 sends to publish an s3:IntelligentTiering event. For more information, see Event message structure.

{  
   "Records":[  
      {  
         "eventVersion":"2.3",
         "eventSource":"aws:s3",
         "awsRegion":"us-west-2",
         "eventTime":"1970-01-01T00:00:00.000Z",
         "eventName":"IntelligentTiering",
         "userIdentity":{  
            "principalId":"s3.amazonaws.com"
         },
         "requestParameters":{  
            "sourceIPAddress":"s3.amazonaws.com"
         },
         "responseElements":{  
            "x-amz-request-id":"C3D13FE58DE4C810",
            "x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
         },
         "s3":{  
            "s3SchemaVersion":"1.0",
            "configurationId":"testConfigRule",
            "bucket":{  
               "name":"mybucket",
               "ownerIdentity":{  
                  "principalId":"A3NL1KOZZKExample"
               },
               "arn":"arn:aws:s3:::mybucket"
            },
            "object":{  
               "key":"HappyFace.jpg",
               "size":1024,
               "eTag":"d41d8cd98f00b204e9800998ecf8427e",              
            }
         },
         "intelligentTieringEventData":{
            "destinationAccessTier": "ARCHIVE_ACCESS"
            }
      }
   ]
}

You can also use a HEAD object request to view an object's archive status. If an object is stored in the S3 Intelligent-Tiering storage class and is in one of the archive tiers, the HEAD object response shows the current archive tier. To show the archive tier, the request uses the x-amz-archive-status header.

The following HEAD object request returns the metadata of an object (in this case, my-image.jpg).

HEAD /my-image.jpg HTTP/1.1
Host: bucket.s3.region.amazonaws.com
Date: Wed, 28 Oct 2009 22:32:00 GMT
Authorization: AWS AKIAIOSFODNN7EXAMPLE:02236Q3V0RonhpaBX5sCYVf1bNRuU=

You can also use HEAD object requests to monitor the status of a restore-object request. If the archive restoration is in progress, the HEAD object response includes the x-amz-restore header.

The following sample HEAD object response shows an object archived by using S3 Intelligent-Tiering with a restore request in progress.

HTTP/1.1 200 OK
x-amz-id-2: FSVaTMjrmBp3Izs1NnwBZeu7M19iI8UbxMbi0A8AirHANJBo+hEftBuiESACOMJp
x-amz-request-id: E5CEFCB143EB505A
Date: Fri, 13 Nov 2020 00:28:38 GMT
Last-Modified: Mon, 15 Oct 2012 21:58:07 GMT
ETag: "1accb31fcf202eba0c0f41fa2f09b4d7"
x-amz-storage-class: 'INTELLIGENT_TIERING'
x-amz-archive-status: 'ARCHIVE_ACCESS'
x-amz-restore: 'ongoing-request="true"'
x-amz-restore-request-date: 'Fri, 13 Nov 2020 00:20:00 GMT'
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 300
Server: AmazonS3

Restoring objects from the S3 Intelligent-Tiering Archive Access and Deep Archive Access tiers

To access objects in the S3 Intelligent-Tiering Archive Access and Deep Archive Access tiers, you must initiate the restore request, and then wait until the object is moved into the Frequent Access tier. For more information about archived objects, see Working with archived objects.

When you restore an object from the Archive Access tier or Deep Archive Access tier, the object moves back into the Frequent Access tier. Afterwards, if the object isn't accessed for 30 consecutive days, it automatically moves into the Infrequent Access tier. Then, after a minimum of 90 consecutive days of no access, the object moves into the Archive Access tier. After a minimum of 180 consecutive days of no access, the object moves into the Deep Archive Access tier. For more information, see How S3 Intelligent-Tiering works.

Note

When you restore an object from S3 Intelligent-Tiering there are no retrieval charges for Standard or Bulk retrievals. Subsequent restore requests called on archived objects that have already been restored are billed as a GET request. For information about pricing, see Amazon S3 pricing.

The following table summarizes archived object retrieval speeds.

Storage class or tier	Expedited	Standard (with Batch Operations)	Standard (without Batch Operations)	Bulk
S3 Intelligent-Tiering Archive Access	1–5 minutes	Minutes–5 hours	3–5 hours	5–12 hours
S3 Intelligent-Tiering Deep Archive Access	Not available	9-12 hours	Within 12 hours	Within 48 hours

Note

Expedited retrievals are a premium feature available for the S3 Intelligent-Tiering Archive Access tier and are charged at the Expedited request and retrieval rate.

For information about Amazon S3 pricing, see Amazon S3 Pricing.

You can restore an archived object by using the Amazon S3 console, S3 Batch Operations, the REST API, and the AWS Command Line Interface (AWS CLI).

Using the S3 console

To restore an object by using the Amazon S3 console

1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.

2. In the left navigation pane, choose Buckets. In the Buckets list, choose the name of the bucket that contains the objects that you want to restore.

3. In the Objects list, select the check box next to one or more of the objects that you are restoring. Choose Actions, and then choose Restore from S3 Intelligent-Tiering Archive Access or Deep Archive Access.

4. Choose Restore.

   Note

   Objects from the S3 Intelligent-Tiering Archive Access and Deep Archive Access tiers are automatically restored to the Frequent Access tier.

Using the AWS CLI

To restore objects from the S3 Intelligent-Tiering Archive Access or Deep Archive Access tiers, use the restore-object command.

The following example command restores the object dir1/example.obj in the DOC-EXAMPLE-BUCKET bucket. To use this example command, replace the user input placeholders with your own information.

aws s3api restore-object --bucket DOC-EXAMPLE-BUCKET --key dir1/example.obj --restore-request '{}'

To monitor the status of your restore-object request, you can use the following example command. To use this example command, replace the user input placeholders with your own information.

aws s3api head-object --bucket DOC-EXAMPLE-BUCKET --key dir1/example.obj

For more information, see restore-object in the AWS CLI Command Reference.

Note

Unlike in the S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage classes, restore requests for S3 Intelligent-Tiering objects don't accept the Days value.

Using the REST API

Amazon S3 provides an API operation for you to initiate an archive restoration. For more information, see RestoreObject in the Amazon Simple Storage Service API Reference.

Using S3 Batch Operations

To restore more than one archived object with a single request, you can use S3 Batch Operations. You provide S3 Batch Operations with a list of objects to operate on. S3 Batch Operations calls the respective API operation to perform the specified operation. A single Batch Operations job can perform the specified operation on billions of objects containing exabytes of data.

To create a Batch Operations job, you must have a manifest that contains only the objects that you want to restore. You can create a manifest by using S3 Inventory, or you can supply a CSV file with the necessary information. For more information, see Specifying a manifest.

Before creating and running S3 Batch Operations jobs, you must grant permissions to Amazon S3 to perform S3 Batch Operations on your behalf. For the required permissions, see Granting permissions for Amazon S3 Batch Operations.

Note

Batch Operations jobs can operate either on S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage class objects or on S3 Intelligent-Tiering Archive Access and Deep Archive Access storage tier objects. Batch Operations can't operate on both types of archived objects in the same job. To restore objects of both types, you must create separate Batch Operations jobs.

For more information about using Batch Operations to restore archive objects, see Restore objects with Batch Operations.

To create an S3 Initiate Restore Object Batch Operations job

1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.

2. In the left navigation pane, choose Batch Operations.

3. Choose Create job.

4. For AWS Region, choose the Region where you want to create your job.

5. Under Manifest format, choose the type of manifest to use.

   • If you choose S3 inventory report, enter the path to the manifest.json object that Amazon S3 generated as part of the CSV-formatted inventory report. If you want to use a manifest version other than the most recent, enter the version ID for the manifest.json object.

   • If you choose CSV, enter the path to a CSV-formatted manifest object. The manifest object must follow the format described in the console. If you want to use a version other than the most recent, you can optionally include the version ID for the manifest object.

6. Choose Next.

7. In the Operation section, choose Restore.

8. In the Restore section, for Restore source, choose Intelligent-Tiering Archive Access tier or Deep Archive Access tier. For Retrieval tier, choose the tier that you want to use.

9. Choose Next.

10. On the Configure additional options page, fill out the following sections:

    • In the Additional options section, provide a description for the job and specify a priority number for the job. Higher numbers indicate a higher priority. For more information, see Assigning job priority.

    • In the Completion report section, select whether Batch Operations should create a completion report. For more information about completion reports, see Completion reports.

    • In the Permissions section, you must grant permissions to Amazon S3 to perform Batch Operations on your behalf. For the required permissions, see Granting permissions for Amazon S3 Batch Operations.

    • (Optional) In the Job tags section, add tags in key-value pairs. For more information, see Controlling access and labeling jobs using tags.

    When you're finished, choose Next.

11. On the Review page, verify the settings. If you need to make changes, choose Previous. Otherwise, choose Create job.

For more information about Batch Operations, see Restore objects with Batch Operations and Creating an S3 Batch Operations job.

Checking the restore status of an object

You can check the progress of your object's restoration on the object's detail page on the Amazon S3 console, the AWS CLI, or the REST API. For more information, see Checking restore status and expiration date.

You can be notified of object restoration completion by using the s3:ObjectRestore:Completed action with the Amazon S3 Event Notifications feature.