Object Lifecycle Management
This section provides an overview of the Amazon S3 lifecycle feature that you can use to manage lifecycle of objects in your bucket.
What Is Lifecycle Configuration?
You manage an object's lifecycle by using a lifecycle configuration, which defines how Amazon S3 manages objects during their lifetime. Lifecycle configuration enables you to simplify the lifecycle management of your objects, such as automated transition of less-frequently accessed objects to low-cost storage alternatives and scheduled deletions. You can configure as many as 1000 lifecycle rules per bucket.
You can define lifecycle configuration rules for objects that have a well-defined lifecycle. You can use lifecycle configurations for objects you want to switch to different storage classes or delete during their lifecycle, for example:
If you are uploading periodic logs to your bucket, your application might need these logs for a week or a month after creation, and after that you might want to delete them.
Some documents are frequently accessed for a limited period of time. After that, these documents are less frequently accessed. Over time, you might not need real-time access to these objects, but your organization or regulations might require you to archive them for a longer period and then optionally delete them later.
You might also upload some types of data to Amazon S3 primarily for archival purposes, for example digital media archives, financial and healthcare records, raw genomics sequence data, long-term database backups, and data that must be retained for regulatory compliance.
How Do I Configure a Lifecycle?
You can specify a lifecycle configuration as XML. A lifecycle configuration comprises a set of rules with predefined actions that you want Amazon S3 to perform on objects during their lifetime. These actions include:
Transition actions in which you define when objects transition to another Amazon S3 storage class. For example, you may choose to transition objects to the STANDARD_IA (IA, for infrequent access) storage class 30 days after creation, or archive objects to the GLACIER storage class one year after creation.
Expiration actions in which you specify when the objects expire. Then, Amazon S3 deletes the expired objects on your behalf.
For more information about lifecycle rules, see Lifecycle Configuration Elements.
Amazon S3 stores the configuration as a "lifecycle" subresource attached to your
bucket. Using the Amazon S3 API, you can
DELETE a lifecycle configuration. For more information, see PUT Bucket lifecycle, GET Bucket lifecycle, or
DELETE Bucket lifecycle. You can also configure the lifecycle by using the Amazon S3 console or
programmatically by using the AWS SDK wrapper libraries, and if you need to you can also
make the REST API calls directly. Then, Amazon S3 applies the lifecycle rules to all or
specific objects identified in the rule.
Transitioning Objects: General Considerations
You can add rules in a lifecycle configuration to transition objects to another Amazon S3 storage class. For example, you might transition objects to the STANDARD_IA storage class when you know those objects are infrequently accessed. You might also want to archive objects that don't need real-time access to the GLACIER storage class. The following sections describe transitioning related considerations and constraints.
In a lifecycle configuration you can define rules to transition objects from one storage class to another. The following are supported transitions:
From the STANDARD or REDUCED_REDUNDANCY storage classes to STANDARD_IA. The following constraints apply:
Amazon S3 does not transition objects less than 128 Kilobytes in size to the STANDARD_IA storage class. Cost benefits of transitioning to STANDARD_IA can be realized for larger objects. For smaller objects it is not cost effective and Amazon S3 will not transition them.
Objects must be stored at least 30 days in the current storage class before you can transition them to STANDARD_IA. For example, you cannot create a lifecycle rule to transition objects to the STANDARD_IA storage class one day after creation.
Transitions before the first 30 days are not supported because often younger objects are accessed more frequently or deleted sooner than is suitable for STANDARD_IA.
If you are transitioning noncurrent objects (versioned bucket scenario), you can transition to STANDARD_IA only objects that are at least 30 days noncurrent.
From any storage class to GLACIER.
For more information, see GLACIER Storage Class: Additional Lifecycle Configuration Considerations.
You can combine these rules to manage an object's complete lifecycle, including a first transition to STANDARD_IA, a second transition to GLACIER for archival, and an expiration.
When configuring lifecycle, the API will not allow you to create a lifecycle policy in which you specify both of these transitions, but the GLACIER transition occurs less than 30 days after the STANDARD_IA transition. This is because such a lifecycle policy may increase costs because of the minimum 30 day storage charge associated with the STANDARD_IA storage class. For more information about cost considerations, see Amazon S3 Pricing.
For example, suppose the objects you create have a well-defined lifecycle. Initially the objects are frequently accessed for a period of 30 days. After the initial period, the frequency of access diminishes where objects are infrequently accessed for up to 90 days. After that, the objects are no longer needed. You may choose to archive or delete them. You can use a lifecycle configuration to define transition and expiration of objects that matches this example scenario (transition to STANDARD_IA 30 days after creation and transition to GLACIER 90 days after creation, and perhaps expire them after certain number of days). As you tier down the object's storage class in the transition, you can benefit from the storage cost savings. For more information about cost considerations, see Amazon S3 Pricing.
You can think of lifecycle transitions as supporting storage class tiers (see Storage Classes), which offer different costs and benefits. You may choose to transition an object to another storage class in the object's lifetime for cost saving considerations and lifecycle configuration enables you to do that. For example, to manage storage costs, you might configure lifecycle to change an object's storage class from the STANDARD, which is most available and durable storage class, to the STANDARD_IA (IA, for infrequent access), and then to the GLACIER storage class (where the objects are archived and only available after you restore). These transitions can lower your storage costs.
The following are not supported transitions:
You cannot transition from STANDARD_IA to STANDARD or REDUCED_REDUNDANCY.
You cannot transition from GLACIER to any other storage class.
You cannot transition from any storage class to REDUCED_REDUNDANCY.
Transitioning to the GLACIER storage class (Object Archival)
Using lifecycle configuration, you can transition objects to the
GLACIER storage class—that is, archive data to Amazon Glacier, a
lower-cost storage solution. Before you archive objects, note the following:
Objects in the
GLACIERstorage class are not available in real time.
Archived objects are Amazon S3 objects, but before you can access an archived object, you must first restore a temporary copy of it. The restored object copy is available only for the duration you specify in the restore request. After that, Amazon S3 deletes the temporary copy, and the object remains archived in Amazon Glacier.
Note that object restoration from an archive can take up to five hours.
You can restore an object by using the Amazon S3 console or programmatically by using the AWS SDKs wrapper libraries or the Amazon S3 REST API in your code. For more information, see POST Object restore.
The transition of objects to the
GLACIERstorage class is one-way.
You cannot use a lifecycle configuration rule to convert the storage class of an object from
GLACIERto Standard or RRS. If you want to change the storage class of an already archived object to either Standard or RRS, you must use the restore operation to make a temporary copy first. Then use the copy operation to overwrite the object as a STANDARD, STANDARD_IA, or REDUCED_REDUNDANCY object.
GLACIERstorage class objects are visible and available only through Amazon S3, not through Amazon Glacier.
Amazon S3 stores the archived objects in Amazon Glacier; however, these are Amazon S3 objects, and you can access them only by using the Amazon S3 console or the API. You cannot access the archived objects through the Amazon Glacier console or the API.
Expiring Objects: General Considerations
When an object reaches the end of its lifetime, Amazon S3 queues it for removal and removes it asynchronously. There may be a delay between the expiration date and the date at which Amazon S3 removes an object. You are not charged for storage time associated with an object that has expired.
There are additional cost considerations if you put lifecycle policy to expire objects that have been in STANDARD_IA for less than 30 days, or GLACIER for less than 90 days. For more information about cost considerations, see Amazon S3 Pricing.
Lifecycle and Other Bucket Configurations
In addition to lifecycle configuration your bucket can have other configurations associated. This is section explains how lifecycle configuration relates to other bucket configurations.
Lifecycle and Versioning
You can add lifecycle configuration to nonversioned buckets and versioning-enabled buckets. For more information, see Object Versioning. A versioning-enabled bucket maintains one current and zero or more noncurrent object versions. You can define separate lifecycle rules for current and noncurrent versions.
Lifecycle and MFA Enabled Buckets
Lifecycle configuration on MFA-enabled buckets is not supported.
Lifecycle and Logging
If you have logging enabled on your bucket, Amazon S3 reports the results of expiration action as follows:
If the lifecycle expiration action results in Amazon S3 permanently removing the object, Amazon S3 reports it as operation
S3.EXPIRE.OBJECTin the log record.
For a versioning-enabled bucket, if the lifecycle expiration action results in a logical deletion of current version, in which Amazon S3 adds a delete marker, Amazon S3 reports the logical deletion as operation
S3.CREATE.DELETEMARKERin the log record. For more information, see Object Versioning.
When Amazon S3 transitions object to the GLACIER storage class it reports it as operation
S3.TRANSITION.OBJECTin the log record to indicate it has initiated the operation. When it is transition to the STANDARD_IA storage class, it is reported as