Automatically copy your Amazon S3 Glacier vault archives to an S3 bucket and storage classes
Publication date: December 2023 (last update: October 2024)
Data Transfer from Amazon S3 Glacier Vaults to Amazon S3 is a serverless solution that automates and
optimizes the restore, copy, and transfer process of Amazon Simple Storage Service Glacier (Amazon S3 Glacier) vault
archives. The solution copies all of the vault's archives to a defined Amazon Simple Storage Service
Important
Amazon S3 and Amazon S3 Glacier are different AWS services.
Amazon S3 Glacier is an object storage service for low-cost data archiving and long-term backup. It stores archives in vaults. It doesn't offer storage classes. The Amazon S3 Glacier service provides a console. However, any archive operation, such as upload, download, or deletion, requires you to use the AWS CLI or write code. There is no console support for archive operations.
Amazon S3 is an object storage service for any
type of data. It stores objects in
buckets. It offers different storage classes
for frequent access, infrequent access, archives, and optimized
tiering. You can interact with the Amazon S3 service by using the
Amazon S3 console or
AWS Command Line Interface
The S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive storage classes are features of the Amazon S3 service. The S3 Glacier Flexible Retrieval storage class offers the same features as the Amazon S3 Glacier service. The Amazon S3 Glacier service doesn't offer storage classes.
For example, Saanvi works at AnyCompany Archives. Five years ago, she used the Amazon S3 Glacier service to store scanned copies of historical documents in a vault. AnyCompany just announced that they will have a different online exhibit each month, featuring documents that are stored in the S3 Glacier vault. To address this change of business:
-
Saanvi wants to take advantage of the storage classes offered with the Amazon S3 service, including more flexibility in how files are stored and accessed.
-
Using Data Transfer from Amazon S3 Glacier Vaults to Amazon S3, Saanvi can copy all of her document archives from her S3 Glacier vault to an S3 bucket. She can assign them to the S3 storage classes that best fit her use cases. For example, she can use the S3 Standard storage class for documents that will be featured in the first exhibit and accessed daily, and the S3 Glacier Deep Archive storage class for documents that won't be featured in any of the exhibits.
-
Now that the documents are stored in the Amazon S3 service, Saanvi can also apply S3 Lifecycle configurations, tag her data, and use the Amazon S3 console.
Note
This solution doesn't delete the original archives or the source S3 Glacier vault. You must manually delete the archives and vault. For more information, refer to Deleting an Archive in Amazon S3 Glacier in the Amazon S3 Glacier Developer Guide.
If your source S3 Glacier vault has a Vault Lock policy that prevents deletion, you must delete this policy before deleting the original archives. However, if your Vault Lock policy is in the Locked state, you can't delete it. See S3 Glacier Vault Lock and Abort Vault Lock (DELETE lock-policy) in the Amazon S3 Glacier Developer Guide for more information.
This implementation guide provides an overview of the Data Transfer from Amazon S3 Glacier Vaults to Amazon S3 solution, its reference architecture and components, considerations for planning the deployment, and configuration steps for deploying the solution to the Amazon Web Services (AWS) Cloud.
The intended audience for using this solution's features and capabilities in their environment includes solution architects, business decision makers, DevOps engineers, data scientists, and cloud professionals. Practical experience with the AWS Cloud, Amazon S3 Glacier vaults, Amazon S3 buckets, and Amazon S3 storage classes is preferred.
Use this navigation table to quickly find answers to these questions:
| If you want to . . . | Read . . . |
|---|---|
|
Know the cost for running this solution. The estimated cost for running this solution in the US East (Ohio) Region is USD $153.57 to copy 100,000 S3 Glacier vault archives, totaling 100 TB of data, from an S3 Glacier vault to an S3 bucket. |
Cost |
| Understand the security considerations for this solution. | Security |
|
Know how to plan for quotas for this solution.
This solution uses
AWS Lambda |
Quotas |
| Know which AWS Regions support this solution. | Supported AWS Regions |
| View or download the AWS CloudFormation template included in this solution to automatically deploy the infrastructure resources (the "stack") for this solution. | AWS CloudFormation template |
| Access the source code and optionally use the AWS Cloud Development Kit (AWS CDK) to deploy the solution. | GitHub repository |
