Overview

Customers of all sizes and industries use Amazon Simple Storage Service (Amazon S3) for its scalability, data durability, security, and performance. Amazon S3 Standard, S3 Standard–IA, S3 One Zone-IA, S3 Intelligent-Tiering, S3 Glacier, and S3 Glacier Deep Archive storage classes are all designed to provide 99.999999999% durability of objects over a given year. These services are designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy, and they also regularly verify the integrity of data using checksums.

Amazon S3 employs a combination of mechanisms to detect data corruption including use of Content-MD5 checksums and cyclic redundancy checks (CRCs). Amazon S3 performs these checksums on data at rest and repairs any corruption using redundant data. In addition, the service also calculates checksums on all network traffic to detect corruption of data packets when storing or retrieving data.

Certain AWS customers, for example some focused on digital preservation and archiving, manage records such as national archives and university libraries, and require object checksums to comply with regulations, certifications, or other requirements. These organizations may be subject to requirements such as the National Digital Stewardship Alliance (NDSA) Levels of Digital Preservation (LoP) or ISO 16363/Trusted Digital Repository (TDR).

The Serverless Fixity for Digital Preservation Compliance solution makes it easier for customers who require an on-demand fixity check process to validate the checksums. Using this solution, AWS customers can check the integrity of their objects stored in any Amazon S3 storage class using either the MD5 or SHA1 checksum algorithm without having to incur the cost and complexity of third-party software. Customers can easily perform periodic re-computations and run comparisons of the checksums against previous checksum computation results or against external records of checksum calculation for that object.

Customers can use this solution to start the fixity check process using the AWS Management Console, Amazon API Gateway, or the AWS Command Line Interface (CLI). Customers can also receive Amazon Simple Notification Service (Amazon SNS) notifications with the results of the fixity checks.

Cost

You are responsible for the cost of the AWS services used while running this solution. The total cost for running this solution depends on the number and size of Amazon S3 objects and the storage class of those objects being validated by the fixity solution. For a cost breakdown, see Appendix A.