使用總和檢查碼搭配使用開發套件的 Amazon S3 物件 AWS - Amazon Simple Storage Service

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

使用總和檢查碼搭配使用開發套件的 Amazon S3 物件 AWS

下列程式碼範例示範如何使用總和檢查搭配 Amazon S3 物件。

Java
適用於 Java 2.x 的 SDK
注意

還有更多關於 GitHub。尋找完整範例,並了解如何在AWS 設定和執行程式碼範例儲存庫

這些程式碼範例使用下列匯入的子集。

import org.slf4j.Logger; import org.slf4j.LoggerFactory; import software.amazon.awssdk.core.exception.SdkException; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.ChecksumAlgorithm; import software.amazon.awssdk.services.s3.model.ChecksumMode; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.GetObjectResponse; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.model.UploadPartResponse; import software.amazon.awssdk.services.s3.waiters.S3Waiter; import software.amazon.awssdk.transfer.s3.S3TransferManager; import software.amazon.awssdk.transfer.s3.model.FileUpload; import software.amazon.awssdk.transfer.s3.model.UploadFileRequest; import java.io.FileInputStream; import java.io.IOException; import java.io.RandomAccessFile; import java.net.URISyntaxException; import java.net.URL; import java.nio.ByteBuffer; import java.nio.file.Paths; import java.security.DigestInputStream; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.util.ArrayList; import java.util.Base64; import java.util.List; import java.util.Objects; import java.util.UUID;

當您建置 PutObjectRequest時,為 putObject 方法指定總和檢查演算法。

public void putObjectWithChecksum() { s3Client.putObject(b -> b .bucket(bucketName) .key(key) .checksumAlgorithm(ChecksumAlgorithm.CRC32), RequestBody.fromString("This is a test")); }

建置時,請驗證getObject方法的總和檢查碼。 GetObjectRequest

public GetObjectResponse getObjectWithChecksum() { return s3Client.getObject(b -> b .bucket(bucketName) .key(key) .checksumMode(ChecksumMode.ENABLED)) .response(); }

當您建置 PutObjectRequest時,為 putObject 方法預先計算總和檢查。

public void putObjectWithPrecalculatedChecksum(String filePath) { String checksum = calculateChecksum(filePath, "SHA-256"); s3Client.putObject((b -> b .bucket(bucketName) .key(key) .checksumSHA256(checksum)), RequestBody.fromFile(Paths.get(filePath))); }

除了 AWS CRT 型 S3 用戶端之外,內容大小超過閾值時使用 S3 Transfer Manager 明確執行分段上傳。預設閾值大小為 8 MB。

您可以指定要讓 SDK 使用的總和檢查演算法。根據預設,SDK 會使用 CRC32 演算法。

public void multipartUploadWithChecksumTm(String filePath) { S3TransferManager transferManager = S3TransferManager.create(); UploadFileRequest uploadFileRequest = UploadFileRequest.builder() .putObjectRequest(b -> b .bucket(bucketName) .key(key) .checksumAlgorithm(ChecksumAlgorithm.SHA1)) .source(Paths.get(filePath)) .build(); FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest); fileUpload.completionFuture().join(); transferManager.close(); }

使用 S3 客戶端 API 或(S3 AsyncClient API)執行多部分上傳。如果您指定額外的總和檢查,則必須指定在上傳初始化時要使用的演算法。您還必須為每一個分段請求指定演算法,並在每一個分段上傳後提供為其計算的總和檢查。

public void multipartUploadWithChecksumS3Client(String filePath) { ChecksumAlgorithm algorithm = ChecksumAlgorithm.CRC32; // Initiate the multipart upload. CreateMultipartUploadResponse createMultipartUploadResponse = s3Client.createMultipartUpload(b -> b .bucket(bucketName) .key(key) .checksumAlgorithm(algorithm)); // Checksum specified on initiation. String uploadId = createMultipartUploadResponse.uploadId(); // Upload the parts of the file. int partNumber = 1; List<CompletedPart> completedParts = new ArrayList<>(); ByteBuffer bb = ByteBuffer.allocate(1024 * 1024 * 5); // 5 MB byte buffer try (RandomAccessFile file = new RandomAccessFile(filePath, "r")) { long fileSize = file.length(); long position = 0; while (position < fileSize) { file.seek(position); long read = file.getChannel().read(bb); bb.flip(); // Swap position and limit before reading from the buffer. UploadPartRequest uploadPartRequest = UploadPartRequest.builder() .bucket(bucketName) .key(key) .uploadId(uploadId) .checksumAlgorithm(algorithm) // Checksum specified on each part. .partNumber(partNumber) .build(); UploadPartResponse partResponse = s3Client.uploadPart( uploadPartRequest, RequestBody.fromByteBuffer(bb)); CompletedPart part = CompletedPart.builder() .partNumber(partNumber) .checksumCRC32(partResponse.checksumCRC32()) // Provide the calculated checksum. .eTag(partResponse.eTag()) .build(); completedParts.add(part); bb.clear(); position += read; partNumber++; } } catch (IOException e) { System.err.println(e.getMessage()); } // Complete the multipart upload. s3Client.completeMultipartUpload(b -> b .bucket(bucketName) .key(key) .uploadId(uploadId) .multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build())); }

如需 AWS SDK 開發人員指南和程式碼範例的完整清單,請參閱透過 AWS SDK 使用此服務。此主題也包含有關入門的資訊和舊版 SDK 的詳細資訊。