Work with Amazon S3 objects - AWS SDK for Java version 2

Work with Amazon S3 objects

An Amazon S3 object represents a file or collection of data. Every object must be contained in a bucket.

Note

Best Practice

We recommend that you enable the AbortIncompleteMultipartUpload lifecycle rule on your Amazon S3 buckets.

This rule directs Amazon S3 to abort multipart uploads that don’t complete within a specified number of days after being initiated. When the set time limit is exceeded, Amazon S3 aborts the upload and then deletes the incomplete upload data.

For more information, see Lifecycle Configuration for a Bucket with Versioning in the Amazon S3 User Guide.

Note

These code snippets assume that you understand the material in Using the AWS SDK for Java 2.0, and have configured default AWS credentials using the information in Set up AWS credentials and region for development.

Upload an object

Build a PutObjectRequest and supply a bucket name and key name. Then use the S3Client’s putObject method with a RequestBody that contains the object content and the PutObjectRequest object. The bucket must exist, or the service will return an error.

Imports

import java.io.IOException; import java.nio.ByteBuffer; import java.nio.file.Paths; import java.util.Random; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.core.sync.ResponseTransformer;

Code

Region region = Region.US_WEST_2; s3 = S3Client.builder().region(region).build(); String bucket = "bucket" + System.currentTimeMillis(); String key = "key"; createBucket(s3,bucket, region); // Put Object s3.putObject(PutObjectRequest.builder().bucket(bucket).key(key) .build(), RequestBody.fromByteBuffer(getRandomByteBuffer(10_000)));

See the complete example on GitHub.

Upload objects in multiple parts

Use the S3Client’s createMultipartUpload method to get an upload ID. Then use the uploadPart method to upload each part. Finally, use the S3Client’s completeMultipartUpload method to tell Amazon S3 to merge all the uploaded parts and finish the upload operation.

Imports

import java.io.IOException; import java.nio.ByteBuffer; import java.nio.file.Paths; import java.util.Random; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.core.sync.ResponseTransformer;

Code

// First create a multipart upload and get upload id CreateMultipartUploadRequest createMultipartUploadRequest = CreateMultipartUploadRequest.builder() .bucket(bucketName).key(key) .build(); CreateMultipartUploadResponse response = s3.createMultipartUpload(createMultipartUploadRequest); String uploadId = response.uploadId(); System.out.println(uploadId); // Upload all the different parts of the object UploadPartRequest uploadPartRequest1 = UploadPartRequest.builder().bucket(bucketName).key(key) .uploadId(uploadId) .partNumber(1).build(); String etag1 = s3.uploadPart(uploadPartRequest1, RequestBody.fromByteBuffer(getRandomByteBuffer(5 * MB))).eTag(); CompletedPart part1 = CompletedPart.builder().partNumber(1).eTag(etag1).build(); UploadPartRequest uploadPartRequest2 = UploadPartRequest.builder().bucket(bucketName).key(key) .uploadId(uploadId) .partNumber(2).build(); String etag2 = s3.uploadPart(uploadPartRequest2, RequestBody.fromByteBuffer(getRandomByteBuffer(3 * MB))).eTag(); CompletedPart part2 = CompletedPart.builder().partNumber(2).eTag(etag2).build(); // Finally call completeMultipartUpload operation to tell S3 to merge all uploaded // parts and finish the multipart operation. CompletedMultipartUpload completedMultipartUpload = CompletedMultipartUpload.builder().parts(part1, part2).build(); CompleteMultipartUploadRequest completeMultipartUploadRequest = CompleteMultipartUploadRequest.builder().bucket(bucketName).key(key).uploadId(uploadId) .multipartUpload(completedMultipartUpload).build(); s3.completeMultipartUpload(completeMultipartUploadRequest);

See the complete example on GitHub.

Download an object

Build a GetObjectRequest and supply a bucket name and key name. Use the S3Client’s getObject method, passing it the GetObjectRequest object and a ResponseTransformer object. The ResponseTransformer creates a response handler that writes the response content to the specified file or stream.

The following example specifies a file name to write the object content to.

Imports

import java.io.IOException; import java.nio.ByteBuffer; import java.nio.file.Paths; import java.util.Random; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.core.sync.ResponseTransformer;

Code

s3.getObject(GetObjectRequest.builder().bucket(bucket).key(key).build(), ResponseTransformer.toFile(Paths.get("multiPartKey")));

See the complete example on GitHub.

Delete an object

Build a DeleteObjectRequest and supply a bucket name and key name. Use the S3Client’s deleteObject method, and pass it the name of a bucket and object to delete. The specified bucket and object key must exist, or the service will return an error.

Imports

import java.io.IOException; import java.nio.ByteBuffer; import java.nio.file.Paths; import java.util.Random; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.core.sync.ResponseTransformer;

Code

DeleteObjectRequest deleteObjectRequest = DeleteObjectRequest.builder().bucket(bucket).key(key).build(); s3.deleteObject(deleteObjectRequest);

See the complete example on GitHub.

Copy an object

Build a CopyObjectRequest and supply a bucket name that the object is coped into, a URL encoded string value (see the URLEncoder.encode method), and the key name of the object. Use the S3Client’s copyObject method, and pass the CopyObjectRequest object. The specified bucket and object key must exist, or the service will return an error.

Imports

import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CopyObjectRequest; import software.amazon.awssdk.services.s3.model.CopyObjectResponse; import software.amazon.awssdk.services.s3.model.S3Exception; import java.io.UnsupportedEncodingException; import java.net.URLEncoder; import java.nio.charset.StandardCharsets;

Code

public static String CopyBucketObject (S3Client s3, String fromBucket, String objectKey, String toBucket) { String encodedUrl = null; try { encodedUrl = URLEncoder.encode(fromBucket + "/" + objectKey, StandardCharsets.UTF_8.toString()); } catch (UnsupportedEncodingException e) { System.out.println("URL could not be encoded: " + e.getMessage()); } CopyObjectRequest copyReq = CopyObjectRequest.builder() .copySource(encodedUrl) .bucket(toBucket) .key(objectKey) .build(); try { CopyObjectResponse copyRes = s3.copyObject(copyReq); return copyRes.copyObjectResult().toString(); } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); }

See the complete example on GitHub.

List objects

Build a ListObjectsRequest and supply the bucket name. Then invoke the S3Client’s listObjects method and pass the ListObjectsRequest object. This method returns a ListObjectsResponse that contains all of the objects in the bucket. You can invoke this object’s contents method to get a list of objects. You can iterate through this list to display the objects, as shown in the following code example.

Imports

import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.ListObjectsRequest; import software.amazon.awssdk.services.s3.model.ListObjectsResponse; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.S3Object; import java.util.List; import java.util.ListIterator;

Code

public static void ListBucketObjects( S3Client s3, String bucketName ) { try { ListObjectsRequest listObjects = ListObjectsRequest .builder() .bucket(bucketName) .build(); ListObjectsResponse res = s3.listObjects(listObjects); List<S3Object> objects = res.contents(); for (ListIterator iterVals = objects.listIterator(); iterVals.hasNext(); ) { S3Object myValue = (S3Object) iterVals.next(); System.out.print("\n The name of the key is " + myValue.key()); System.out.print("\n The object is " + calKb(myValue.size()) + " KBs"); System.out.print("\n The owner is " + myValue.owner()); } } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); } } //convert bytes to kbs private static long calKb(Long val) { return val/1024; }

See the complete example on GitHub.