We announced the upcoming end-of-support for AWS SDK for JavaScript v2.
We recommend that you migrate to AWS SDK for JavaScript v3. For dates, additional details, and information on how to migrate, please refer to the linked announcement.

Class: AWS.S3.ManagedUpload

Inherits:
Object
  • Object
show all
Defined in:
lib/s3/managed_upload.js

Overview

The managed uploader allows for easy and efficient uploading of buffers, blobs, or streams, using a configurable amount of concurrency to perform multipart uploads where possible. This abstraction also enables uploading streams of unknown size due to the use of multipart uploads.

To construct a managed upload object, see the constructor() function.

Tracking upload progress

The managed upload object can also track progress by attaching an 'httpUploadProgress' listener to the upload manager. This event is similar to AWS.Request~httpUploadProgress but groups all concurrent upload progress into a single event. See httpUploadProgress for more information.

Handling Multipart Cleanup

By default, this class will automatically clean up any multipart uploads when an individual part upload fails. This behavior can be disabled in order to manually handle failures by setting the leavePartsOnError configuration option to true when initializing the upload object.

Constructor Summary collapse

Event Summary collapse

Property Summary collapse

Method Summary collapse

Constructor Details

new AWS.S3.ManagedUpload(options) ⇒ void

Note:

A "Body" parameter is required to be set prior to calling send().

Note:

In Node.js, sending "Body" as object-mode stream may result in upload hangs. Using buffer stream is preferable.

Note:

ContentMD5 should not be provided when using the managed upload object. Instead, setting "computeChecksums" to true will enable automatic ContentMD5 generation by the managed upload object.

Creates a managed upload object with a set of configuration options.

Examples:

Creating a default uploader for a stream object

var upload = new AWS.S3.ManagedUpload({
  params: {Bucket: 'bucket', Key: 'key', Body: stream}
});

Creating an uploader with concurrency of 1 and partSize of 10mb

var upload = new AWS.S3.ManagedUpload({
  partSize: 10 * 1024 * 1024, queueSize: 1,
  params: {Bucket: 'bucket', Key: 'key', Body: stream}
});

Creating an uploader with tags

var upload = new AWS.S3.ManagedUpload({
  params: {Bucket: 'bucket', Key: 'key', Body: stream},
  tags: [{Key: 'tag1', Value: 'value1'}, {Key: 'tag2', Value: 'value2'}]
});

Options Hash (options):

  • params (map) —

    a map of parameters to pass to the upload requests. The "Body" parameter is required to be specified either on the service or in the params option.

  • queueSize (Number) — default: 4 —

    the size of the concurrent queue manager to upload parts in parallel. Set to 1 for synchronous uploading of parts. Note that the uploader will buffer at most queueSize * partSize bytes into memory at any given time.

  • partSize (Number) — default: 5mb —

    the size in bytes for each individual part to be uploaded. Adjust the part size to ensure the number of parts does not exceed maxTotalParts. See minPartSize for the minimum allowed part size.

  • leavePartsOnError (Boolean) — default: false —

    whether to abort the multipart upload if an error occurs. Set to true if you want to handle failures manually.

  • service (AWS.S3) —

    an optional S3 service object to use for requests. This object might have bound parameters used by the uploader.

  • tags (Array<map>) —

    The tags to apply to the uploaded object. Each tag should have a Key and Value keys.

See Also:

Event Details

'httpUploadProgress' — function (progress)

Note:

The total property may not be set if the stream being uploaded has not yet finished chunking. In this case the total will be undefined until the total stream size is known.

Note:

This event will not be emitted in Node.js 0.8.x.

Triggered when the uploader has uploaded more data.

Context (this):

  • (Request) —

    the request object that was sent

Parameters:

  • progress (map) —

    An object containing the loaded and total bytes of the request and the key of the S3 object. Note that total may be undefined until the payload size is known.

Property Details

maxTotalParts ⇒ Number (static, readonly)

Default Value:

AWS.S3.ManagedUpload.maxTotalParts = 10000

minPartSize ⇒ Number (static, readonly)

Returns the minimum number of bytes for an individual part upload.

Default Value:

AWS.S3.ManagedUpload.minPartSize = 1024 * 1024 * 5

Returns:

  • (Number) —

    the minimum number of bytes for an individual part upload.

Method Details

abort() ⇒ void

Note:

By default, calling this function will cleanup a multipart upload if one was created. To leave the multipart upload around after aborting a request, configure leavePartsOnError to true in the constructor().

Note:

Calling abort() in the browser environment will not abort any requests that are already in flight. If a multipart upload was created, any parts not yet uploaded will not be sent, and the multipart upload will be cleaned up.

Aborts a managed upload, including all concurrent upload requests.

Examples:

Aborting an upload

var params = {
  Bucket: 'bucket', Key: 'key',
  Body: Buffer.alloc(1024 * 1024 * 25) // 25MB payload
};
var upload = s3.upload(params);
upload.send(function (err, data) {
  if (err) console.log("Error:", err.code, err.message);
  else console.log(data);
});

// abort request in 1 second
setTimeout(upload.abort.bind(upload), 1000);

promise() ⇒ Promise

Returns a 'thenable' promise.

Two callbacks can be provided to the then method on the returned promise. The first callback will be called if the promise is fulfilled, and the second callback will be called if the promise is rejected.

Examples:

Sending an upload request using promises.

var upload = s3.upload({Bucket: 'bucket', Key: 'key', Body: stream});
var promise = upload.promise();
promise.then(function(data) { ... }, function(err) { ... });

Callbacks:

  • function(data) { ... }

    Called if the promise is fulfilled.

    Parameters:

    • data (map) —

      The response data from the successful upload: Location (String) the URL of the uploaded object ETag (String) the ETag of the uploaded object Bucket (String) the bucket to which the object was uploaded Key (String) the key to which the object was uploaded

  • function(err) { ... }

    Called if the promise is rejected.

    Parameters:

    • err (Error) —

      an error or null if no error occurred.

Returns:

  • (Promise) —

    A promise that represents the state of the upload request.

send(callback) ⇒ void

Initiates the managed upload for the payload.

Examples:

Sending a managed upload object

var params = {Bucket: 'bucket', Key: 'key', Body: stream};
var upload = new AWS.S3.ManagedUpload({params: params});
upload.send(function(err, data) {
  console.log(err, data);
});

Callback (callback):

  • function(err, data) { ... }

    Parameters:

    • err (Error) —

      an error or null if no error occurred.

    • data (map) —

      The response data from the successful upload: * Location (String) the URL of the uploaded object * ETag (String) the ETag of the uploaded object * Bucket (String) the bucket to which the object was uploaded * Key (String) the key to which the object was uploaded