Menu
AWS Import/Export
Developer Guide

Import to Amazon S3 Manifest File Options

The Amazon S3 import manifest file includes information related to importing your data into Amazon S3 and returning your storage device. This section explains options that are unique to Amazon S3 import manifest files. For information about options that are common to all manifest files, see Common Manifest File Options.

Amazon S3 import and export use manifest file version 2.0. Set manifestVersion to 2.0 for all Amazon Amazon S3 import and export manifest files.

Option Description Required
bucket Specifies the Amazon S3 bucket where AWS Import/Export loads your data. You must create a bucket before we can load data into it; we do not create the bucket for you. The AWS account doing the load must own this bucket. The following is an example of a bucket name.
Copy
bucket: data-load-bucket

Type: String

Default: None

Constraints: Must be an existing Amazon S3 bucket name

Yes
cacheControl

Specifies the value of the Cache-Control HTTP header field for all imported files. We do not validate this value. For more information on cacheControl, please go to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.

Copy
cacheControl: max-age=3600

Type: String

Default: none

No
contentDisposition Specifies the Content-Disposition HTTP header to apply to all loaded files. For more details, go to RFC 2616 section 19. The following is an example contentDisposition value.
Copy
contentDisposition: attachment

Note

This example causes browsers to prompt to save the Amazon S3 object rather than displaying the object directly in the browser.

Type: String

Default: None

No
contentLanguage

Specifies the value of the HTTP Content-Language header field for all imported files. We do not validate this value. For more information about the Content-Language header, go to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.12.

Copy
contentLanguage: es

Type: String

Default: none

No
contentTypes

Contains a map of case-insensitive strings that are key/value pairs that set the Content-Type header. Each key/value pair contains a key that is the file extension (for example, jpg) and a corresponding value for the Content-Type (for example, image/jpeg). These values take precedence over the default file extension to Content-Type mapping. For more details on the default mappings, see Appendix C: File Extension to MIME Types.

The following is an example set of key/value pairs.

Copy
contentTypes: jpg: image/jpeg JPG: image/jpeg gif: image/gif GIF: image/gif mp3: audio/mpeg txt: text/plain zip: application/zip

Type: Map of string to string

Default: None

No
diskTimestampMetadataKey

A user-defined key for adding a storage device's last modified time stamp in the metadata of the content imported to Amazon S3. The value of this key has the same RFC 1123 format as the Amazon S3 Date header: [ DAY "," ] DD-MON-YYY HH:MM:SS zone time (for example, Wed, 23 Feb 2011 01:55:59 GMT). The key name is prefixed with x-amz-meta-.

For example, if you specify:
Copy
diskTimestampMetadataKey: disk-timestamp

the following is the result of a GET operation on the Amazon S3 content:

Copy
x-amz-meta-disk-timestamp: Wed, 23 Feb 2011 01:55:59 GMT

Type: String

Default: None

Constraints: The user-defined key must be a valid HTTP header key (ASCII-only, no colons, no spaces). The key name chosen here must not also be a key name used in the staticMetadata feature.

No
eraseDevice

Acknowledges that AWS will erase the contents of your storage device after importing the data. All writable blocks on your storage device will be overwritten with zeros. You will need to repartition and format your device after the wipe.

The following is an example entry.

Copy
eraseDevice: yes

Type: String

Default: None

Valid Values: yes | true

Yes
expires

Specifies the value of the HTTP Expires header field for all imported files. Expire values should be dates using the RFC 1123 date format. For more information about the Expires header, go to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21.

Copy
expires: Thu, 07 Jan 2010 16:00:00 GMT

Type: String (24 bytes)

Default: None

Constraints: Must follow RFC 1123 date time format, [ DAY "," ] DD-MON-YYY HH:MM:SS zone time, for example "Mon , 12 Apr 2010 23:20:00 UT". For more information, go to RFC 1123 and RFC 822.

No
ignore Specifies directories, files, or file types that we should not load from your storage device. For more information about ignore, see Excluding Files and Directories .

Type: Array of strings

Default: We load all files except the SIGNATURE file

Constraints: The dash followed by a space, “- “, is YAML syntax and must precede each filter

No

pinCode

If a storage device with hardware-based encryption was supplied, the PIN code for the device. For more information, see Encrypting Your Data.

Copy
pinCode: 1234

Important

Do not lose your PIN code, or you will not be able to decrypt your data. AWS Import/Export will not send the PIN code with the encrypted device.

Type: Integer

Default: None

No
prefix

Specifies a string to insert between the bucket name and full path of each file located on your storage device. For more information, see Specifying Key Prefix.

Type: String

Default: None

Constraints: A key is a sequence of unicode characters whose UTF-8 encoding is at most 1024 bytes long. If the logPrefix + import-log-JOBID is longer than 1024 bytes. AWS Import/Export returns an InvalidmanifestField error from the CreateJob response.

No
setContentEncodingForGzFiles

Specifies that files compressed with gzip should have their Content-Encoding header set to gzip and that the Content-Type header should ignore the file endings .gz and .gzip when determining file content types. For more information, see Handling Gzip Files . We do not validate this value.

Copy
setContentEncodingForGzFiles: Yes

Type: String

Default: None

No
staticMetadata This option defines a user-defined key/value pairs to apply to every file imported into Amazon S3. The key/value pair is prefixed with x-amz-meta-. For example, if you specify:
Copy
staticMetadata: import-timestamp: Wed, 23 Feb 2011 01:55:59 GMT application: MyBackupApp

The following is the result of a GET operation on the Amazon S3 content:

Copy
x-amz-meta-import-timestamp: Wed, 23 Feb 2011 01:55:59 GMT x-amz-meta-application: MyBackupApp

Type: Map of string to string

Default: None

Constraints: The user-defined keys must be valid HTTP header keys (ASCII-only, no colons, no spaces); their values must be ASCII-only. The key name chosen here must not also be a key name used in the diskTimestampMetadataKey feature.

No
storageClass Specifies the Amazon S3 storage class for the import job. For more information about the Amazon S3 storage class, go to the Amazon S3 Developer Guide.
Copy
storageClass: REDUCED_REDUNDANCY

Type: Enum

Default: STANDARD

Valid Values: STANDARD | REDUCED_REDUNDANCY

No
substitutions

This option allows you to provide strings to replace existing values (such as object keys or file names) between the Amazon S3 bucket and the designated device during imports or exports. Matches for your provided values are substituted for the existing values before the import or export operation. The values specified on the left side of the colon are replaced with the values on the right side. Substitutions are applied in length order; the longest matching option is applied first. For more information about working with substitutions, see Working with Amazon S3 Substitutions.

Copy
substitutions: "," : "$COMMA$"

Type: String

Default: None

No

trueCryptPassword

If the data was encrypted using TrueCrypt, the password that was provided when the device was encrypted. For more information about encrypting data using TrueCrypt, go to Encrypting Your Data.

The password must not contain an asterisk ( * ) or quote character ( ' " ). If the password contains other special characters, enclose the string in double quote characters.

The following is an example of trueCryptPassword

Copy
trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE

Type: String

Default: none

No

Example Import to Amazon S3 Manifest

The following example is the manifest file content for an import job to Amazon S3, including expedited shipping and the required eraseDevice and notificationEmail options.

Copy
manifestVersion: 2.0 deviceId: ABCDE eraseDevice: yes notificationEmail: john.doe@example.com;jane.roe@example.com bucket: my-amz-bucket cacheControl: max-age=3600 contentDisposition: attachment contentLanguage: en contentTypes: csv: application/vnd.ms-excel diskTimestampMetadataKey: disk-timestamp generator: AWS Import Export Docs ignore: - \.psd$ - \.PSD$ logPrefix: logs logBucket: iemanifest-bucket prefix: imported/ setContentEncodingForGzFiles: yes staticMetadata: import-timestamp: Wed, 23 Feb 2011 01:55:59 GMT application: AppImportedData pinCode:4321 trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE returnAddress: name: Jane Roe company: Example Corp. street1: 123 Any Street city: Anytown stateOrProvince: WA postalCode: 91011-1111 phoneNumber: 206-555-1111 country: USA serviceLevel: expeditedShipping