AWS Import/Export
Developer Guide (API Version 2010-06-03)
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.Did this page help you?  Yes | No |  Tell us about it...

Export from Amazon S3 Manifest File Options

The export manifest file contains information related to exporting your data from Amazon S3 onto your storage device. This section describes options that are unique to export manifest files. For information about options that are common to import and export manifest files, see Common Manifest File Options.

Amazon S3 import and export only use manifest file version 2.0. Set manifestVersion to 2.0 for all Amazon S3 import and export manifest files.

OptionDescriptionRequired
fileSystem

Specifies the file system on your storage device. The following is an example fileSystem entry.

fileSystem: NTFS

Type: String

Default: None

Valid Values: NTFS | EXT4

Yes
operations

A container that takes subfields that specify the operations to perform on each bucket for this job. The subfields are name-value pairs. For example, exportBucket specifies the bucket from which to export data. For more information, see Operations Export Manifest Option. The following example shows operations with two subfields, exportBucket and endMarker.

operations:
  - exportBucket: bucket-data-comes-from
    targetDirectory: myexport/stuff
    beginMarker: images/starwars
    endMarker: images/xena
    prefix: exported/

For information about each subfield, see Operations Subfields.

Type: String

Default: None

Yes
recoveryDirectory

This is the directory on your storage device where we save files whose filenames AWS Import/Export must change from their keys. This happens often because characters that are valid in keys are not always valid for filenames. The following example names the recovery directory LostAndFound.

recoveryDirectory: LostAndFound

Type: String

Default: EXPORT-RECOVERY

No
substitutions

This option allows you to provide strings to replace existing values (such as object keys or file names) between the Amazon S3 bucket and the designated device during imports or exports. Matches for your provided values are substituted for the existing values before the import or export operation. The values specified on the left side of the colon are replaced with the values on the right side. Substitutions are applied in length order; the longest matching option is applied first. For more information about working with substitutions, see Working with Amazon S3 Substitutions.

substitutions: 
    "," : "$COMMA$"

Type: String

Default: None

No

trueCryptPassword

AWS Import/Export will use the password to encrypt your data using TrueCrypt during the export.

Important

Do not lose your password, or you will not be able to decrypt your data. AWS Import/Export will not send the password with the encrypted device.

For more information about encrypting data by using TrueCrypt, go to Using TrueCrypt Encryption.

The following is an example of trueCryptPassword

trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE

Type: String

Default: none

Yes

Operations Subfields

The following table describes the subfields of the Operations parameter

Operations SubfieldDescriptionRequired
beginMarker

Indicates where in the bucket to begin exporting data. The export job only includes keys that occur alphabetically after, but not including, beginMarker. For example, if beginMarker is alpha and endMarker is omega, then keys alpha-1 and omega are exported, but alpha and omega-1 are not exported. This option is convenient for limiting the number of files to export and for splitting large export jobs between multiple devices. The following is an example beginMarker value.

operations:
  - exportBucket: bucket-data-comes-from
    beginMarker: images/alpha
    endMarker: images/omega

Type: String

Default: none

No
endMarker

Indicates where in the bucket to stop exporting data. The export job only includes keys that occur alphabetically before and including the key that matches endMarker. For example, if beginMarker is alpha and endMarker is omega, then keys alpha-1 and omega are exported, but alpha and omega-1 are not exported. This option is convenient for limiting the number of files to export and splitting large export jobs across multiple devices. The following is an example endMarker value.

operations:
  - exportBucket: bucket-data-comes-from
    beginMarker: images/alpha
    endMarker: images/omega

Type: String

Default: none

No
exportBucket

Specifies the Amazon S3 bucket where AWS Import/Export retrieves your data. Each listing under operations must have an exportBucket option. The AWS IAM user that created the job must have read permissions on the bucket and objects. If no other options are specified with the listing, AWS Import/Export exports all files in the bucket to your storage device. The following is an example of an exportBucket name.

operations:
  - exportBucket: bucket-data-comes-from

Type: String

Default: None

Constraints: Must be an existing Amazon S3 bucket name

Yes
prefix

Specifies the path in the exportBucket where AWS Import/Export finds the data to export. All keys that contain the prefix are saved to your storage device. For more information, go to Managing File Exports. Note that prefix has a different meaning when used in import manifest files.

If the logPrefix + import-log-JOBID is longer than 1024 bytes, AWS Import/Export returns an InvalidManifestField error from the CREATE JOB command.

The following example exports all files under the directory images.

operations:
  - prefix: images

Type: String

Default: None

Constraints: A key is a sequence of unicode characters whose UTF-8 encoding is at most 1024 bytes long.

No
targetDirectory

Indicates where to export the data to on your storage device. In the following example, AWS Import/Export exports the contents of the bucket, bucket-data-comes-from, to myexport/stuff on the storage device.

operations:
  - exportBucket: bucket-data-comes-from
    targetDirectory: myexport/stuff

Type: String

Default: The specified exportBucket.

No

Example Export from Amazon S3 Manifest

The following example is the manifest file content for an export job from Amazon S3, including expedited shipping.

manifestVersion: 2.0
deviceId: ABCDE
fileSystem: NTFS
generator: AWS Import Export Docs
logPrefix: logs
logBucket: iemanifest-bucket
notificationEmail: john.doe@example.com;jane.roe@example.com
trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE 
operations:
  - exportBucket: my-amz-bucket
    targetDirectory: myexport/stuff
    beginMarker: images/starwars
    endMarker: images/xena
    prefix: exported/
recoveryDirectory: LostAndFound
returnAddress:
   name: Jane Roe 
   company: Example Corp.   
   street1: 123 Any Street
   city: Anytown
   stateOrProvince: WA
   postalCode: 91011-1111
   phoneNumber: 206-555-1111
   country: USA
serviceLevel: expeditedShipping