Menu
AWS Import/Export
Developer Guide

Export from Amazon S3 Manifest File Options

The export manifest file contains information related to exporting your data from Amazon S3 onto your storage device. This section describes options that are unique to export manifest files. For information about options that are common to import and export manifest files, see Common Manifest File Options.

Amazon S3 import and export only use manifest file version 2.0. Set manifestVersion to 2.0 for all Amazon S3 import and export manifest files.

Option Description Required
fileSystem

Specifies the file system on your storage device. The following is an example fileSystem entry.

Copy
fileSystem: NTFS

Type: String

Default: None

Valid Values: NTFS | EXT4

Yes
operations

A container that takes subfields that specify the operations to perform on each bucket for this job. The subfields are name-value pairs. For example, exportBucket specifies the bucket from which to export data. For more information, see Operations Export Manifest Option. The following example shows operations with two subfields, exportBucket and endMarker.

Copy
operations: - exportBucket: bucket-data-comes-from targetDirectory: myexport/stuff beginMarker: images/starwars endMarker: images/xena prefix: exported/

For information about each subfield, see Operations Subfields.

Type: String

Default: None

Yes

pinCode

If a storage device with hardware-based encryption was supplied, the PIN code for the device.

*Either pinCode or trueCryptPassword is required for all export jobs. For more information, see Encrypting Your Data.

Copy
pinCode: 1234

Important

Do not lose your PIN code, or you will not be able to decrypt your data. AWS Import/Export will not send the PIN code with the encrypted device.

Type: Integer

Default: None

Yes*
recoveryDirectory

This is the directory on your storage device where we save files whose filenames AWS Import/Export must change from their keys. This happens often because characters that are valid in keys are not always valid for filenames. The following example names the recovery directory LostAndFound.

Copy
recoveryDirectory: LostAndFound

Type: String

Default: EXPORT-RECOVERY

No
substitutions

This option allows you to provide strings to replace existing values (such as object keys or file names) between the Amazon S3 bucket and the designated device during imports or exports. Matches for your provided values are substituted for the existing values before the import or export operation. The values specified on the left side of the colon are replaced with the values on the right side. Substitutions are applied in length order; the longest matching option is applied first. For more information about working with substitutions, see Working with Amazon S3 Substitutions.

Copy
substitutions: "," : "$COMMA$"

Type: String

Default: None

No

trueCryptPassword

AWS Import/Export will use the password to encrypt your data using TrueCrypt during the export.

The password must not contain an asterisk ( * ) or quote character ( ' " ). If the password contains other special characters, enclose the string in double quote characters.

*Either pinCode or trueCryptPassword is required for all export jobs. For more information, see Encrypting Your Data.

Important

Do not lose your password, or you will not be able to decrypt your data. AWS Import/Export will not send the password with the encrypted device.

For more information about encrypting data by using TrueCrypt, go to Encrypting Your Data.

The following is an example of trueCryptPassword

Copy
trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE

Type: String

Default: none

Yes*

Operations Subfields

The following table describes the subfields of the Operations parameter

Operations Subfield Description Required
beginMarker

Indicates where in the bucket to begin exporting data. The export job only includes keys that occur alphabetically after, but not including, beginMarker. For example, if beginMarker is alpha and endMarker is omega, then keys alpha-1 and omega are exported, but alpha and omega-1 are not exported. This option is convenient for limiting the number of files to export and for splitting large export jobs between multiple devices. The following is an example beginMarker value.

Copy
operations: - exportBucket: bucket-data-comes-from beginMarker: images/alpha endMarker: images/omega

Type: String

Default: none

No
endMarker

Indicates where in the bucket to stop exporting data. The export job only includes keys that occur alphabetically before and including the key that matches endMarker. For example, if beginMarker is alpha and endMarker is omega, then keys alpha-1 and omega are exported, but alpha and omega-1 are not exported. This option is convenient for limiting the number of files to export and splitting large export jobs across multiple devices. The following is an example endMarker value.

Copy
operations: - exportBucket: bucket-data-comes-from beginMarker: images/alpha endMarker: images/omega

Type: String

Default: none

No
exportBucket

Specifies the Amazon S3 bucket where AWS Import/Export retrieves your data. Each listing under operations must have an exportBucket option. The AWS IAM user that created the job must have read permissions on the bucket and objects. If no other options are specified with the listing, AWS Import/Export exports all files in the bucket to your storage device. The following is an example of an exportBucket name.

Copy
operations: - exportBucket: bucket-data-comes-from

Type: String

Default: None

Constraints: Must be an existing Amazon S3 bucket name

Note

For export, your Amazon S3 log bucket and your Amazon S3 export bucket must be in the same region. If your buckets are in different regions, your job request will fail.

Yes
prefix

Specifies the path in the exportBucket where AWS Import/Export finds the data to export. All keys that contain the prefix are saved to your storage device. For more information, go to Managing File Exports. Note that prefix has a different meaning when used in import manifest files.

If the logPrefix + import-log-JOBID is longer than 1024 bytes, AWS Import/Export returns an InvalidManifestField error from the CREATE JOB command.

The following example exports all files under the directory images.

Copy
operations: - prefix: images

Type: String

Default: None

Constraints: A key is a sequence of unicode characters whose UTF-8 encoding is at most 1024 bytes long.

No
targetDirectory

Indicates where to export the data to on your storage device. In the following example, AWS Import/Export exports the contents of the bucket, bucket-data-comes-from, to myexport/stuff on the storage device.

Copy
operations: - exportBucket: bucket-data-comes-from targetDirectory: myexport/stuff

Type: String

Default: The specified exportBucket.

No

Example Export from Amazon S3 Manifest

The following example is the manifest file content for an export job from Amazon S3, including expedited shipping.

Copy
manifestVersion: 2.0 deviceId: ABCDE fileSystem: NTFS generator: AWS Import Export Docs logPrefix: logs logBucket: iemanifest-bucket notificationEmail: john.doe@example.com;jane.roe@example.com trueCryptPassword: SimpleButEasi1yCrackedEXAMPLE operations: - exportBucket: my-amz-bucket targetDirectory: myexport/stuff beginMarker: images/starwars endMarker: images/xena prefix: exported/ recoveryDirectory: LostAndFound returnAddress: name: Jane Roe company: Example Corp. street1: 123 Any Street city: Anytown stateOrProvince: WA postalCode: 91011-1111 phoneNumber: 206-555-1111 country: USA serviceLevel: expeditedShipping