You may need to limit which files are exported from a bucket. For instance, you might not want all of the files in a bucket, or you may have too many files for a single device and need to export to multiple devices.
This section describes how to configure the
manifest file option to manage those tasks.
If you need to export files from multiple buckets, you can specify multiple exportBucket options in the same manifest file.
operations manifest option specifies a list of one
or more export operations. AWS Import/Export exports keys in alphabetical order.
That is, it carries out the operations in the order listed and then in the
alphabetical order of the keys affected by each export operation. The files on
the storage device remain in strict alphabetical order even if the data to
export exceeds the capacity of the storage device; AWS Import/Export does not,
for example, squeeze in an extra file or two that are out of order just because
there is room on the storage device. This alphabetical order enables you to
continue the export easily: create a new export job using the key of the last
file exported as the
beginMarker in your new export
limit the data exported from a bucket.
Restricts the keys exported to those located in the path
Limits the keys exported to those that occur alphabetically after this option value.
A subfield of
AWS Import/Export uses the entire key when naming the objects exported to your
storage device. For example, AWS Import/Export would map the key
starwars.s3.amazonaws.com/images/jedi.jpg to the
following file on your storage device
/starwars/images/jedi.jpg. For Microsoft Windows,
AWS Import/Export would map the same key to
The following sections talk in depth about some of the export fields you use
specify the beginning and the ending of the keys to export.
The export job starts with the key after the
beginMarker. In other words, the
beginMarker is not included in the export job,
endMarker is included. For example, the
following options export all of the keys that occur alphabetically between
The export will not include
html/index.html, will include
images/kate.jpg, and will not include
operations: - exportBucket: mybucket beginMarker: html/index.html endMarker: images/kate.jpg
It’s possible to construct an export operation that includes no keys. For
example, if you specify
endMarker, there would be no
matching keys because
foo comes alphabetically before
To export multiple subsets of data from one or more buckets, specify multiple
exportBucket operations followed by appropriate
export options. Each dash (-) after
a new set of data to export. Each entry requires an
exportBucket manifest option. The following example
shows four data sets to export from three different buckets:
All buckets must be in the same Amazon S3 region and you must have read permissions on each bucket and each object.
To export data from more than one bucket
exportBucket to specify each bucket you want to
export data from.
In the following example, AWS Import/Export exports all keys under
starwars/image-backup on the storage device.
operations: - exportBucket: mybucket prefix: images beginMarker: images/starwars/jedi.jpg endMarker: images/starwars/masterwindoo.jpg - exportBucket: lost - exportBucket: threescompany prefix: images endMarker: images/characters/mrfurley.jpg
The order in which AWS Import/Export exports keys is the order of the operations you specify
operations manifest option, and then within each
operation, by the alphabetical order of the keys to be exported. In the previous
procedure, for example, AWS Import/Export exports all of the keys from
mybucket in alphabetical order, and then from
in alphabetical order.
AWS Import/Export exports keys in parallel, but the finished order of files on
the storage device is by operation and then alphabetical. This functionality
becomes important when the storage device has insufficient capacity to hold
all of the data that needs to be exported. This ordering enables you to
easily continue the export with a new export job using the key of the last
key exported as your
beginMarker in your new export
AWS Import/Export gives you the ability to specify the location for the exported
files on your storage device using the export manifest options
To export data to a specific directory
targetDirectory to specify the directory on
the storage device you want to export files into.
In the following example, AWS Import/Export exports the contents of the bucket
bucket-data-comes-from to the directory,
myexport/stuff, on the storage device.
targetDirectory is the bucket name.
operations: - exportBucket: bucket-data-comes-from targetDirectory: myexport/stuff