Requesting a journal export in QLDB

Amazon QLDB provides an API to request an export of your journal blocks for a specified date and time range and a specified Amazon S3 bucket destination. A journal export job can write the data objects in either the text or binary representation of Amazon Ion format, or in JSON Lines text format. You can use the AWS Management Console, an AWS SDK, or the AWS Command Line Interface (AWS CLI) to create an export job.

AWS Management Console

Follow these steps to submit a journal export request in QLDB using the QLDB console.

To request an export (console)

1. Sign in to the AWS Management Console, and open the Amazon QLDB console at https://console.aws.amazon.com/qldb.

2. In the navigation pane, choose Export.

3. Choose Create export job.

4. On the Create export job page, enter the following export settings:

   • Ledger – The ledger whose journal blocks you want to export.

   • Start date and time – The inclusive start timestamp in Coordinated Universal Time (UTC) of the range of journal blocks to export. This timestamp must be earlier than the End date and time. If you provide a start timestamp that is earlier than the ledger's CreationDateTime, QLDB defaults it to the ledger's CreationDateTime.

   • End date and time – The exclusive end timestamp (UTC) of the range of journal blocks to export. This date and time can't be in the future.

   • Destination for journal blocks – The Amazon S3 bucket and prefix name in which your export job writes the data objects. Use the following Amazon S3 URI format.

     s3://DOC-EXAMPLE-BUCKET/prefix/

     You must specify an S3 bucket name and an optional prefix name for the output objects. The following is an example.

     s3://DOC-EXAMPLE-BUCKET/journalExport/

     The bucket name and prefix must both comply with the Amazon S3 naming rules and conventions. For more information about bucket naming, see Bucket restrictions and limitations in the Amazon S3 Developer Guide. For more information about key name prefixes, see Object key and metadata.

     Note

     Cross-region exports are not supported. The specified Amazon S3 bucket must be in the same AWS Region as your ledger.

   • S3 Encryption – The encryption settings that are used by your export job to write data in an Amazon S3 bucket. To learn more about server-side encryption options in Amazon S3, see Protecting data using server-side encryption in the Amazon S3 Developer Guide.

     • Bucket default encryption – Use the default encryption settings of the specified Amazon S3 bucket.

     • AES-256 – Use server-side encryption with Amazon S3 managed keys (SSE-S3).

     • AWS-KMS – Use server-side encryption with AWS KMS managed keys (SSE-KMS).

       If you choose this type along with the Choose a different AWS KMS key option, you must also specify a symmetric encryption KMS key in the following Amazon Resource Name (ARN) format.

       arn:aws:kms:aws-region:account-id:key/key-id

   • Service access – The IAM role that grants QLDB write permissions in your Amazon S3 bucket. If applicable, the IAM role must also grant QLDB permissions to use your KMS key.

     To pass a role to QLDB when requesting a journal export, you must have permissions to perform the iam:PassRole action on the IAM role resource.

     • Create and use a new service role – Let the console create a new role for you with the required permissions for the specified Amazon S3 bucket.

     • Use an existing service role – To learn how to manually create this role in IAM, see Export permissions.

   • Output format – The output format of your exported journal data

     • Ion text – (Default) Text representation of Amazon Ion

     • Ion binary – Binary representation of Amazon Ion

     • JSON – Newline-delimited JSON text format

       If you choose JSON, QLDB down-converts the Ion journal data to JSON in your exported data objects. For more information, see Down-converting to JSON.

5. When the settings are as you want them, choose Create export job.

   The amount of time it takes for your export job to finish varies depending on the data size. If your request submission is successful, the console returns to the main Export page and lists your export jobs with their current status.

6. You can see your export objects on the Amazon S3 console.

   Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

   To learn more about the format of these output objects, see Journal export output in QLDB.

Note

Export jobs expire seven days after they complete. For more information, see Export job expiration.

QLDB API

You can also request a journal export by using the Amazon QLDB API with an AWS SDK or the AWS CLI. The QLDB API provides the following operations for use by application programs:

• ExportJournalToS3 – Exports journal contents within a date and time range from a given ledger into a specified Amazon S3 bucket. An export job can write the data as objects in either the text or binary representation of Amazon Ion format, or in JSON Lines text format.

• DescribeJournalS3Export – Returns detailed information about a journal export job. The output includes its current status, creation time, and the parameters of your original export request.

• ListJournalS3Exports – Returns a list of journal export job descriptions for all ledgers that are associated with the current AWS account and Region. The output of each export job description includes the same details that are returned by DescribeJournalS3Export.

• ListJournalS3ExportsForLedger – Returns a list of journal export job descriptions for a given ledger. The output of each export job description includes the same details that are returned by DescribeJournalS3Export.

For complete descriptions of these API operations, see the Amazon QLDB API reference.

For information about exporting journal data using the AWS CLI, see the AWS CLI Command Reference.

Sample application (Java)

For Java code examples of basic export operations, see the GitHub repository aws-samples/amazon-qldb-dmv-sample-java. For instructions on how to download and install this sample application, see Installing the Amazon QLDB Java sample application. Before requesting an export, make sure that you follow Steps 1–3 in the Java tutorial to create a sample ledger and load it with sample data.

The tutorial code in the following classes provide examples of creating an export, checking the status of an export, and processing the output of an export.

Class	Description
ExportJournal	Exports journal blocks from the vehicle-registration sample ledger for a timestamp range of 10 minutes ago until now. Writes the output objects in a specified S3 bucket, or creates a unique bucket if one isn't provided.
DescribeJournalExport	Describes a journal export job for a specified exportId in the vehicle-registration sample ledger.
ListJournalExports	Returns a list of journal export job descriptions for the vehicle-registration sample ledger.
ValidateQldbHashChain	Validates the hash chain of the vehicle-registration sample ledger using a given exportId. If not provided, requests a new export to use for hash chain validation.

Export job expiration

Completed journal export jobs are subject to a 7-day retention period. They're automatically hard-deleted after this limit expires. This expiration period is a hard limit and can't be changed.

After a completed export job is deleted, you can no longer use the QLDB console or the following API operations to retrieve metadata about the job:

• DescribeJournalS3Export

• ListJournalS3Exports

• ListJournalS3ExportsForLedger

However, this expiration has no impact on the exported data itself. All of the metadata is preserved in the manifest files that are written by your exports. This expiration is designed to provide a smoother experience for the API operations that list journal export jobs. QLDB removes old export jobs to ensure that you only see recent exports without having to parse multiple pages of jobs.