Requesting a journal export in QLDB - Amazon Quantum Ledger Database (Amazon QLDB)

Requesting a journal export in QLDB

Amazon QLDB provides an API to request an export of your journal blocks for a specified date and time range and a specified Amazon Simple Storage Service (Amazon S3) bucket destination. You can use the AWS Management Console, an AWS SDK, or the AWS Command Line Interface (AWS CLI) to create an export job.

AWS Management Console

Follow these steps to submit a journal export request in QLDB using the QLDB console.

To request an export (console)

  1. Sign in to the AWS Management Console, and open the Amazon QLDB console at https://console.aws.amazon.com/qldb.

  2. In the navigation pane, choose Export.

  3. Choose Create export job.

  4. On the Create export job page, enter the following export settings:

    • Ledger—The ledger whose journal blocks you want to export.

    • Start date and time—The inclusive start timestamp in Coordinated Universal Time (UTC) of the range of journal blocks to export. This timestamp must be earlier than the End date and time. If you provide a start timestamp that is earlier than the ledger's CreationDateTime, QLDB defaults it to the ledger's CreationDateTime.

    • End date and time—The exclusive end timestamp (UTC) of the range of journal blocks to export. This date and time cannot be in the future.

    • Destination for journal blocks—The Amazon S3 bucket and prefix name in which your export job writes the data objects. Use the following Amazon S3 URI format.

      s3://DOC-EXAMPLE-BUCKET/prefix/

      You must specify an S3 bucket name and an optional prefix name for the output objects. The following is an example.

      s3://DOC-EXAMPLE-BUCKET/journalExport/

      The bucket name and prefix must both comply with the Amazon S3 naming rules and conventions. For more information about bucket naming, see Bucket restrictions and limitations in the Amazon S3 Developer Guide. For more information about key name prefixes, see Object key and metadata.

      Note

      Cross-region exports are not supported. The specified Amazon S3 bucket must be in the same AWS Region as your ledger.

    • S3 Encryption Configuration—The encryption settings that are used by your export job to write data in an Amazon S3 bucket. To learn more about server-side encryption options in Amazon S3, see Protecting data using server-side encryption in the Amazon S3 Developer Guide.

      • Bucket default encryption—Use the default encryption settings of the specified Amazon S3 bucket.

      • AES-256—Use server-side encryption with Amazon S3 managed keys (SSE-S3).

      • AWS-KMS—Use server-side encryption with AWS KMS managed keys (SSE-KMS).

        If you choose this type along with the Customer managed CMK option, you must also specify a symmetric customer master key (CMK) in the following Amazon Resource Name (ARN) format.

        arn:aws:kms:aws-region:account-id:key/key-id
    • Permission—The IAM role that grants QLDB write permissions in your Amazon S3 bucket. If applicable, the IAM role must also grant QLDB permissions to use your CMK in AWS KMS. To learn how to create this role, see Export permissions.

  5. When the settings are as you want them, choose Create export job.

    The amount of time it takes for your export job to finish varies depending on the data size. If your request submission is successful, the console returns to the main Export page and lists your export jobs with their current status.

  6. You can see your export objects on the Amazon S3 console.

    Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

    To learn more about the format of these output objects, see Journal export output in QLDB.

Note

Export jobs expire seven days after they complete. For more information, see Export job expiration.

QLDB API

You can also request a journal export by using the Amazon QLDB API with an AWS SDK or the AWS CLI. The QLDB API provides the following actions for use by application programs:

  • ExportJournalToS3 – Exports journal contents within a date and time range from a given ledger into a specified Amazon S3 bucket. The data is written as objects in Amazon Ion format.

  • DescribeJournalS3Export – Returns detailed information about a journal export job. The output includes its current status, creation time, and the parameters of your original export request.

  • ListJournalS3Exports – Returns a list of journal export job descriptions for all ledgers that are associated with the current AWS account and Region. The output of each export job description includes the same details that are returned by DescribeJournalS3Export.

  • ListJournalS3ExportsForLedger – Returns a list of journal export job descriptions for a given ledger. The output of each export job description includes the same details that are returned by DescribeJournalS3Export.

For complete descriptions of these API operations, see the Amazon QLDB API reference.

For information about exporting journal data using the AWS CLI, see the AWS CLI Command Reference.

Sample application

For Java code examples, see the GitHub repository aws-samples/amazon-qldb-dmv-sample-java. For instructions on how to download and install this sample application, see Installing the Amazon QLDB Java sample application. Before requesting an export, make sure that you follow Steps 1–3 in the Java tutorial to create a sample ledger and load it with sample data.

The tutorial code in the following classes provide examples of creating an export, checking the status of an export, and processing the output of an export.

Class Description
ExportJournal Exports journal blocks from the vehicle-registration sample ledger for a timestamp range of 10 minutes ago until now. Writes the output objects in a specified S3 bucket, or creates a unique bucket if one isn't provided.
DescribeJournalExport Describes a journal export job for a specified exportId in the vehicle-registration sample ledger.
ListJournalExports Returns a list of journal export job descriptions for the vehicle-registration sample ledger.
ValidateQldbHashChain Validates the hash chain of the vehicle-registration sample ledger using a given exportId. If not provided, requests a new export to use for hash chain validation.

Export job expiration

Journal export jobs expire seven days after they complete. This expiration period is a hard limit and cannot be changed. This means that after an export job expires, you can no longer use the QLDB console or the following API actions to retrieve metadata about the job:

  • DescribeJournalS3Export

  • ListJournalS3Exports

  • ListJournalS3ExportsForLedger

However, this expiration has no impact on the exported data itself. All of the metadata is preserved in the manifest files that are written by your exports. This expiration is designed to provide a smoother experience for the API actions that list journal export jobs. QLDB removes old export jobs to ensure that you only see recent exports without having to parse multiple pages of jobs.