Overview of Amazon QLDB - Amazon Quantum Ledger Database (Amazon QLDB)

Overview of Amazon QLDB

The following sections provide a high-level overview of Amazon QLDB service components and how they interact.

Journal first

In traditional database architecture, you generally write data in tables as part of a transaction. A transaction log—typically an internal implementation—records all of the transactions and the database modifications that they make. The transaction log is a critical component of the database. You need the log to replay transactions in the event of a system failure, disaster recovery, or data replication. However, database transaction logs aren't immutable and aren't designed to provide direct and easy access to users.

In Amazon QLDB, the journal is the core of the database. Structurally similar to a transaction log, the journal is an immutable, append-only data structure that stores your application data along with the associated metadata. All write transactions, including updates and deletes, are committed to the journal first.

QLDB uses the journal to determine the current state of your ledger data by materializing it into queryable, user-defined tables. These tables also provide an accessible history of all transaction data, including document revisions and metadata. In addition, the journal handles concurrency, sequencing, cryptographic verification, and availability of the ledger data.

The following diagram illustrates the QLDB journal architecture.


                    Diagram titled QLDB: the journal is the database, showing journal
                        architecture, with an application that connects to a ledger and commits
                        transactions to the journal, which are materialized into tables.
  • In this example, an application connects to a ledger and runs transactions that insert, update, and delete a document into a table named cars.

  • The data is first written to the journal in sequenced order.

  • Then the data is materialized into the table with built-in views. These views let you query both the current state and the complete history of the car, with each revision assigned a version number.

  • You can also export or stream data directly from the journal.

Immutable

Because the QLDB journal is append-only, it keeps a full record of all changes to your data that can't be modified or overwritten. There are no APIs or other methods to alter any committed data in place. This journal structure lets you access and query the full history of your ledger.

Note

The only exception to immutability that QLDB supports is data redaction. With this feature, you can comply with regulatory statutes such as the General Data Protection Regulation (GDPR) in the European Union and the California Consumer Privacy Act (CCPA).

QLDB provides a redaction operation that lets you permanently delete inactive document revisions in the history of a table. This operation deletes only the user data in the specified revision, and leaves the journal sequence and the document metadata unchanged. This maintains the overall data integrity of your ledger. For more information, see Redacting document revisions.

QLDB writes one block to the journal in a transaction. Each block contains entry objects that represent the documents that you insert, update, and delete, along with the statements that you ran to commit them. These blocks are sequenced and hash-chained to guarantee data integrity.

The following diagram illustrates this journal structure.


                    Diagram titled records cannot be altered, showing the immutable,
                        append-only journal structure in QLDB, with the sequence number of each
                        hash-chained journal block.

The diagram shows that transactions are committed to the journal as blocks that are hash-chained for verification. Each block has a sequence number to specify its address.

Cryptographically verifiable

Journal blocks are sequenced and chained together with cryptographic hashing techniques, similar to blockchains. QLDB uses the journal's hash chain to provide transactional data integrity using a cryptographic verification method. Using a digest (a hash value that represents a journal's full hash chain as of a point in time) and a Merkle audit proof (a mechanism that proves the validity of any node within a binary hash tree), you can verify that there have been no unintended changes to your data at any time.

The following diagram shows a digest that covers a journal's full hash chain at a point in time.


                    Diagram titled hash chaining using SHA-256, showing a digest covering
                        the full hash chain of a journal, with the structure of a journal block
                        containing entries that represent Ion documents, PartiQL statements, and
                        metadata.

In this diagram, the journal blocks are hashed using the SHA-256 cryptographic hash function and are sequentially chained to subsequent blocks. Each block contains entries that include your data documents, metadata, and the PartiQL statements that ran in the transaction.

For more information, see Data verification in Amazon QLDB.

SQL-like and document flexible

QLDB uses PartiQL as its query language and Amazon Ion as its document-oriented data model. PartiQL is an open-source, SQL-compatible query language that has been extended to work with Ion. With PartiQL, you can insert, query, and manage your data with familiar SQL operators. When you're querying flat documents, the syntax is the same as using SQL to query relational tables. To learn more about the QLDB implementation of PartiQL, see the Amazon QLDB PartiQL reference.

Amazon Ion is a superset of JSON. Ion is an open-source, document-based data format that gives you the flexibility of storing and processing structured, semistructured, and nested data. To learn more about Ion in QLDB, see the Amazon Ion data format reference in Amazon QLDB.

For a high-level comparison of the core components and features in traditional relational databases versus QLDB, see From relational to ledger.

Open-source developer tools

To simplify application development, QLDB provides open-source drivers in various programming languages. You can use these drivers to interact with the transactional data API by running PartiQL statements on a ledger and processing the results of those statements. For information and tutorials about the driver languages that are currently supported, see Getting started with the Amazon QLDB driver.

Amazon Ion also provides client libraries that process Ion data for you. For developer guides and code examples of processing Ion data, see the Amazon Ion documentation on GitHub.

Serverless and highly available

QLDB is fully managed, serverless, and highly available. The service automatically scales to support the demands of your application, and you don't need to provision instances or capacity. Multiple copies of your data are replicated within an Availability Zone and across Availability Zones in an AWS Region.

Enterprise grade

QLDB transactions are fully compliant with atomicity, consistency, isolation, and durability (ACID) properties. QLDB uses optimistic concurrency control (OCC), and transactions operate with full serializability—the highest level of isolation. This means that there's no risk of seeing phantom reads, dirty reads, write skew, or other similar concurrency issues. For more information, see Amazon QLDB concurrency model.