Amazon QLDB Driver recommendations - Amazon Quantum Ledger Database (Amazon QLDB)

Amazon QLDB Driver recommendations

This section describes best practices for configuring and using the Amazon QLDB Driver for any supported language. The code examples provided are specifically for Java.

These recommendations apply for most typical use cases, but one size doesn't fit all. Use the following recommendations as you see fit for your application.

Configuring the QldbDriver object

The QldbDriver object manages connections to your ledger by maintaining a pool of sessions that are reused across transactions. A session represents a single connection to the ledger. QLDB supports one actively executing transaction per session.

Important

For older driver versions, the session pooling functionality is still in the PooledQldbDriver object instead of QldbDriver. If you're using one of the following versions, replace any mentions of QldbDriver with PooledQldbDriver for the rest of this topic.

Driver Version
Java 1.1.0 or earlier
.NET 0.1.0-beta
Node.js 1.0.0-rc.1 or earlier
Python 2.0.2 or earlier

The PooledQldbDriver object is deprecated in the latest version of the drivers. We recommend that you upgrade to the latest version and convert any instances of PooledQldbDriver to QldbDriver.

Configure QldbDriver as a global object

To optimize the use of drivers and sessions, ensure that only one global instance of the driver exists in your application instance. For example in Java, you can use dependency injection frameworks such as Spring, Google Guice, or Dagger. The following code example shows how to configure QldbDriver as a singleton.

@Singleton public QldbDriver qldbDriver (AWSCredentialsProvider credentialsProvider, @Named(LEDGER_NAME_CONFIG_PARAM) String ledgerName) { QldbSessionClientBuilder builder = QldbSessionClient.builder(); if (null != credentialsProvider) { builder.credentialsProvider(credentialsProvider); } return QldbDriver.builder() .ledger(ledgerName) .transactionRetryPolicy(RetryPolicy .builder() .maxRetries(3) .build()) .sessionClientBuilder(builder) .build(); }

Configure the retry attempts

The driver automatically retries transactions when common transient exceptions (such as SocketTimeoutException or NoHttpResponseException) occur. To set the maximum number of retry attempts, you can use the maxRetries parameter of the transactionRetryPolicy configuration object when creating an instance of QldbDriver. (For older driver versions as listed in the previous section, use the retryLimit parameter of PooledQldbDriver.)

The default value of maxRetries is 4.

Client-side errors such as InvalidParameterException can't be retried. When they occur, the transaction is aborted, the session is returned to the pool, and the exception is thrown to the driver's client.

Configure the maximum number of concurrent sessions and transactions

The maximum number of ledger sessions that are used by an instance of QldbDriver to execute transactions is defined by its maxConcurrentTransactions parameter. (For older driver versions as listed in the previous section, this is defined by the poolLimit parameter of PooledQldbDriver.)

This limit must be greater than zero and less than or equal to the maximum number of open HTTP connections that the session client allows, as defined by the specific AWS SDK. For example in Java, the maximum number of connections is set in the ClientConfiguration object.

The default value of maxConcurrentTransactions is the maximum connection setting of your AWS SDK.

When you configure the QldbDriver in your application, take the following scaling considerations:

  • Your pool should always have at least as many sessions as the number of concurrently executing transactions that you plan to have.

  • In a multi-threaded model where a supervisor thread delegates to worker threads, the driver should have at least as many sessions as the number of worker threads. Otherwise, at peak load, threads will be waiting in line for an available session.

  • The service limit of concurrent active sessions per ledger is defined in Quotas in Amazon QLDB. Ensure that you don't configure more than this limit of concurrent sessions to be used for a single ledger across all clients.

Retrying exceptions

When retrying exceptions that occur in QLDB, consider the following recommendations.

Retrying OccConflictException

Optimistic concurrency control (OCC) conflict exceptions occur when the data that the transaction is accessing has changed since the start of the transaction. QLDB throws this exception while trying to commit the transaction. The driver retries the transaction up to as many times as maxRetries is configured.

For more information about OCC and best practices for using indexes to limit OCC conflicts, see Amazon QLDB concurrency model.

Retrying other exceptions outside of the QldbDriver

To retry a transaction outside of the driver when custom, application-defined exceptions are thrown during execution, you must wrap the transaction. For example in Java, the following code shows how to use the Reslience4J library to retry a transaction in QLDB.

private final RetryConfig retryConfig = RetryConfig.custom() .maxAttempts(MAX_RETRIES) .intervalFunction(IntervalFunction.ofExponentialRandomBackoff()) // Retry this exception .retryExceptions(InvalidSessionException.class, MyRetryableException.class) // But fail for any other type of exception extended from RuntimeException .ignoreExceptions(RuntimeException.class) .build(); // Method callable by a client public void myTransactionWithRetries(Params params) { Retry retry = Retry.of("registerDriver", retryConfig); Function<Params, Void> transactionFunction = Retry.decorateFunction( retry, parameters -> transactionNoReturn(params)); transactionFunction.apply(params); } private Void transactionNoReturn(Params params) { try (driver.execute(txn -> { // Transaction code }); } return null; }
Note

Retrying a transaction outside of the QLDB Driver has a multiplier effect. For example, if QldbDriver is configured to retry three times, and the custom retry logic also retries three times, the same transaction can be retried up to nine times.

Making transactions idempotent

As a best practice, make your write transactions idempotent to avoid any unexpected side effects in the case of retries. A transaction is idempotent if it can be executed multiple times and produce identical results each time.

To learn more, see Amazon QLDB concurrency model.

Optimizing performance

To optimize performance when you execute transactions using the driver, take the following considerations:

  • The execute method always makes a minimum of three SendCommand API calls to QLDB, including the following commands:

    1. StartTransaction

    2. ExecuteStatement

      This command is executed for each PartiQL statement that you run in the execute method block.

    3. CommitTransaction

    Consider the total number of API calls that are made when you calculate the overall workload of your application.

  • In general, we recommend starting with a single-threaded writer and optimizing transactions by batching multiple statements within a single transaction. Maximize the quotas on transaction size, document size, and number of documents per transaction, as defined in Quotas in Amazon QLDB.

  • If batching is not sufficient for large transaction loads, you can try multi-threading by adding additional writers. However, you should carefully consider your application requirements for document and transaction sequencing and the additional complexity that this introduces.

Running multiple statements per transaction

As described in the previous section, you can run multiple statements per transaction to optimize performance of your application. In the following Java example, you create a table and then create an index on that table within a transaction. You do this by passing a lambda expression to the execute method.

public static void main(final String... args) { ConnectToLedger.getDriver().execute(txn -> { final Result createTableResult = txn.execute("CREATE TABLE Vehicle"); final Result createIndexResult = txn.execute("CREATE INDEX ON Vehicle (VIN)"); }); log.info("Vehicle table created successfully with index on VIN."); }

The driver's execute method implicitly starts a session and a transaction in that session. Each statement that you run in the lambda expression is wrapped in the transaction. After all of the statements run, the driver auto-commits the transaction. If any statement fails after the automatic retry limit is exhausted, the transaction is aborted.

Propagate exceptions in a transaction

When running multiple statements per transaction, we generally don't recommend that you catch and swallow exceptions within the transaction.

For example in Java, the following program catches any instance of RuntimeException, logs the error, and continues. This code example is considered bad practice because the transaction succeeds even when the UPDATE statement fails. So, the client might assume that the update succeeded when it didn't.

Warning

Do not use this code example. It's provided to show an anti-pattern example that is considered bad practice.

// DO NOT USE this code example because it is considered bad practice public static void main(final String... args) { ConnectToLedger.getDriver().execute(txn -> { final Result selectTableResult = txn.execute("SELECT * FROM Vehicle WHERE VIN ='123456789'"); // Catching an error inside the transaction is an anti-pattern because the operation might // not succeed. // In this example, the transaction succeeds even when the update statement fails. // So, the client might assume that the update succeeded when it didn't. try { processResults(selectTableResult); String model = // some code that extracts the model final Result updateResult = txn.execute("UPDATE Vehicle SET model = ? WHERE VIN = '123456789'", Constants.MAPPER.writeValueAsIonValue(model)); } catch (RuntimeException e) { log.error("Exception when updating the Vehicle table {}", e.getMessage()); } }); log.info("Vehicle table updated successfully."); }

Instead, you should propagate the exception. If any part of the transaction fails, let the execute method abort the transaction so that the client can handle the exception accordingly.