Configuring the QldbDriver object Retrying on exceptions Optimizing performance Running multiple statements per transaction

Amazon QLDB driver recommendations

Important

End of support notice: Existing customers will be able to use Amazon QLDB until end of support on 07/31/2025. For more details, see Migrate an Amazon QLDB Ledger to Amazon Aurora PostgreSQL.

This section describes best practices for configuring and using the Amazon QLDB driver for any supported language. The code examples provided are specifically for Java.

These recommendations apply for most typical use cases, but one size doesn't fit all. Use the following recommendations as you see fit for your application.

Topics

Configuring the QldbDriver object
Retrying on exceptions
Optimizing performance
Running multiple statements per transaction

Configuring the QldbDriver object

The QldbDriver object manages connections to your ledger by maintaining a pool of sessions that are reused across transactions. A session represents a single connection to the ledger. QLDB supports one actively running transaction per session.

Important

For older driver versions, the session pooling functionality is still in the PooledQldbDriver object instead of QldbDriver. If you're using one of the following versions, replace any mentions of QldbDriver with PooledQldbDriver for the rest of this topic.

Driver	Version
Java	`1.1.0` or earlier
.NET	`0.1.0-beta`
Node.js	`1.0.0-rc.1` or earlier
Python	`2.0.2` or earlier

The PooledQldbDriver object is deprecated in the latest version of the drivers. We recommend that you upgrade to the latest version and convert any instances of PooledQldbDriver to QldbDriver.

Configure QldbDriver as a global object

To optimize the use of drivers and sessions, ensure that only one global instance of the driver exists in your application instance. For example in Java, you can use dependency injection frameworks such as Spring, Google Guice, or Dagger. The following code example shows how to configure QldbDriver as a singleton.


@Singleton
public QldbDriver qldbDriver (AWSCredentialsProvider credentialsProvider,
                                    @Named(LEDGER_NAME_CONFIG_PARAM) String ledgerName) {
    QldbSessionClientBuilder builder = QldbSessionClient.builder();
    if (null != credentialsProvider) {
        builder.credentialsProvider(credentialsProvider);
    }
    return QldbDriver.builder()
            .ledger(ledgerName)
            .transactionRetryPolicy(RetryPolicy
                .builder()
                .maxRetries(3)
                .build())
            .sessionClientBuilder(builder)
            .build();
}

Configure the retry attempts

The driver automatically retries transactions when common transient exceptions (such as SocketTimeoutException or NoHttpResponseException) occur. To set the maximum number of retry attempts, you can use the maxRetries parameter of the transactionRetryPolicy configuration object when creating an instance of QldbDriver. (For older driver versions as listed in the previous section, use the retryLimit parameter of PooledQldbDriver.)

The default value of maxRetries is 4.

Client-side errors such as InvalidParameterException can't be retried. When they occur, the transaction is aborted, the session is returned to the pool, and the exception is thrown to the driver's client.

Configure the maximum number of concurrent sessions and transactions

The maximum number of ledger sessions that are used by an instance of QldbDriver to run transactions is defined by its maxConcurrentTransactions parameter. (For older driver versions as listed in the previous section, this is defined by the poolLimit parameter of PooledQldbDriver.)

This limit must be greater than zero and less than or equal to the maximum number of open HTTP connections that the session client allows, as defined by the specific AWS SDK. For example in Java, the maximum number of connections is set in the ClientConfiguration object.

The default value of maxConcurrentTransactions is the maximum connection setting of your AWS SDK.

When you configure the QldbDriver in your application, take the following scaling considerations:

Your pool should always have at least as many sessions as the number of concurrently running transactions that you plan to have.
In a multi-threaded model where a supervisor thread delegates to worker threads, the driver should have at least as many sessions as the number of worker threads. Otherwise, at peak load, threads will be waiting in line for an available session.
The service limit of concurrent active sessions per ledger is defined in Quotas and limits in Amazon QLDB. Ensure that you don't configure more than this limit of concurrent sessions to be used for a single ledger across all clients.

Retrying on exceptions

When retrying on exceptions that occur in QLDB, consider the following recommendations.

Retrying on OccConflictException

Optimistic concurrency control (OCC) conflict exceptions occur when the data that the transaction is accessing has changed since the start of the transaction. QLDB throws this exception while trying to commit the transaction. The driver retries the transaction up to as many times as maxRetries is configured.

For more information about OCC and best practices for using indexes to limit OCC conflicts, see Amazon QLDB concurrency model.

Retrying on other exceptions outside of QldbDriver

To retry a transaction outside of the driver when custom, application-defined exceptions are thrown during runtime, you must wrap the transaction. For example in Java, the following code shows how to use the Reslience4J library to retry a transaction in QLDB.


private final RetryConfig retryConfig = RetryConfig.custom()
        .maxAttempts(MAX_RETRIES)
        .intervalFunction(IntervalFunction.ofExponentialRandomBackoff())
        // Retry this exception
        .retryExceptions(InvalidSessionException.class, MyRetryableException.class)
        // But fail for any other type of exception extended from RuntimeException
        .ignoreExceptions(RuntimeException.class)
        .build();

// Method callable by a client
public void myTransactionWithRetries(Params params) {
    Retry retry = Retry.of("registerDriver", retryConfig);

    Function<Params, Void> transactionFunction = Retry.decorateFunction(
            retry,
            parameters ->  transactionNoReturn(params));
    transactionFunction.apply(params);
}

private Void transactionNoReturn(Params params) {
    try (driver.execute(txn -> {
            // Transaction code
        });
    }
    return null;
}

Note

Retrying a transaction outside of the QLDB driver has a multiplier effect. For example, if QldbDriver is configured to retry three times, and the custom retry logic also retries three times, the same transaction can be retried up to nine times.

Making transactions idempotent

As a best practice, make your write transactions idempotent to avoid any unexpected side effects in the case of retries. A transaction is idempotent if it can run multiple times and produce identical results each time.

To learn more, see Amazon QLDB concurrency model.

Optimizing performance

To optimize performance when you run transactions using the driver, take the following considerations:

The execute operation always makes a minimum of three SendCommand API calls to QLDB, including the following commands:
1. StartTransaction
2. ExecuteStatement
  
  This command is invoked for each PartiQL statement that you run in the execute block.
3. CommitTransaction
Consider the total number of API calls that are made when you calculate the overall workload of your application.
In general, we recommend starting with a single-threaded writer and optimizing transactions by batching multiple statements within a single transaction. Maximize the quotas on transaction size, document size, and number of documents per transaction, as defined in Quotas and limits in Amazon QLDB.
If batching isn't sufficient for large transaction loads, you can try multi-threading by adding additional writers. However, you should carefully consider your application requirements for document and transaction sequencing and the additional complexity that this introduces.

Running multiple statements per transaction

As described in the previous section, you can run multiple statements per transaction to optimize performance of your application. In the following code example, you query a table and then update a document in that table within a transaction. You do this by passing a lambda expression to the execute operation.

Java


// This code snippet is intentionally trivial. In reality you wouldn't do this because you'd
// set your UPDATE to filter on vin and insured, and check if you updated something or not.
public static boolean InsureCar(QldbDriver qldbDriver, final String vin) {
    final IonSystem ionSystem = IonSystemBuilder.standard().build();
    final IonString ionVin = ionSystem.newString(vin);

    return qldbDriver.execute(txn -> {
        Result result = txn.execute(
                "SELECT insured FROM Vehicles WHERE vin = ? AND insured = FALSE",
                ionVin);
        if (!result.isEmpty()) {
            txn.execute("UPDATE Vehicles SET insured = TRUE WHERE vin = ?", ionVin);
            return true;
        }
        return false;
    });
}

.NET


// This code snippet is intentionally trivial. In reality you wouldn't do this because you'd
// set your UPDATE to filter on vin and insured, and check if you updated something or not.
public static async Task<bool> InsureVehicle(IAsyncQldbDriver driver, string vin)
{
    ValueFactory valueFactory = new ValueFactory();
    IIonValue ionVin = valueFactory.NewString(vin);

    return await driver.Execute(async txn =>
    {
        // Check if the vehicle is insured.
        Amazon.QLDB.Driver.IAsyncResult result = await txn.Execute(
            "SELECT insured FROM Vehicles WHERE vin = ? AND insured = FALSE", ionVin);

        if (await result.CountAsync() > 0)
        {
            // If the vehicle is not insured, insure it.
            await txn.Execute(
                "UPDATE Vehicles SET insured = TRUE WHERE vin = ?", ionVin);
            return true;
        }
        return false;
    });
}

Go


// This code snippet is intentionally trivial. In reality you wouldn't do this because you'd
// set your UPDATE to filter on vin and insured, and check if you updated something or not.
func InsureCar(driver *qldbdriver.QLDBDriver, vin string) (bool, error) {
    insured, err := driver.Execute(context.Background(), func(txn qldbdriver.Transaction) (interface{}, error) {

        result, err := txn.Execute(
            "SELECT insured FROM Vehicles WHERE vin = ? AND insured = FALSE", vin)
        if err != nil {
            return false, err
        }

        hasNext := result.Next(txn)
        if !hasNext && result.Err() != nil {
            return false, result.Err()
        }

        if hasNext {
            _, err = txn.Execute(
                "UPDATE Vehicles SET insured = TRUE WHERE vin = ?", vin)
            if err != nil {
                return false, err
            }
            return true, nil
        }
        return false, nil
    })
    if err != nil {
        panic(err)
    }

    return insured.(bool), err
}

Node.js


// This code snippet is intentionally trivial. In reality you wouldn't do this because you'd
// set your UPDATE to filter on vin and insured, and check if you updated something or not.
async function insureCar(driver: QldbDriver, vin: string): Promise<boolean> {

    return await driver.executeLambda(async (txn: TransactionExecutor) => {
        const results: dom.Value[] = (await txn.execute(
            "SELECT insured FROM Vehicles WHERE vin = ? AND insured = FALSE", vin)).getResultList();

        if (results.length > 0) {
            await txn.execute(
                "UPDATE Vehicles SET insured = TRUE WHERE vin = ?", vin);
            return true;
        }
        return false;
    });
};

Python


# This code snippet is intentionally trivial. In reality you wouldn't do this because you'd
# set your UPDATE to filter on vin and insured, and check if you updated something or not.

def do_insure_car(transaction_executor, vin):
    cursor = transaction_executor.execute_statement(
        "SELECT insured FROM Vehicles WHERE vin = ? AND insured = FALSE", vin)
    first_record = next(cursor, None)
    if first_record:
        transaction_executor.execute_statement(
            "UPDATE Vehicles SET insured = TRUE WHERE vin = ?", vin)
        return True
    else:
        return False

def insure_car(qldb_driver, vin_to_insure):
    return qldb_driver.execute_lambda(
        lambda executor: do_insure_car(executor, vin_to_insure))

The driver's execute operation implicitly starts a session and a transaction in that session. Each statement that you run in the lambda expression is wrapped in the transaction. After all of the statements run, the driver auto-commits the transaction. If any statement fails after the automatic retry limit is exhausted, the transaction is aborted.

Propagate exceptions in a transaction

When running multiple statements per transaction, we generally don't recommend that you catch and swallow exceptions within the transaction.

For example in Java, the following program catches any instance of RuntimeException, logs the error, and continues. This code example is considered bad practice because the transaction succeeds even when the UPDATE statement fails. So, the client might assume that the update succeeded when it didn't.

Warning

Don't use this code example. It's provided to show an anti-pattern example that is considered bad practice.


// DO NOT USE this code example because it is considered bad practice
public static void main(final String... args) {
    ConnectToLedger.getDriver().execute(txn -> {
        final Result selectTableResult = txn.execute("SELECT * FROM Vehicle WHERE VIN ='123456789'");
        // Catching an error inside the transaction is an anti-pattern because the operation might
        // not succeed.
        // In this example, the transaction succeeds even when the update statement fails.
        // So, the client might assume that the update succeeded when it didn't.
        try {
            processResults(selectTableResult);
            String model = // some code that extracts the model
            final Result updateResult = txn.execute("UPDATE Vehicle SET model = ? WHERE VIN = '123456789'",
                    Constants.MAPPER.writeValueAsIonValue(model));
        } catch (RuntimeException e) {
            log.error("Exception when updating the Vehicle table {}", e.getMessage());
        }
    });
    log.info("Vehicle table updated successfully.");
}

Propagate (bubble up) the exception instead. If any part of the transaction fails, let the execute operation abort the transaction so that the client can handle the exception accordingly.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Session management with the driver

Driver retry policy