Enabling client-side buffering and request batching with Amazon SQS
The AWS SDK for JavaAmazonSQSBufferedAsyncClient
which accesses Amazon SQS. This client allows
for simple request batching using client-side buffering—calls made from the
client are first buffered and then sent as a batch request to Amazon SQS.
Client-side buffering allows up to 10 requests to be buffered and sent as a batch
request, decreasing your cost of using Amazon SQS and reducing the number of sent requests.
AmazonSQSBufferedAsyncClient
buffers both synchronous and asynchronous
calls. Batched requests and support for long
polling can also help increase throughput. For more information, see Increasing throughput
using horizontal scaling and action batching with Amazon SQS.
Because AmazonSQSBufferedAsyncClient
implements the same interface as
AmazonSQSAsyncClient
, migrating from AmazonSQSAsyncClient
to AmazonSQSBufferedAsyncClient
typically requires only minimal changes to
your existing code.
Note
The Amazon SQS Buffered Asynchronous Client doesn't currently support FIFO queues.
Using AmazonSQSBufferedAsyncClient
Before you begin, complete the steps in Setting up Amazon SQS.
Important
The AWS SDK for Java 2.x isn't currently compatible with the
AmazonSQSBufferedAsyncClient
.
You can create a new AmazonSQSBufferedAsyncClient
based on
AmazonSQSAsyncClient
, for example:
// Create the basic Amazon SQS async client final AmazonSQSAsync sqsAsync = new AmazonSQSAsyncClient(); // Create the buffered client final AmazonSQSAsync bufferedSqs = new AmazonSQSBufferedAsyncClient(sqsAsync);
After you create the new AmazonSQSBufferedAsyncClient
, you can use it
to send multiple requests to Amazon SQS (just as you can with
AmazonSQSAsyncClient
), for example:
final CreateQueueRequest createRequest = new CreateQueueRequest().withQueueName("MyQueue"); final CreateQueueResult res = bufferedSqs.createQueue(createRequest); final SendMessageRequest request = new SendMessageRequest(); final String body = "Your message text" + System.currentTimeMillis(); request.setMessageBody( body ); request.setQueueUrl(res.getQueueUrl()); final Future<SendMessageResult> sendResult = bufferedSqs.sendMessageAsync(request); final ReceiveMessageRequest receiveRq = new ReceiveMessageRequest() .withMaxNumberOfMessages(1) .withQueueUrl(queueUrl); final ReceiveMessageResult rx = bufferedSqs.receiveMessage(receiveRq);
Configuring AmazonSQSBufferedAsyncClient
AmazonSQSBufferedAsyncClient
is preconfigured with settings that work
for most use cases. You can further configure
AmazonSQSBufferedAsyncClient
, for example:
-
Create an instance of the
QueueBufferConfig
class with the required configuration parameters. -
Provide the instance to the
AmazonSQSBufferedAsyncClient
constructor.
// Create the basic Amazon SQS async client final AmazonSQSAsync sqsAsync = new AmazonSQSAsyncClient(); final QueueBufferConfig config = new QueueBufferConfig() .withMaxInflightReceiveBatches(5) .withMaxDoneReceiveBatches(15); // Create the buffered client final AmazonSQSAsync bufferedSqs = new AmazonSQSBufferedAsyncClient(sqsAsync, config);
QueueBufferConfig configuration parameters | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Parameter | Default value | Description | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
longPoll |
true |
When |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
longPollWaitTimeoutSeconds |
20 s |
The maximum amount of time (in seconds) which a
NoteWhen long polling is disabled, this setting has no effect. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxBatchOpenMs |
200 ms |
The maximum amount of time (in milliseconds) that an outgoing call waits for other calls with which it batches messages of the same type. The higher the setting, the fewer batches are required to perform the same amount of work (however, the first call in a batch has to spend a longer time waiting). When you set this parameter to |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxBatchSize |
10 requests per batch |
The maximum number of messages that are batched together in a single request. The higher the setting, the fewer batches are required to carry out the same number of requests. Note10 requests per batch is the maximum allowed value for Amazon SQS. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxBatchSizeBytes |
256 KiB |
The maximum size of a message batch, in bytes, that the client attempts to send to Amazon SQS. Note256 KiB is the maximum allowed value for Amazon SQS. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxDoneReceiveBatches |
10 batches |
The maximum number of receive batches that
The higher the setting, the more receive requests can be satisfied without having to make a call to Amazon SQS (however, the more messages are prefetched, the longer they remain in the buffer, causing their own visibility timeout to expire). Note
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxInflightOutboundBatches |
5 batches |
The maximum number of active outbound batches that can be processed at the same time. The higher the setting, the faster outbound batches can be
sent (subject to quotas such as CPU or bandwidth) and the more
threads are consumed by
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
maxInflightReceiveBatches |
10 batches |
The maximum number of active receive batches that can be processed at the same time. The higher the setting, the more messages can be received
(subject to quotas such as CPU or bandwidth), and the more
threads are consumed by
Note
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
visibilityTimeoutSeconds |
-1 |
When this parameter is set to a positive, non-zero value, the visibility timeout set here overrides the visibility timeout set on the queue from which messages are consumed. Note
You can't set visibility timeout to |