Retry behavior

Retry behavior includes settings regarding how the SDKs attempt to recover from failures resulting from requests made to AWS services.

Configure this functionality by using the following:

max_attempts - shared AWS config file setting

AWS_MAX_ATTEMPTS - environment variable

aws.maxAttempts - JVM system property: Java/Kotlin only

Specifies the maximum number attempts to make on a request.

Default value: If this value is not specified, its default depends on the value of the retry_mode setting:

• If retry_mode is legacy – Uses a default value specific to your SDK (check your specific SDK guide or your SDK's code base for max_attempts default).

• If retry_mode is standard – Makes three attempts.

• If retry_mode is adaptive – Makes three attempts.

Valid values: Number greater than 0.

retry_mode - shared AWS config file setting

AWS_RETRY_MODE - environment variable

aws.retryMode - JVM system property: Java/Kotlin only

Specifies how the SDK or developer tool attempts retries.

Default value: legacy is the default retry strategy.

Valid values:

• legacy – Specific to your SDK (check your specific SDK guide or your SDK's code base).

• standard – The standard set of retry rules across AWS SDKs. This mode includes a standard set of errors that are retried, and support for retry quotas. The default maximum number of attempts with this mode is three, unless max_attempts is explicitly configured.

• adaptive – An experimental retry mode that includes the functionality of standard mode but includes automatic client-side throttling. Because this mode is experimental, it might change behavior in the future.

Choosing between standard and adaptive retry modes

We recommend you use the standard retry mode unless you are certain that your usage is better suited for adaptive.

Note

The adaptive mode assumes that you are pooling clients based on the scope at which the backend service may throttle requests. If you don’t do this, throttles in one resource could delay requests for an unrelated resource if you are using the same client for both resources.

Standard	Adaptive
Application use-cases: All.	Application use-cases: Not sensitive to latency. Client only accesses a single resource, or, you are providing logic to pool your clients separately by the service resource that is being accessed.
Supports circuit-breaking to prevent the SDK from retrying during outages.	Supports circuit-breaking to prevent the SDK from retrying during outages.
Uses jittered exponential backoff in the event of failures.	Uses dynamic backoff durations to attempt to minimize the number of failed requests, in exchange for the potential for increased latency.
Never delays the first request attempt, only the retries.	Can throttle or delay the initial request attempt.

If you choose to use adaptive mode, your application must construct clients that are designed around each resource that might be throttled. A resource, in this case, is finer-tuned than just thinking of each AWS service. AWS services can have additional dimensions that they use to throttle requests. Let's use the Amazon DynamoDB service as an example. DynamoDB uses AWS Region plus the table being accessed to throttle requests. This means that one table that your code is accessing might be throttled more than others. If your code used the same client to access all the tables, and requests to one of those tables is throttled, then adaptive retry mode will reduce the request rate for all tables. Your code should be designed to have one client per Region-and-table pair. If you experience unexpected latency when using adaptive mode, see the specific AWS documentation guide for the service you are using.

Retry mode implementation details

Following is the high-level pseudocode for both the standard and adaptive retry modes:

MakeSDKRequest() {
  attempts = 0
  loop {
    GetSendToken()
    response = SendHTTPRequest()
    RequestBookkeeping(response)
    if not Retryable(response)
      return response
    attempts += 1
    if attempts >= MAX_ATTEMPTS:
      return response
    if not HasRetryQuota(response)
      return response
    delay = ExponentialBackoff(attempts)
    sleep(delay)
  }
}

Following are more details about the components used in the pseudocode:

GetSendToken:

Token buckets are only used in adaptive retry mode. Token buckets enforce a maximum request rate by requiring a token to be available in order to initiate a request. The SDK client is configurable to either fast fail the request or block until a token becomes available.

Client Side Rate Limiting is an algorithm that initially lets requests be made at any rate up to the token allowance. However, after a throttled response is detected, the client rate-of-request is then limited accordingly. The token allowance is also increased accordingly if successful responses are received.

With adaptive rate limiting, SDKs can slow down the rate at which requests are sent in order to better accommodate the capacity of AWS services.

SendHTTPRequest:

Most AWS SDKs use an HTTP library that uses connection pools so that you can reuse an existing connection when making an HTTP request. Generally, connections are reused when retrying requests due to throttling errors. Requests are not reused when retrying due to transient errors.

RequestBookkeeping:

The retry quota should be updated if the request is successful. For adaptive retry mode only, the state variable maxsendrate is updated based on the type of response received.

Retryable:

This step determines whether a response can be retried based on the following:

• The HTTP status code.

• The error code returned from the service.

• Connection errors, defined as any error received by the SDK in which an HTTP response from the service is not received.

Transient errors (HTTP status codes 400, 408, 500, 502, 503, and 504) and throttling errors (HTTP status codes 400, 403, 429, 502, 503, and 509) can all potentially be retried. SDK retry behavior is determined in combination with error codes or other data from the service.

MAX_ATTEMPTS:

Specified by the config file setting or the environment variable.

HasRetryQuota

This step throttles retry requests by requiring a token to be available in the retry quota bucket. Retry quota buckets are a mechanism to prevent retries that are unlikely to succeed. These quotas are SDK-dependent, are often client-dependent, and are sometimes even dependent on service endpoints. The available retry quota tokens are removed when requests fail for various reasons, and replenished when they succeed. When no tokens remain, the retry loop is exited.

ExponentialBackoff

For an error that can be retried, the retry delay is calculated using truncated exponential backoff. The SDKs use truncated binary exponential backoff with jitter. The following algorithm shows how the amount of time to sleep, in seconds, is defined for a response for request i:

seconds_to_sleep_i = min(b*r^i, MAX_BACKOFF)

In the preceding algorithm, the following values apply:

b = random number within the range of: 0 <= b <= 1

r = 2

MAX_BACKOFF = 20 seconds for most SDKs. See your specific SDK guide or source code for confirmation.

Compatibility with AWS SDKs

The following SDKs support the features and settings described in this topic. Any partial exceptions are noted. Any JVM system property settings are supported by the AWS SDK for Java and the AWS SDK for Kotlin only.

SDK	Supported	Notes or more information
AWS CLI v2	Yes
SDK for C++	Yes
SDK for Go V2 (1.x)	Yes
SDK for Go 1.x (V1)	No
SDK for Java 2.x	Yes
SDK for Java 1.x	Yes
SDK for JavaScript 3.x	Yes
SDK for JavaScript 2.x	No	Supports a maximum number of retries, exponential backoff with jitter, and an option for a custom method for retry backoff.
SDK for Kotlin	Yes
SDK for .NET 3.x	Yes
SDK for PHP 3.x	Yes
SDK for Python (Boto3)	Yes
SDK for Ruby 3.x	Yes
SDK for Rust	Yes
Tools for PowerShell	Yes