View a markdown version of this page

REL05-BP05 Set client timeouts - AWS Well-Architected Framework

REL05-BP05 Set client timeouts

A call with no timeout can wait forever and hold resources while a dependency is impaired. Set both connection and request timeouts on every call across a process boundary, verify them, and do not rely on framework defaults. Framework defaults do not reflect your workload and are often infinite or far higher than your service goals allow.

Desired outcome:

  • Your service dependency calls have explicit connection and request timeouts based on measured, normal completion times.

  • Your clients handle timeouts gracefully, backing off and retrying or opening a circuit breaker rather than reissuing calls that worsen an impairment.

  • Your resources are conserved when a remote call responds abnormally slowly, because the timeout releases them.

  • Your timeout values are tested for both connection and request scenarios rather than assumed.

Common anti-patterns:

  • Teams are unaware of system or default timeout values.

  • Teams do not know the normal completion time of the requests they make.

  • Teams do not consider why a request might take abnormally long, or the cost of waiting on it.

  • Timeout scenarios are not tested for connections or requests.

  • Timeouts set too high cause long waits and hold resources while the client waits.

  • Timeouts set too low cause artificial failures and retry-driven traffic increases.

  • Patterns for handling timeout errors, such as circuit breakers and retries, are overlooked.

  • Error rates, latency service level objectives, and latency outliers are not monitored, so timeout values cannot be tuned.

Benefits of establishing this best practice:

  • Resources are conserved when remote calls respond abnormally slowly, because timeouts release them promptly.

  • Timeout errors are handled gracefully by clients instead of stalling the workload.

  • Timeout values can be tuned over time, because monitoring reveals whether they are too aggressive or too permissive.

Level of risk exposed if this best practice is not established: High

Implementation guidance

A timeout shorter than a request's normal completion time manufactures failures out of requests that would have succeeded. The resulting retries add load to the backend and latency to the client, and at worst every slow response becomes a retry and overloads the backend.

The right timeout depends on why a request is slow. When a request is expensive by design, time it out without retry to preserve resources on both sides. A localized service impairment usually calls for a timeout followed by a retry, which is cheap and conserves client resources. If the network fails to deliver the request or response, the call fails regardless, so an early timeout releases client resources at no cost.

A circuit breaker stops issuing calls to a dependency that is timing out and tests periodically for recovery. Retries with exponential backoff and jitter handle transient failures. Together, timeouts, circuit breakers, and bounded retries complement fail-fast.

Use the AWS SDKs and AWS CLI to configure connection and request timeouts alongside retries with backoff and jitter. AWS Lambda functions have a configurable timeout, and AWS Step Functions provides pre-built service integrations for low-code circuit breakers. For container workloads, Amazon VPC Lattice provides timeout and retry controls at the service network level. Whichever you use, instrument error rates and latency outliers to tune timeouts over time.

Implementation steps

  1. Set connection and request timeouts on remote calls: Configure both timeout types on every call across a process boundary, using built-in language timeout features or an open-source timeout library. Base the values on measured normal completion times, not defaults.

  2. Configure AWS SDK timeouts per language: When your workload calls an AWS SDK, review the language-specific timeout configuration for Python, PHP, .NET, Ruby, Java, Go, Node.js, and C++.

  3. Set SDK and CLI default timeout values: For AWS SDK and AWS CLI usage, set configuration defaults for connectTimeoutInMillis and tlsNegotiationTimeoutInMillis. Apply the command line options cli-connect-timeout and cli-read-timeout for one-off CLI commands.

  4. Configure timeouts on compute and integration services: Set timeouts on Lambda functions. Account for the API Gateway 50-millisecond to 29-second integration timeout, and implement client retries because API Gateway does not retry a timed-out integration.

  5. Monitor timeouts and tune values: Set alarms on persistent timeout errors, and use CloudWatch metrics and CloudWatch anomaly detection on error rates, latency service level objectives, and latency outliers to tune overly aggressive or permissive timeouts.

  6. Implement circuit breakers for timing-out calls: Apply the circuit breaker pattern to avoid calling a dependency that is timing out, opening the circuit during failures and closing it when calls respond normally. Build low-code circuit breakers with AWS Step Functions, or use Amazon VPC Lattice timeout and retry features for container and service workloads.

Resources

Related best practices:

Related documents:

Related examples:

Related tools: