SUS03-BP02 Flatten demand peaks for consistently high resource utilization
When demand is unpredictable and spiky, provision enough capacity to meet peak loads. Techniques such as buffering and throttling smooth out the load and maintain a consistently high level of resource utilization, reducing the need for excess capacity to handle peaks and improving overall efficiency. By transforming spiky workload patterns into steadier, more predictable ones, you can provision resources that operate at high utilization and resource efficiency levels most of the time.
Desired outcome:
-
You smooth demand patterns so your workload runs efficiently on more consistently utilized infrastructure, reducing the excess capacity that sits idle during normal operations.
-
You achieve higher average resource utilization and a smaller environmental footprint.
Common anti-patterns:
-
Provisioning for peak demand without effective mechanisms to smooth demand.
-
Processing non-urgent workloads or operations immediately, even when non-functional requirements allow time for buffering.
-
Provisioning infrastructure for peak load that rarely or never occurs.
-
Not scaling in or down at times of low demand.
-
Failing to define limits or implement mechanisms for throttling traffic spikes.
Benefits of establishing this best practice:
-
Minimizing idle or underutilized resources to handle the demand lowers the required share on hardware related impacts, from raw material extraction to end of life treatment (Scope 3 emissions).
-
High utilization of resources reduces the effect of energy overhead spent for operating the infrastructure (Scope 2 emissions).
Level of risk exposed if this best practice is not established: Medium
Implementation guidance
Implementing mechanisms to flatten demand peaks requires understanding your workload patterns and applying appropriate techniques to smooth out spikes. This may involve adding queuing mechanisms, buffering, throttling, or redesigning processes to operate in batches during periods of lower demand.
Implementation steps
-
Analyze your workload demand patterns:
-
Use Amazon CloudWatch metrics
to identify peak periods and their frequency. -
Categorize workload components based on their time-sensitivity requirements.
-
Identify opportunities to defer or batch non-critical processing.
-
Continuously evaluate the balance between responsiveness and resource efficiency.
-
Consider adding caching when receiving consistently high volumes of identical incoming requests that can be serviced with the same response.
-
-
Implement queuing and buffering for asynchronous workloads:
-
Use Amazon SQS
to decouple components and buffer workload. -
Implement auto-scaling based on queue depth using metric math.
-
Implement Amazon Kinesis
with micro-buffering for real-time data streaming with controlled processing. -
Configure appropriate processing rates and concurrency limits for consumers.
-
-
Apply throttling for request-heavy APIs and services:
-
Configure API Gateway
throttling to limit request rates. -
Implement client-side retry mechanisms and leverage the AWS SDKs implementation of exponential backoff and jitter.
-
Consider different throttling limits for different client types or operations.
-
Use circuit breakers to turn off non-critical features and protect the systems and user experience in periods of high demand.
-
-
Monitor and optimize your flattening mechanisms:
-
Monitor performance metrics such as queue depth, processing latency, and throttling events.
-
Set thresholds on performance metrics to detect exceptions, and drive automated scaling actions when necessary to maintain SLOs.
-
Regularly review and adjust buffer sizes, processing rates, and throttling limits as needed.
-
Resources
Related best practices:
Related documents:
Related videos:
Related examples: