PERF02-BP05 Use optimized hardware-based compute accelerators
Some workloads run far faster on specialized hardware than on a general-purpose processor. Accelerators, such as GPUs and purpose-built chips, handle massively parallel work such as model training or media processing at a throughput CPUs cannot match. They cost more than general-purpose processors and only pay off when the application offloads work to them and keeps them busy.
Desired outcome:
-
You run workloads that benefit from parallel hardware on accelerators, and you keep CPU-appropriate work on CPUs.
-
You benchmark accelerated instances against general-purpose options before committing, and you can show the accelerator wins on performance or cost.
-
You monitor accelerator utilization as a distinct signal, not just CPU and memory.
Common anti-patterns:
-
Teams have not benchmarked a general-purpose instance against a purpose-built one that could deliver higher performance at lower cost.
-
Teams do not monitor GPU utilization, so they cannot tell whether the hardware is being used.
-
Teams leave accelerated instances running after the job finishes.
Benefits of establishing this best practice:
-
Suitable workloads complete faster because parallel hardware handles work CPUs process serially.
-
Cost per unit of work drops when a purpose-built accelerator outperforms a larger general-purpose fleet.
-
Utilization stays high because accelerators run only while there is work for them.
-
Performance keeps improving as you adopt optimized libraries and newer accelerator generations.
Level of risk exposed if this best practice is not established: Medium
Implementation guidance
GPUs excel at the massively parallel arithmetic behind rendering, transcoding, and machine learning. You can configure field programmable gate arrays (FPGAs) for custom operations, such as data pattern matching. Purpose-built chips target a single domain, such as machine learning training or inference, and trade generality for efficiency.
Before choosing an accelerator, benchmark your workload on general-purpose and accelerated instances to compare performance and cost.
Accelerated workloads such as rendering, transcoding, and training tend to be spiky, with intense use followed by idle time. For this pattern, consider alternatives to on-demand capacity, such as Amazon EC2 Capacity Blocks for ML.
An accelerator only helps if the software uses it. Gains depend on drivers, runtime libraries, and application code that offload the right operations to the hardware. Keep them current and tune I/O so data reaches the accelerator without stalling.
On AWS,
accelerated computing instances provide GPUs and FPGAs. Purpose-built
chips such as
AWS Trainium
Implementation steps
-
Confirm your application supports acceleration: Verify that the workload has parallelizable operations and that its frameworks or libraries can offload them to an accelerator.
-
Identify the accelerated instances that fit: Match your requirements to the available accelerated computing instances, and benchmark the candidates against a general-purpose baseline on both performance and cost.
-
Use purpose-built hardware where it applies: For machine learning, evaluate AWS Trainium
for training and AWS Inferentia for inference, which target these tasks specifically. -
Collect accelerator utilization metrics: Track accelerator-specific signals, such as utilization_gpu and utilization_memory with configuring the CloudWatch agent for NVIDIA GPUs, so you can tell whether the hardware is actually being used.
-
Optimize code, drivers, and settings: Tune the application, I/O, and hardware settings so the accelerator stays fully fed, and keep drivers and performance libraries current. Check out NVIDIA drivers for your Amazon EC2 instance and GPU monitoring and optimization in the Deep Learning AMI.
-
Release capacity when idle: Automate teardown of accelerated instances when the job completes, so expensive hardware does not run without work.
Resources
Related best practices:
Related documents:
Related videos: