MLPER-05: Optimize training and inference instance types

Determine how the model type and data velocity affect the choice of training and inference instance types. Identify the right instance type that supports memory intensive training, or compute intensive training with high throughput and low latency real-time inference. The speed of model inferences is directly impacted by model complexity. Selection of high compute instances can accelerate inference speed. GPUs are often the preferred processor type to train many deep learning models. CPUs are often sufficient for the inference workloads.

Implementation plan

• Experiment with alternative instance types to train and deploy - Determine which instance types are most appropriate for your ML algorithm and use case. Use multiple instances for training for large datasets to take advantage of scale.

Documents

• Use Amazon SageMaker Notebook Instances

• Train a Model with Amazon SageMaker

• Distributed training libraries

Blogs

• Learn how to select ML instances on the fly in Amazon SageMaker Studio

• Optimizing I/O for GPU performance tuning of deep learning training in Amazon SageMaker

• Right-sizing resources and avoiding unnecessary costs in Amazon SageMaker

Videos

• How to choose the right instance type for ML inference

• The right instance type in Amazon SageMaker

Examples

• Amazon SageMaker End-to-End Example