Model parallelism and large model inference

Amazon SageMaker includes specialized deep learning containers (DLCs), libraries, and tooling for model parallelism and large model inference (LMI). In the following sections, you can find resources to get started with LMI on SageMaker.

Topics

• The large model inference (LMI) container documentation
• SageMaker endpoint parameters for large model inference
• Deploying uncompressed models
• Large model inference with TorchServe