AWS::SageMaker::InferenceComponent InferenceComponentComputeResourceRequirements

Defines the compute resources to allocate to run a model, plus any adapter models, that you assign to an inference component. These resources include CPU cores, accelerators, and memory.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

{
  "MaxMemoryRequiredInMb" : Integer,
  "MinMemoryRequiredInMb" : Integer,
  "NumberOfAcceleratorDevicesRequired" : Number,
  "NumberOfCpuCoresRequired" : Number
}

YAML

  MaxMemoryRequiredInMb: Integer
  MinMemoryRequiredInMb: Integer
  NumberOfAcceleratorDevicesRequired: 
    Number
  NumberOfCpuCoresRequired: 
    Number

Properties

MaxMemoryRequiredInMb

The maximum MB of memory to allocate to run a model that you assign to an inference component.

Required: No

Type: Integer

Minimum: 128

Update requires: No interruption

MinMemoryRequiredInMb

The minimum MB of memory to allocate to run a model that you assign to an inference component.

Required: No

Type: Integer

Minimum: 128

Update requires: No interruption

NumberOfAcceleratorDevicesRequired

The number of accelerators to allocate to run a model that you assign to an inference component. Accelerators include GPUs and AWS Inferentia.

Required: No

Type: Number

Minimum: 1

Update requires: No interruption

NumberOfCpuCoresRequired

The number of CPU cores to allocate to run a model that you assign to an inference component.

Required: No

Type: Number

Minimum: 0

Update requires: No interruption