InferenceComponentComputeResourceRequirementsProperty Struct
Defines the compute resources to allocate to run a model, plus any adapter models, that you assign to an inference component.
These resources include CPU cores, accelerators, and memory.
Example
ts
// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import { aws_sagemaker as sagemaker } from 'aws-cdk-lib';
const inferenceComponentComputeResourceRequirementsProperty: sagemaker.CfnInferenceComponent.InferenceComponentComputeResourceRequirementsProperty = {
maxMemoryRequiredInMb: 123,
minMemoryRequiredInMb: 123,
numberOfAcceleratorDevicesRequired: 123,
numberOfCpuCoresRequired: 123,
};
Properties
maxMemoryRequiredInMboptionalnumbermax_memory_required_in_mboptionalint | floatmaxMemoryRequiredInMboptionalNumberMaxMemoryRequiredInMboptionaldoubleMaxMemoryRequiredInMboptional*float64#The maximum MB of memory to allocate to run a model that you assign to an inference component.
minMemoryRequiredInMboptionalnumbermin_memory_required_in_mboptionalint | floatminMemoryRequiredInMboptionalNumberMinMemoryRequiredInMboptionaldoubleMinMemoryRequiredInMboptional*float64#The minimum MB of memory to allocate to run a model that you assign to an inference component.
numberOfAcceleratorDevicesRequiredoptionalnumbernumber_of_accelerator_devices_requiredoptionalint | floatnumberOfAcceleratorDevicesRequiredoptionalNumberNumberOfAcceleratorDevicesRequiredoptionaldoubleNumberOfAcceleratorDevicesRequiredoptional*float64#The number of accelerators to allocate to run a model that you assign to an inference component.
Accelerators include GPUs and AWS Inferentia.
numberOfCpuCoresRequiredoptionalnumbernumber_of_cpu_cores_requiredoptionalint | floatnumberOfCpuCoresRequiredoptionalNumberNumberOfCpuCoresRequiredoptionaldoubleNumberOfCpuCoresRequiredoptional*float64#The number of CPU cores to allocate to run a model that you assign to an inference component.