SageMaker / Client / create_ai_recommendation_job

create_ai_recommendation_job

SageMaker.Client.create_ai_recommendation_job(**kwargs)

Creates a recommendation job that generates intelligent optimization recommendations for generative AI inference deployments. The job analyzes your model, workload configuration, and performance targets to recommend optimal instance types, model optimization techniques (such as quantization and speculative decoding), and deployment configurations.

See also: AWS API Documentation

Request Syntax

response = client.create_ai_recommendation_job(
    AIRecommendationJobName='string',
    ModelSource={
        'S3': {
            'S3Uri': 'string'
        }
    },
    OutputConfig={
        'S3OutputLocation': 'string',
        'ModelPackageGroupIdentifier': 'string'
    },
    AIWorkloadConfigIdentifier='string',
    PerformanceTarget={
        'Constraints': [
            {
                'Metric': 'ttft-ms'|'throughput'|'cost'
            },
        ]
    },
    RoleArn='string',
    InferenceSpecification={
        'Framework': 'LMI'|'VLLM'
    },
    OptimizeModel=True|False,
    ComputeSpec={
        'InstanceTypes': [
            'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge',
        ],
        'CapacityReservationConfig': {
            'CapacityReservationPreference': 'capacity-reservations-only',
            'MlReservationArns': [
                'string',
            ]
        }
    },
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
)

Parameters:

• AIRecommendationJobName (string) –

  [REQUIRED]

  The name of the AI recommendation job. The name must be unique within your Amazon Web Services account in the current Amazon Web Services Region.

• ModelSource (dict) –

  [REQUIRED]

  The source of the model to optimize. Specify the Amazon S3 location of the model artifacts.

  Note

  This is a Tagged Union structure. Only one of the following top level keys can be set: S3.

  • S3 (dict) –

    The Amazon S3 location of the model artifacts.

    • S3Uri (string) –

      The Amazon S3 URI of the model artifacts.

• OutputConfig (dict) –

  [REQUIRED]

  The output configuration for the recommendation job, including the Amazon S3 location for results and an optional model package group where the optimized model is registered.

  • S3OutputLocation (string) –

    The Amazon S3 URI where recommendation results are stored.

  • ModelPackageGroupIdentifier (string) –

    The name or Amazon Resource Name (ARN) of the model package group where the optimized model is registered as a new model package version.

• AIWorkloadConfigIdentifier (string) –

  [REQUIRED]

  The name or Amazon Resource Name (ARN) of the AI workload configuration to use for this recommendation job.

• PerformanceTarget (dict) –

  [REQUIRED]

  The performance targets for the recommendation job. Specify constraints on metrics such as time to first token ( ttft-ms), throughput, or cost.

  • Constraints (list) – [REQUIRED]

    An array of performance constraints that define the optimization objectives.

    • (dict) –

      A performance constraint for an AI recommendation job.

      • Metric (string) – [REQUIRED]

        The performance metric. Valid values are ttft-ms (time to first token in milliseconds), throughput, and cost.

• RoleArn (string) –

  [REQUIRED]

  The Amazon Resource Name (ARN) of an IAM role that enables Amazon SageMaker AI to perform tasks on your behalf.

• InferenceSpecification (dict) –

  The inference framework configuration. Specify the framework (such as LMI or vLLM) for the recommendation job.

  • Framework (string) –

    The inference framework. Valid values are LMI and VLLM.

• OptimizeModel (boolean) – Whether to allow model optimization techniques such as quantization, speculative decoding, and kernel tuning. The default is true.

• ComputeSpec (dict) –

  The compute resource specification for the recommendation job. You can specify up to 3 instance types to consider, and optionally provide capacity reservation configuration.

  • InstanceTypes (list) –

    The list of instance types to consider for recommendations. You can specify up to 3 instance types.

    • (string) –

  • CapacityReservationConfig (dict) –

    The capacity reservation configuration.

    • CapacityReservationPreference (string) –

      The capacity reservation preference. The only valid value is capacity-reservations-only.

    • MlReservationArns (list) –

      The list of ML reservation ARNs to use.

      • (string) –

• Tags (list) –

  The metadata that you apply to Amazon Web Services resources to help you categorize and organize them.

  • (dict) –

    A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.

    You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.

    For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.

    • Key (string) – [REQUIRED]

      The tag key. Tag keys must be unique per resource.

    • Value (string) – [REQUIRED]

      The tag value.

Return type:

dict

Returns:

Response Syntax

{
    'AIRecommendationJobArn': 'string'
}

Response Structure

• (dict) –

  • AIRecommendationJobArn (string) –

    The Amazon Resource Name (ARN) of the created recommendation job.

Exceptions

• SageMaker.Client.exceptions.ResourceNotFound

• SageMaker.Client.exceptions.ResourceInUse

• SageMaker.Client.exceptions.ResourceLimitExceeded