View a markdown version of this page

CreateInferenceRouter - Amazon SageMaker

CreateInferenceRouter

Creates an inference router. An inference router is a Regional SageMaker AI resource that sends each inference request to one of the endpoints or inference components that you list for a routing target. These destinations can be in any AWS account, and in any AWS Region in the same partition as the router.

The router's status is Creating until it's ready to use. SageMaker AI checks that each endpoint is a real-time or asynchronous endpoint, as the routing target requires, and rejects one of the wrong type. It checks endpoints in other accounts with your credentials, so the endpoint policy must allow sagemaker:DescribeEndpoint. SageMaker AI accepts an endpoint that it can't check, for example one that doesn't exist. Requests to it fail if it's the wrong type or doesn't exist. To see the status of each destination, call DescribeInferenceRouter and see the RoutingStatus of each destination.

To invoke the router, use InvokeInferenceRouter or InvokeInferenceRouterAsync. For information about inference routers, see Route requests across Regions with inference routers.

Request Syntax

{ "ClientToken": "string", "InferenceRouterName": "string", "RoutingTargets": [ { "AsyncInferenceConfig": { "ClientConfig": { "MaxTimeInBacklogSeconds": number }, "OutputConfig": { "KmsKeyId": "string", "NotificationConfig": { "ErrorTopic": "string", "SuccessTopic": "string" }, "S3FailurePath": "string", "S3OutputPath": "string" } }, "Destinations": [ { "ResourceArn": "string", "RoutingStatus": "string" } ], "Name": "string", "RoutingPreference": "string" } ], "Tags": [ { "Key": "string", "Value": "string" } ] }

Request Parameters

For information about the parameters that are common to all actions, see Common Parameters.

The request accepts the following data in JSON format.

ClientToken

A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. If you reuse a client token, use the same request parameters. Otherwise, the request fails.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [\x21-\x7E]+

Required: No

InferenceRouterName

The name of the inference router. The name must be unique within your account in the current AWS Region. The name is case sensitive.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 63.

Pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

Required: Yes

RoutingTargets

The routing targets of the inference router. Each routing target is a model that callers name when they invoke the router, and the destinations that serve it.

Type: Array of RoutingTarget objects

Array Members: Minimum number of 1 item. Maximum number of 20 items.

Required: Yes

Tags

A list of key-value pairs to apply to the inference router. For information about tagging, see Tagging AWS resources in the AWS General Reference.

Type: Array of Tag objects

Array Members: Minimum number of 0 items. Maximum number of 50 items.

Required: No

Response Syntax

{ "InferenceRouterArn": "string" }

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

InferenceRouterArn

The Amazon Resource Name (ARN) of the inference router.

Type: String

Length Constraints: Minimum length of 20. Maximum length of 2048.

Pattern: arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:inference-router/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

Errors

For information about the errors that are common to all actions, see Common Error Types.

ConflictException

There was a conflict when you attempted to modify a SageMaker entity such as an Experiment or Artifact.

HTTP Status Code: 400

ResourceAlreadyExists

The resource that you are trying to create already exists. Use a different resource name, or delete the existing resource before retrying.

HTTP Status Code: 400

ResourceLimitExceeded

You have exceeded an SageMaker resource limit. For example, you might have too many training jobs created.

HTTP Status Code: 400

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: