CreateInferenceRouter
Creates an inference router. An inference router is a Regional SageMaker AI resource that sends each inference request to one of the endpoints or inference components that you list for a routing target. These destinations can be in any AWS account, and in any AWS Region in the same partition as the router.
The router's status is Creating until it's ready to use.
SageMaker AI checks that each endpoint is a real-time or asynchronous endpoint, as
the routing target requires, and rejects one of the wrong type. It checks endpoints in
other accounts with your credentials, so the endpoint policy must allow
sagemaker:DescribeEndpoint. SageMaker AI accepts an endpoint that it
can't check, for example one that doesn't exist. Requests to it fail if it's the wrong
type or doesn't exist. To see the status of each destination, call DescribeInferenceRouter and see the RoutingStatus of each
destination.
To invoke the router, use InvokeInferenceRouter or
InvokeInferenceRouterAsync. For information about inference routers, see
Route requests
across Regions with inference routers.
Request Syntax
{
"ClientToken": "string",
"InferenceRouterName": "string",
"RoutingTargets": [
{
"AsyncInferenceConfig": {
"ClientConfig": {
"MaxTimeInBacklogSeconds": number
},
"OutputConfig": {
"KmsKeyId": "string",
"NotificationConfig": {
"ErrorTopic": "string",
"SuccessTopic": "string"
},
"S3FailurePath": "string",
"S3OutputPath": "string"
}
},
"Destinations": [
{
"ResourceArn": "string",
"RoutingStatus": "string"
}
],
"Name": "string",
"RoutingPreference": "string"
}
],
"Tags": [
{
"Key": "string",
"Value": "string"
}
]
}
Request Parameters
For information about the parameters that are common to all actions, see Common Parameters.
The request accepts the following data in JSON format.
- ClientToken
-
A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. If you reuse a client token, use the same request parameters. Otherwise, the request fails.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 64.
Pattern:
[\x21-\x7E]+Required: No
- InferenceRouterName
-
The name of the inference router. The name must be unique within your account in the current AWS Region. The name is case sensitive.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 63.
Pattern:
[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}Required: Yes
- RoutingTargets
-
The routing targets of the inference router. Each routing target is a model that callers name when they invoke the router, and the destinations that serve it.
Type: Array of RoutingTarget objects
Array Members: Minimum number of 1 item. Maximum number of 20 items.
Required: Yes
- Tags
-
A list of key-value pairs to apply to the inference router. For information about tagging, see Tagging AWS resources in the AWS General Reference.
Type: Array of Tag objects
Array Members: Minimum number of 0 items. Maximum number of 50 items.
Required: No
Response Syntax
{
"InferenceRouterArn": "string"
}
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
- InferenceRouterArn
-
The Amazon Resource Name (ARN) of the inference router.
Type: String
Length Constraints: Minimum length of 20. Maximum length of 2048.
Pattern:
arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:inference-router/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}
Errors
For information about the errors that are common to all actions, see Common Error Types.
- ConflictException
-
There was a conflict when you attempted to modify a SageMaker entity such as an
ExperimentorArtifact.HTTP Status Code: 400
- ResourceAlreadyExists
-
The resource that you are trying to create already exists. Use a different resource name, or delete the existing resource before retrying.
HTTP Status Code: 400
- ResourceLimitExceeded
-
You have exceeded an SageMaker resource limit. For example, you might have too many training jobs created.
HTTP Status Code: 400
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: