[ aws . sagemaker ]

describe-inference-router

Description

Returns information about an inference router, including the routing status of each destination.

SageMaker AI checks each destination with your permissions when you call this operation, so callers with different permissions can see a different RoutingStatus . To get the router’s tags, use ListTags .

See also: AWS API Documentation

Synopsis

  describe-inference-router
--inference-router-name <value>
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
[--cli-error-format <value>]
[--help]

Options

--inference-router-name (string) [required]

The name of the inference router to describe.

Constraints:

  • min: 1
  • max: 63
  • pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

--cli-input-json | --cli-input-yaml (string) Reads arguments from the JSON string provided. The JSON string follows the format provided by --generate-cli-skeleton. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with --cli-input-yaml.

--generate-cli-skeleton (string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value input, prints a sample input JSON that can be used as an argument for --cli-input-json. Similarly, if provided yaml-input it will print a sample input YAML that can be used with --cli-input-yaml. If provided with the value output, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.

Global Options

--debug (boolean)

Turn on debug logging.

--endpoint-url (string)

Override command’s default URL with the given URL.

--no-verify-ssl (boolean)

By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

--no-paginate (boolean)

Disable automatic pagination. If automatic pagination is disabled, the AWS CLI will only make one call, for the first page of results.

--output (string)

The formatting style for command output.

  • json
  • text
  • table
  • yaml
  • yaml-stream
  • off

--query (string)

A JMESPath query to use in filtering the response data.

--profile (string)

Use a specific profile from your credential file.

--region (string)

The region to use. Overrides config/env settings.

--version (string)

Display the version of this tool.

--color (string)

Turn on/off color output.

  • on
  • off
  • auto

--no-sign-request (boolean)

Do not sign requests. Credentials will not be loaded if this argument is provided.

--ca-bundle (string)

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.

--cli-read-timeout (int)

The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.

--cli-connect-timeout (int)

The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.

--cli-binary-format (string)

The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob fileb:// will always be treated as binary and use the file contents directly regardless of the cli-binary-format setting. When using file:// the file contents will need to properly formatted for the configured cli-binary-format.

  • base64
  • raw-in-base64-out

--no-cli-pager (boolean)

Disable cli pager for output.

--cli-auto-prompt (boolean)

Automatically prompt for CLI input parameters.

--no-cli-auto-prompt (boolean)

Disable automatically prompt for CLI input parameters.

--cli-error-format (string)

The formatting style for error output. By default, errors are displayed in enhanced format.

  • legacy
  • json
  • yaml
  • text
  • table
  • enhanced

--help (boolean)

Display help for the command/subcommand.

Output

InferenceRouterName -> (string)

The name of the inference router.

Constraints:

  • min: 1
  • max: 63
  • pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

InferenceRouterArn -> (string)

The Amazon Resource Name (ARN) of the inference router.

Constraints:

  • min: 20
  • max: 2048
  • pattern: arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:inference-router/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

RoutingTargets -> (list)

The routing targets of the inference router. Each destination includes its RoutingStatus . You can send these routing targets back in an UpdateInferenceRouter request without changes.

Constraints:

  • min: 1
  • max: 20

(structure)

A model that an inference router serves, and the destinations that can serve it. When you invoke the router, you name the routing target, and the router sends the request to one of its destinations.

Every destination in a routing target must serve the same model, with the same request and response format.

Name -> (string) [required]

The name of the routing target. Callers specify this name in the RoutingTargetName parameter when they invoke the inference router. The name must be unique within the router.

Constraints:

  • min: 1
  • max: 63
  • pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

Destinations -> (list) [required]

The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint . If an endpoint can’t be checked, for example because it isn’t found or can’t be read in time, it’s accepted, and requests to it fail if it’s the wrong mode.

Constraints:

  • min: 1
  • max: 10

(structure)

An endpoint or inference component that serves a routing target.

ResourceArn -> (string) [required]

The Amazon Resource Name (ARN) of the endpoint or inference component. The destination can be in any account, and in any Amazon Web Services Region in the same partition as the inference router. To use a destination in another account, its owner must allow your account to invoke it through your router, in an endpoint policy.

Constraints:

  • min: 20
  • max: 2048
  • pattern: arn:aws[a-z-]*:sagemaker:[a-z0-9-]+:[0-9]{12}:(endpoint|inference-component)/[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

RoutingStatus -> (string)

Whether the inference router can send requests to the destination, as checked when you called DescribeInferenceRouter . This value isn’t the destination’s own status. SageMaker AI ignores this field in CreateInferenceRouter and UpdateInferenceRouter requests.

Routable

The destination is serving and the router can send it requests. A routable destination can still be overloaded.

Unroutable

The destination can’t serve requests. For example, it’s being created or has failed, or it has no capacity and no way to get any. A destination in your account that doesn’t exist is also Unroutable .

Unknown

SageMaker AI couldn’t check the destination, for example because you aren’t allowed to describe it or the check timed out. The destination might still be serving.

Possible values:

  • Routable
  • Unroutable
  • Unknown

AsyncInferenceConfig -> (structure)

The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

Asynchronous routing targets aren’t available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router .

Because UpdateInferenceRouter replaces all routing targets, include this setting in every update of an asynchronous routing target.

For information about asynchronous routing targets, see Send asynchronous requests through an inference router .

OutputConfig -> (structure) [required]

The Amazon S3 locations for the results of asynchronous requests, and the notification settings.

S3OutputPath -> (string) [required]

The Amazon S3 location for the results of successful requests. The bucket must be in the inference router’s Amazon Web Services Region and owned by the router’s account.

Constraints:

  • min: 1
  • max: 512
  • pattern: (https|s3)://([^/])/?(.*)

S3FailurePath -> (string) [required]

The Amazon S3 location for the failure details of failed requests. The bucket must be in the inference router’s Amazon Web Services Region and owned by the router’s account.

Constraints:

  • min: 1
  • max: 512
  • pattern: (https|s3)://([^/])/?(.*)

KmsKeyId -> (string)

The KMS key that encrypts the results in Amazon S3. You can specify a key ID, key ARN, alias name, or alias ARN. If any destination is in a different account or Amazon Web Services Region from the inference router, specify a key ARN or an alias ARN.

Constraints:

  • min: 0
  • max: 2048
  • pattern: [a-zA-Z0-9:/_-]*

NotificationConfig -> (structure)

The Amazon SNS topics that the inference router publishes to when a request completes or fails.

SuccessTopic -> (string)

The ARN of the Amazon SNS topic to publish to when a request completes. The topic must be a standard topic, and can be in any Amazon Web Services Region.

Constraints:

  • min: 20
  • max: 2048
  • pattern: arn:aws[a-z-]*:sns:[a-z0-9-]*:[0-9]{12}:[a-zA-Z0-9_.-]+

ErrorTopic -> (string)

The ARN of the Amazon SNS topic to publish to when a request fails, or when the router can’t place it on any destination. The topic must be a standard topic, and can be in any Amazon Web Services Region.

Constraints:

  • min: 20
  • max: 2048
  • pattern: arn:aws[a-z-]*:sns:[a-z0-9-]*:[0-9]{12}:[a-zA-Z0-9_.-]+

ClientConfig -> (structure)

How the inference router submits requests to destinations.

MaxTimeInBacklogSeconds -> (integer)

The maximum time, in seconds, that a request can wait at a destination before the inference router moves it to another destination. The default is 900 seconds.

Constraints:

  • min: 60
  • max: 21600

RoutingPreference -> (string)

How the inference router chooses among the routing target’s destinations. The default is LocalRegionFirst .

LocalRegionFirst

The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

InstancePoolOrder

The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn’t exclude them.

Possible values:

  • LocalRegionFirst
  • InstancePoolOrder

Status -> (string)

The status of the inference router.

Possible values:

  • InService
  • Creating
  • Updating
  • Failed
  • Deleting

StatusReason -> (string)

If the status is Failed , the reason that the inference router failed.

Constraints:

  • min: 0
  • max: 1024

CreationTime -> (timestamp)

The date and time that the inference router was created.

LastModifiedTime -> (timestamp)

The date and time that the inference router was last updated.