Class: Aws::SageMaker::Types::RoutingTarget

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb

Overview

A model that an inference router serves, and the destinations that can serve it. When you invoke the router, you name the routing target, and the router sends the request to one of its destinations.

Every destination in a routing target must serve the same model, with the same request and response format.

Constant Summary collapse

SENSITIVE =
[]

Instance Attribute Summary collapse

Instance Attribute Details

#async_inference_config ⇒ Types::RoutingTargetAsyncInferenceConfig

The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.

Because UpdateInferenceRouter replaces all routing targets, include this setting in every update of an asynchronous routing target.

For information about asynchronous routing targets, see Send asynchronous requests through an inference router.



52443
52444
52445
52446
52447
52448
52449
52450
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443

class RoutingTarget < Struct.new(
  :name,
  :destinations,
  :async_inference_config,
  :routing_preference)
  SENSITIVE = []
  include Aws::Structure
end

#destinations ⇒ Array<Types::Destination>

The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

Returns:



52443
52444
52445
52446
52447
52448
52449
52450
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443

class RoutingTarget < Struct.new(
  :name,
  :destinations,
  :async_inference_config,
  :routing_preference)
  SENSITIVE = []
  include Aws::Structure
end

#name ⇒ String

The name of the routing target. Callers specify this name in the RoutingTargetName parameter when they invoke the inference router. The name must be unique within the router.

Returns:

  • (String)


52443
52444
52445
52446
52447
52448
52449
52450
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443

class RoutingTarget < Struct.new(
  :name,
  :destinations,
  :async_inference_config,
  :routing_preference)
  SENSITIVE = []
  include Aws::Structure
end

#routing_preference ⇒ String

How the inference router chooses among the routing target's destinations. The default is LocalRegionFirst.

LocalRegionFirst

The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

InstancePoolOrder

The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.

Returns:

  • (String)


52443
52444
52445
52446
52447
52448
52449
52450
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443

class RoutingTarget < Struct.new(
  :name,
  :destinations,
  :async_inference_config,
  :routing_preference)
  SENSITIVE = []
  include Aws::Structure
end