Class: Aws::SageMaker::Types::RoutingTarget
- Inherits:
-
Struct
- Object
- Struct
- Aws::SageMaker::Types::RoutingTarget
- Defined in:
- gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb
Overview
A model that an inference router serves, and the destinations that can serve it. When you invoke the router, you name the routing target, and the router sends the request to one of its destinations.
Every destination in a routing target must serve the same model, with the same request and response format.
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#async_inference_config ⇒ Types::RoutingTargetAsyncInferenceConfig
The asynchronous inference settings of the routing target.
-
#destinations ⇒ Array<Types::Destination>
The endpoints and inference components that serve the routing target.
-
#name ⇒ String
The name of the routing target.
-
#routing_preference ⇒ String
How the inference router chooses among the routing target's destinations.
Instance Attribute Details
#async_inference_config ⇒ Types::RoutingTargetAsyncInferenceConfig
The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.
Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.
Because UpdateInferenceRouter replaces all routing targets,
include this setting in every update of an asynchronous routing
target.
For information about asynchronous routing targets, see Send asynchronous requests through an inference router.
52443 52444 52445 52446 52447 52448 52449 52450 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443 class RoutingTarget < Struct.new( :name, :destinations, :async_inference_config, :routing_preference) SENSITIVE = [] include Aws::Structure end |
#destinations ⇒ Array<Types::Destination>
The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.
In an asynchronous routing target, every destination must be an
asynchronous inference endpoint. In a real-time routing target, no
destination can be one. SageMaker AI checks an endpoint in another
account with your credentials, so both your IAM policy and the
endpoint policy must allow sagemaker:DescribeEndpoint. If an
endpoint can't be checked, for example because it isn't found or
can't be read in time, it's accepted, and requests to it fail if
it's the wrong mode.
52443 52444 52445 52446 52447 52448 52449 52450 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443 class RoutingTarget < Struct.new( :name, :destinations, :async_inference_config, :routing_preference) SENSITIVE = [] include Aws::Structure end |
#name ⇒ String
The name of the routing target. Callers specify this name in the
RoutingTargetName parameter when they invoke the inference router.
The name must be unique within the router.
52443 52444 52445 52446 52447 52448 52449 52450 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443 class RoutingTarget < Struct.new( :name, :destinations, :async_inference_config, :routing_preference) SENSITIVE = [] include Aws::Structure end |
#routing_preference ⇒ String
How the inference router chooses among the routing target's
destinations. The default is LocalRegionFirst.
- LocalRegionFirst
The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.
- InstancePoolOrder
The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.
With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.
52443 52444 52445 52446 52447 52448 52449 52450 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 52443 class RoutingTarget < Struct.new( :name, :destinations, :async_inference_config, :routing_preference) SENSITIVE = [] include Aws::Structure end |