Interface RoutingTarget.Builder
- All Superinterfaces:
Buildable,CopyableBuilder<RoutingTarget.Builder,,RoutingTarget> SdkBuilder<RoutingTarget.Builder,,RoutingTarget> SdkPojo
- Enclosing class:
RoutingTarget
-
Method Summary
Modifier and TypeMethodDescriptiondefault RoutingTarget.BuilderasyncInferenceConfig(Consumer<RoutingTargetAsyncInferenceConfig.Builder> asyncInferenceConfig) The asynchronous inference settings of the routing target.asyncInferenceConfig(RoutingTargetAsyncInferenceConfig asyncInferenceConfig) The asynchronous inference settings of the routing target.destinations(Collection<Destination> destinations) The endpoints and inference components that serve the routing target.destinations(Consumer<Destination.Builder>... destinations) The endpoints and inference components that serve the routing target.destinations(Destination... destinations) The endpoints and inference components that serve the routing target.The name of the routing target.routingPreference(String routingPreference) How the inference router chooses among the routing target's destinations.routingPreference(InferenceRouterRoutingPreference routingPreference) How the inference router chooses among the routing target's destinations.Methods inherited from interface software.amazon.awssdk.utils.builder.CopyableBuilder
copyMethods inherited from interface software.amazon.awssdk.utils.builder.SdkBuilder
applyMutation, buildMethods inherited from interface software.amazon.awssdk.core.SdkPojo
equalsBySdkFields, sdkFieldNameToField, sdkFields
-
Method Details
-
name
The name of the routing target. Callers specify this name in the
RoutingTargetNameparameter when they invoke the inference router. The name must be unique within the router.- Parameters:
name- The name of the routing target. Callers specify this name in theRoutingTargetNameparameter when they invoke the inference router. The name must be unique within the router.- Returns:
- Returns a reference to this object so that method calls can be chained together.
-
destinations
The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.
In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow
sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.- Parameters:
destinations- The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow
sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.- Returns:
- Returns a reference to this object so that method calls can be chained together.
-
destinations
The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.
In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow
sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.- Parameters:
destinations- The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow
sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.- Returns:
- Returns a reference to this object so that method calls can be chained together.
-
destinations
The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.
In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow
This is a convenience method that creates an instance of thesagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.Destination.Builderavoiding the need to create one manually viaDestination.builder().When the
Consumercompletes,SdkBuilder.build()is called immediately and its result is passed todestinations(List<Destination>).- Parameters:
destinations- a consumer that will call methods onDestination.Builder- Returns:
- Returns a reference to this object so that method calls can be chained together.
- See Also:
-
asyncInferenceConfig
The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.
Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.
Because
UpdateInferenceRouterreplaces all routing targets, include this setting in every update of an asynchronous routing target.For information about asynchronous routing targets, see Send asynchronous requests through an inference router.
- Parameters:
asyncInferenceConfig- The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.
Because
UpdateInferenceRouterreplaces all routing targets, include this setting in every update of an asynchronous routing target.For information about asynchronous routing targets, see Send asynchronous requests through an inference router.
- Returns:
- Returns a reference to this object so that method calls can be chained together.
-
asyncInferenceConfig
default RoutingTarget.Builder asyncInferenceConfig(Consumer<RoutingTargetAsyncInferenceConfig.Builder> asyncInferenceConfig) The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.
Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.
Because
UpdateInferenceRouterreplaces all routing targets, include this setting in every update of an asynchronous routing target.For information about asynchronous routing targets, see Send asynchronous requests through an inference router.
This is a convenience method that creates an instance of theRoutingTargetAsyncInferenceConfig.Builderavoiding the need to create one manually viaRoutingTargetAsyncInferenceConfig.builder().When the
Consumercompletes,SdkBuilder.build()is called immediately and its result is passed toasyncInferenceConfig(RoutingTargetAsyncInferenceConfig).- Parameters:
asyncInferenceConfig- a consumer that will call methods onRoutingTargetAsyncInferenceConfig.Builder- Returns:
- Returns a reference to this object so that method calls can be chained together.
- See Also:
-
routingPreference
How the inference router chooses among the routing target's destinations. The default is
LocalRegionFirst.- LocalRegionFirst
-
The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.
- InstancePoolOrder
-
The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.
With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.
- Parameters:
routingPreference- How the inference router chooses among the routing target's destinations. The default isLocalRegionFirst.- LocalRegionFirst
-
The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.
- InstancePoolOrder
-
The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.
With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.
- Returns:
- Returns a reference to this object so that method calls can be chained together.
- See Also:
-
routingPreference
How the inference router chooses among the routing target's destinations. The default is
LocalRegionFirst.- LocalRegionFirst
-
The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.
- InstancePoolOrder
-
The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.
With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.
- Parameters:
routingPreference- How the inference router chooses among the routing target's destinations. The default isLocalRegionFirst.- LocalRegionFirst
-
The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.
- InstancePoolOrder
-
The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.
With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.
- Returns:
- Returns a reference to this object so that method calls can be chained together.
- See Also:
-