Interface RoutingTarget.Builder

  • Method Details

    • name

      The name of the routing target. Callers specify this name in the RoutingTargetName parameter when they invoke the inference router. The name must be unique within the router.

      Parameters:
      name - The name of the routing target. Callers specify this name in the RoutingTargetName parameter when they invoke the inference router. The name must be unique within the router.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • destinations

      RoutingTarget.Builder destinations(Collection<Destination> destinations)

      The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

      In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

      Parameters:
      destinations - The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

      In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • destinations

      RoutingTarget.Builder destinations(Destination... destinations)

      The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

      In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

      Parameters:
      destinations - The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

      In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • destinations

      RoutingTarget.Builder destinations(Consumer<Destination.Builder>... destinations)

      The endpoints and inference components that serve the routing target. Each destination can appear only once in a routing target.

      In an asynchronous routing target, every destination must be an asynchronous inference endpoint. In a real-time routing target, no destination can be one. SageMaker AI checks an endpoint in another account with your credentials, so both your IAM policy and the endpoint policy must allow sagemaker:DescribeEndpoint. If an endpoint can't be checked, for example because it isn't found or can't be read in time, it's accepted, and requests to it fail if it's the wrong mode.

      This is a convenience method that creates an instance of the Destination.Builder avoiding the need to create one manually via Destination.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to destinations(List<Destination>).

      Parameters:
      destinations - a consumer that will call methods on Destination.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:
    • asyncInferenceConfig

      RoutingTarget.Builder asyncInferenceConfig(RoutingTargetAsyncInferenceConfig asyncInferenceConfig)

      The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

      Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.

      Because UpdateInferenceRouter replaces all routing targets, include this setting in every update of an asynchronous routing target.

      For information about asynchronous routing targets, see Send asynchronous requests through an inference router.

      Parameters:
      asyncInferenceConfig - The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

      Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.

      Because UpdateInferenceRouter replaces all routing targets, include this setting in every update of an asynchronous routing target.

      For information about asynchronous routing targets, see Send asynchronous requests through an inference router.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • asyncInferenceConfig

      default RoutingTarget.Builder asyncInferenceConfig(Consumer<RoutingTargetAsyncInferenceConfig.Builder> asyncInferenceConfig)

      The asynchronous inference settings of the routing target. Specify this setting to make the routing target asynchronous. If you omit it, the routing target serves real-time requests.

      Asynchronous routing targets aren't available in all Amazon Web Services Regions. For information about asynchronous routing targets and Region availability, see Send asynchronous requests through an inference router.

      Because UpdateInferenceRouter replaces all routing targets, include this setting in every update of an asynchronous routing target.

      For information about asynchronous routing targets, see Send asynchronous requests through an inference router.

      This is a convenience method that creates an instance of the RoutingTargetAsyncInferenceConfig.Builder avoiding the need to create one manually via RoutingTargetAsyncInferenceConfig.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to asyncInferenceConfig(RoutingTargetAsyncInferenceConfig).

      Parameters:
      asyncInferenceConfig - a consumer that will call methods on RoutingTargetAsyncInferenceConfig.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:
    • routingPreference

      RoutingTarget.Builder routingPreference(String routingPreference)

      How the inference router chooses among the routing target's destinations. The default is LocalRegionFirst.

      LocalRegionFirst

      The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

      InstancePoolOrder

      The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

      With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.

      Parameters:
      routingPreference - How the inference router chooses among the routing target's destinations. The default is LocalRegionFirst.

      LocalRegionFirst

      The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

      InstancePoolOrder

      The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

      With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:
    • routingPreference

      RoutingTarget.Builder routingPreference(InferenceRouterRoutingPreference routingPreference)

      How the inference router chooses among the routing target's destinations. The default is LocalRegionFirst.

      LocalRegionFirst

      The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

      InstancePoolOrder

      The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

      With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.

      Parameters:
      routingPreference - How the inference router chooses among the routing target's destinations. The default is LocalRegionFirst.

      LocalRegionFirst

      The router uses destinations in its own Region first. When the request leaves the Region, the router spreads it across healthy destinations in other Regions.

      InstancePoolOrder

      The router prefers destinations by instance type first and by Region second. Every destination must be an endpoint that lists the same two or more instance types, each with a different priority, in the same priority order. This value is supported only for asynchronous routing targets.

      With either value, the router uses destinations whose health or capacity is reduced after the others, but doesn't exclude them.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also: