Interface RoutingTargetAsyncInferenceClientConfig.Builder

  • Method Details

    • maxTimeInBacklogSeconds

      RoutingTargetAsyncInferenceClientConfig.Builder maxTimeInBacklogSeconds(Integer maxTimeInBacklogSeconds)

      The maximum time, in seconds, that a request can wait at a destination before the inference router moves it to another destination. The default is 900 seconds.

      Parameters:
      maxTimeInBacklogSeconds - The maximum time, in seconds, that a request can wait at a destination before the inference router moves it to another destination. The default is 900 seconds.
      Returns:
      Returns a reference to this object so that method calls can be chained together.