RoutingTargetAsyncInferenceClientConfig
How the inference router submits asynchronous requests to the destinations of a routing target.
Contents
- MaxTimeInBacklogSeconds
-
The maximum time, in seconds, that a request can wait at a destination before the inference router moves it to another destination. The default is 900 seconds.
Type: Integer
Valid Range: Minimum value of 60. Maximum value of 21600.
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: