AsyncInferenceClientConfig

Configures the behavior of the client used by Amazon SageMaker to interact with the model container during asynchronous inference.

Contents

MaxConcurrentInvocationsPerInstance

The maximum number of concurrent requests sent by the SageMaker client to the model container. If no value is provided, Amazon SageMaker will choose an optimal value for you.

Type: Integer

Valid Range: Minimum value of 1. Maximum value of 1000.

Required: No

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following:

• AWS SDK for C++

• AWS SDK for Go

• AWS SDK for Java V2

• AWS SDK for Ruby V3