PromptModelInferenceConfiguration
Contains inference configurations related to model inference for a prompt. For more information, see Inference parameters.
Contents
- maxTokens
-
The maximum number of tokens to return in the response.
Type: Integer
Valid Range: Minimum value of 0. Maximum value of 4096.
Required: No
- stopSequences
-
A list of strings that define sequences after which the model will stop generating.
Type: Array of strings
Array Members: Minimum number of 0 items. Maximum number of 4 items.
Required: No
- temperature
-
Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.
Type: Float
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
- topK
-
The number of most-likely candidates that the model considers for the next token during generation.
Type: Integer
Valid Range: Minimum value of 0. Maximum value of 500.
Required: No
- topP
-
The percentage of most-likely candidates that the model considers for the next token.
Type: Float
Valid Range: Minimum value of 0. Maximum value of 1.
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
