ModelSpeculativeDecodingConfig
Settings for the model speculative decoding technique that's applied by a model optimization job.
Contents
- Technique
-
The speculative decoding technique to apply during model optimization.
Type: String
Valid Values:
EAGLERequired: Yes
- TrainingDataSource
-
The location of the training data to use for speculative decoding. The data must be formatted as ShareGPT, OpenAI Completions or OpenAI Chat Completions. The input can also be unencrypted captured data from a SageMaker endpoint as long as the endpoint uses one of the above formats.
Type: ModelSpeculativeDecodingTrainingDataSource object
Required: No
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
