EvaluationDatasetMetricConfig
Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.
Contents
- dataset
-
Specifies the prompt dataset.
Type: EvaluationDataset object
Required: Yes
- metricNames
-
The names of the metrics used. For automated model evaluation jobs valid values are
"Builtin.Accuracy","Builtin.Robustness", and"Builtin.Toxicity". In human-based model evaluation jobs the array of strings must match thenameparameter specified inHumanEvaluationCustomMetric.Type: Array of strings
Array Members: Minimum number of 1 item. Maximum number of 10 items.
Length Constraints: Minimum length of 1. Maximum length of 63.
Pattern:
^[0-9a-zA-Z-_.]+$Required: Yes
- taskType
-
The task type you want the model to carry out.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 63.
Pattern:
^[A-Za-z0-9]+$Valid Values:
Summarization | Classification | QuestionAndAnswer | Generation | CustomRequired: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
