EvaluationDatasetMetricConfig
Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.
Contents
- dataset
-
Specifies the prompt dataset.
Type: EvaluationDataset object
Required: Yes
- metricNames
-
The names of the metrics used. For automated model evaluation jobs valid values are
"Builtin.Accuracy"
,"Builtin.Robustness"
, and"Builtin.Toxicity"
. In human-based model evaluation jobs the array of strings must match thename
parameter specified inHumanEvaluationCustomMetric
.Type: Array of strings
Array Members: Minimum number of 1 item. Maximum number of 10 items.
Length Constraints: Minimum length of 1. Maximum length of 63.
Pattern:
^[0-9a-zA-Z-_.]+$
Required: Yes
- taskType
-
The task type you want the model to carry out.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 63.
Pattern:
^[A-Za-z0-9]+$
Valid Values:
Summarization | Classification | QuestionAndAnswer | Generation | Custom
Required: Yes
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following: