Getting the validation results

The validation results contain error information for Terminal manifest content errors and Non terminal JSON line validation errors. There are three validation results files.

• training_manifest_with_validation.json – A copy of the training dataset manifest file with JSON Line error information added.

• testing_manifest_with_validation.json – A copy of the testing dataset manifest file with JSON Line error error information added.

• manifest_summary.json – A summary of manifest content errors and JSON Line errors found in the training and testing datasets. For more information, see Understanding the manifest summary.

For information about the contents of the training and testing validation manifests, see Debugging a failed model training.

Note

• The validation results are created only if no Terminal manifest file errors are generated during training.

• If a service error occurs after the training and testing manifest are validated, the validation results are created, but the response from DescribeProjectVersions doesn't include the validation results file locations.

After training completes or fails, you can download the validation results by using the Amazon Rekognition Custom Labels console or get the Amazon S3 bucket location by calling DescribeProjectVersions API.

Getting validation results (Console)

If you are using the console to train your model, you can download the validation results from a project's list of models, as shown in the following diagram.

You can also access download the validation results from a model's details page.

For more information, see Training a model (Console).

Getting validation results (SDK)

After model training completes, Amazon Rekognition Custom Labels stores the validation results in the Amazon S3 bucket specified during training. You can get the S3 bucket location by calling the DescribeProjectVersions API, after training completes. To train a model, see Training a model (SDK).

A ValidationData object is returned for the training dataset (TrainingDataResult) and the testing dataset (TestingDataResult). The manifest summary is returned in ManifestSummary.

After you get the Amazon S3 bucket location, you can download the validation results. For more information, see How do I download an object from an S3 bucket?. You can also use the GetObject operation.

To get validation data (SDK)

1. If you haven't already done so, install and configure the AWS CLI and the AWS SDKs. For more information, see Step 4: Set up the AWS CLI and AWS SDKs.

2. Use the following example to get the location of the validation results.

   Python

   Replace project_arn with the Amazon Resource Name (ARN) of the project that contains the model. For more information, see Managing an Amazon Rekognition Custom Labels project. Replace version_name with the name of the model version. For more information, see Training a model (SDK).

   import boto3
   import io
   from io import BytesIO
   import sys
   import json
   
   
   def describe_model(project_arn, version_name):
   
       client=boto3.client('rekognition')
       
       response=client.describe_project_versions(ProjectArn=project_arn,
           VersionNames=[version_name])
   
       for model in response['ProjectVersionDescriptions']:
           print(json.dumps(model,indent=4,default=str))
          
   def main():
   
       project_arn='project_arn'
       version_name='version_name'
   
       describe_model(project_arn, version_name)
   
   if __name__ == "__main__":
       main()

3. In the program output, note the Validation field within the TestingDataResult and TrainingDataResult objects. The manifest summary is in ManifestSummary.