Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

Creating a prompt dataset for Retrieve only evaluation jobs

Focus mode
Creating a prompt dataset for Retrieve only evaluation jobs - Amazon Bedrock

A Retrieve only evaluation jobs require a prompt dataset using JSON lines format. You can have up to 1000 prompts in your dataset

Key value pairs used in prompt dataset for Retrieve only evaluation job
  • referenceContexts – This parent key is used to specify the ground truth response you expect the Retrieve would return. Specify the ground truth in the text key. referenceContexts is required if you choose the Context coverage metric in your evaluation job.

  • prompt – This parent key is used to specify the prompt (user query) that you want the model to respond to while the evaluation job is running.

The following prompt is expanded for clarity. In your actual prompt dataset each line (a prompt) must be a valid JSON object.

{ "conversationTurns": [ { "referenceContexts": [ { "content": [ { "text": "This is a reference context" } ] } ], "prompt": { "content": [ { "text": "This is a prompt" } ] } } ] }
PrivacySite termsCookie preferences
© 2025, Amazon Web Services, Inc. or its affiliates. All rights reserved.