Query configurations

You can modify configurations when you query the knowledge base to customize retrieval and response generation. To learn more about a configuration and how to modify it in the console or the API, select from the following topics.

The search type defines how data sources in the knowledge base are queried. The following search types are possible:

Default – Amazon Bedrock decides the search strategy for you.
Hybrid – Combines searching vector embeddings (semantic search) with searching through the raw text. Hybrid search is currently only supported for Amazon OpenSearch Serverless vector stores that contain a filterable text field. If you use a different vector store or your Amazon OpenSearch Serverless vector store doesn't contain a filterable text field, the query uses semantic search.
Semantic – Only searches vector embeddings.

To learn how to define the search type, select the tab corresponding to your method of choice and follow the steps.

Console

Follow the console steps at Query the knowledge base and return results or generate responses. When you open the Configurations pane, you'll see the following options for Search type:

Default – Amazon Bedrock decides which search strategy is best-suited for your vector store configuration.
Hybrid – Amazon Bedrock queries the knowledge base using both the vector embeddings and the raw text. This option is only available if you're using an Amazon OpenSearch Serverless vector store configured with a filterable text field.
Semantic – Amazon Bedrock queries the knowledge base using its vector embeddings.

API

When you make a Retrieve or RetrieveAndGenerate request, include a retrievalConfiguration field, mapped to a KnowledgeBaseRetrievalConfiguration object. To see the location of this field, refer to the Retrieve and RetrieveAndGenerate request bodies in the API reference.

The following JSON object shows the minimal fields required in the KnowledgeBaseRetrievalConfiguration object to set search type configurations:


"retrievalConfiguration": {
    "vectorSearchConfiguration": {
        "overrideSearchType": "HYBRID | SEMANTIC"
    }
}

Specify the search type in the overrideSearchType field. You have the following options:

If you don't specify a value, Amazon Bedrock decides which search strategy is best-suited for your vector store configuration.
HYBRID – Amazon Bedrock queries the knowledge base using both the vector embeddings and the raw text. This option is only available if you're using an Amazon OpenSearch Serverless vector store configured with a filterable text field.
SEMANTIC – Amazon Bedrock queries the knowledge base using its vector embeddings.

When generating responses based off retrieval of information, you can use inference parameters to gain more control over the model’s behavior during inference and influence the model’s outputs. To learn how to modify the inference parameters, select the tab corresponding to your method of choice and follow the steps.

Console

To modify inference parameters when querying a knowledge base – Follow the console steps at Query the knowledge base and return results or generate responses. When you open the Configurations pane, you'll see an Inference parameters section. Modify the parameters as necessary.

To modify inference parameters when chatting with your document – Follow the steps at Chat with your document data using the knowledge base. In the Configurations pane, expand the Inference parameters section and modify the parameters as necessary.

API

You provide the model parameters in the call to the RetrieveAndGenerate API. You can customize the model by providing inference parameters in the inferenceConfig field of either the knowledgeBaseConfiguration (if you query a knowledge base) or the externalSourcesConfiguration (if you chat with your document).

Within the inferenceConfig field is a textInferenceConfig field that contains the following parameters that you can:

temperature
topP
maxTokenCount
stopSequences

You can customize the model by using the following parameters in the inferenceConfig field of both externalSourcesConfiguration and knowledgeBaseConfiguration:

temperature
topP
maxTokenCount
stopSequences

For a detailed explanation of the function of each of these parameters, see Inference parameters.

Additionally, you can provide custom parameters not supported by textInferenceConfig via the additionalModelRequestFields map. You can provide parameters unique to specific models with this argument, for the unique paramaters see Inference parameters for foundation models.

If a parameter is omitted from textInferenceConfig, a default value will be used. Any parameters not recognized in textInferneceConfig will be ignored, while any parameters not recognized in AdditionalModelRequestFields will cause an exception.

A validation exception is thrown if there is the same parameter in both additionalModelRequestFields and TextInferenceConfig.

Using model parameters in RetrieveAndGenerate

The following is an example of the structure for inferenceConfig and additionalModelRequestFields under the generationConfiguration in the RetrieveAndGenerate request body:


"inferenceConfig": {
    "textInferenceConfig": {
        "temperature": 0.5,  
        "topP": 0.5,
        "maxTokens": 2048,
        "stopSequences": ["\nObservation"]
    }
},
"additionalModelRequestFields": {
    "top_k": 50
}

The proceeding example sets a temperature of 0.5, top_p of 0.5, maxTokens of 2048, stops generation if it encounters the string "\nObservation" in the generated response, and passes a custom top_k value of 50.

When you query a knowledge base, Amazon Bedrock returns up to five results in the response by default. Each result corresponds to a source chunk. To modify the maximum number of results to return, select the tab corresponding to your method of choice and follow the steps.

Your data sources can include metadata files associated with the source documents. A metadata file contains attributes as key-value pairs that you define for a source document. For more information about creating metadata for your data source files, see Add metadata to your files to allow for filtering. To use filters during knowledge base query, check that your knowledge base fulfills the following requirements:

The Amazon S3 bucket containing your data source includes at least one .metadata.json file with the same name as the source document it's associated with.
If your knowledge base's vector index is in an Amazon OpenSearch Serverless vector store, check that the vector index is configured with the faiss engine. If the vector index is configured with the nmslib engine, you'll have to do one of the following:
- Create a new knowledge base in the console and let Amazon Bedrock automatically create a vector index in Amazon OpenSearch Serverless for you.
- Create another vector index in the vector store and select faiss as the Engine. Then Create a new knowledge base and specify the new vector index.

You can use the following filtering operators when modifying query configurations for filtering:

Filtering operators
Operator	Console	API filter name	Supported attribute data types	Filtered results
Equals	=	equals	string, number, boolean	Attribute matches the value you provide
Not equals	!=	notEquals	string, number, boolean	Attribute doesn't match the value you provide
Greater than	>	greaterThan	number	Attribute is greater than the value you provide
Greater than or equals	>=	greaterThanOrEquals	number	Attribute is greater than or equal to the value you provide
Less than	<	lessThan	number	Attribute is less than the value you provide
Less than or equals	<=	lessThanOrEquals	number	Attribute is less than or equal to the value you provide
In	:	in	string list	Attribute is in the list you provide
Not in	!:	notIn	string list	Attribute isn't in the list you provide
Starts with	^	startsWith	string	Attribute starts with the string you provide (only supported for Amazon OpenSearch Serverless vector stores)

To combine filtering operators, you can use the following logical operators:

Logical operators
Operator	Console	API filter field name	Filtered results
And	and	andAll	Results fulfill all of the filtering expressions in the group
Or	or	orAll	Results fulfill at least one of the filtering expressions in the group

To learn how to filter results using metadata, select the tab corresponding to your method of choice and follow the steps.

Console

Follow the console steps at Query the knowledge base and return results or generate responses. When you open the Configurations pane, you'll see a Filters section. The following procedures describe different use cases:

To add a filter, create a filtering expression by entering a metadata attribute, filtering operator, and value in the box. Separate each part of the expression with a whitespace. Press Enter to add the filter.

For a list of accepted filtering operators, see the Filtering operators table above. You can also see a list of filtering operators when you add a whitespace after the metadata attribute.

Note
You must surround strings with quotation marks.

For example, you can filter for results from source documents that contain a genre metadata attribute whose value is "entertainment" by adding the following filter: genre = "entertainment".
To add another filter, enter another filtering expression in the box and press Enter. You can add up to 5 filters in the group.
By default, the query will return results that fulfill all the filtering expressions you provide. To return results that fulfill at least one of the filtering expressions, choose the and dropdown menu between any two filtering operations and select or.
To combine different logical operators, select + Add Group to add a filter group. Enter filtering expressions in the new group. You can add up to 5 filter groups.
To change the logical operator used between all the filtering groups, choose the AND dropdown menu between any two filter groups and select OR.
To edit a filter, select it, modify the filtering operation, and choose Apply.
To remove a filter group, choose the trash can icon ( ) next to the group. To remove a filter, choose the delete icon ( ) next to the filter.

The following image shows an example filter configuration that returns all documents written after 2018 whose genre is "entertainment", in addition to documents whose genre is "cooking" or "sports" and whose author starts with "C".

API

The following JSON objects show the minimal fields required in the KnowledgeBaseRetrievalConfiguration object to set filters for different use cases:

Use one filtering operator (see the Filtering operators table above).


"retrievalConfiguration": {
    "vectorSearchConfiguration": {
        "filter": {
            "<filter-type>": {
                "key": "string",
                "value": "string" | number | boolean | ["string", "string", ...]
            }
        }
    }
}

Use a logical operator (see the Logical operators table above) to combine up to 5.


"retrievalConfiguration": {
    "vectorSearchConfiguration": {
        "filter": {
            "andAll | orAll": [
                "<filter-type>": {
                    "key": "string",
                    "value": "string" | number | boolean | ["string", "string", ...]
                },
                "<filter-type>": {
                    "key": "string",
                    "value": "string" | number | boolean | ["string", "string", ...]
                },
                ...
            ]
        }
    }
}

Use a logical operator to combine up to 5 filtering operators into a filter group, and a second logical operator to combine that filter group with another filtering operator.


"retrievalConfiguration": {
    "vectorSearchConfiguration": {
        "filter": {
            "andAll | orAll": [
                "andAll | orAll": [
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    ...
                ],
                "<filter-type>": {
                    "key": "string",
                    "value": "string" | number | boolean | ["string", "string", ...]
                }
            ]
        }
    }
}

Combine up to 5 filter groups by embedding them within another logical operator. You can create one level of embedding.


"retrievalConfiguration": {
    "vectorSearchConfiguration": {
        "filter": {
            "andAll | orAll": [
                "andAll | orAll": [
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    ...
                ],
                "andAll | orAll": [
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    "<filter-type>": {
                        "key": "string",
                        "value": "string" | number | boolean | ["string", "string", ...]
                    },
                    ...
                ]
            ]
        }
    }
}

The following table describes the filter types that you can use:

Field	Supported value data types	Filtered results
`equals`	string, number, boolean	Attribute matches the value you provide
`notEquals`	string, number, boolean	Attribute doesn't match the value you provide
`greaterThan`	number	Attribute is greater than the value you provide
`greaterThanOrEquals`	number	Attribute is greater than or equal to the value you provide
`lessThan`	number	Attribute is less than the value you provide
`lessThanOrEquals`	number	Attribute is less than or equal to the value you provide
`in`	list of strings	Attribute is in the list you provide
`notIn`	list of strings	Attribute isn't in the list you provide
`startsWith`	string	Attribute starts with the string you provide (only supported for Amazon OpenSearch Serverless vector stores)

To combine filter types, you can use one of the following logical operators:

Field	Maps to	Filtered results
`andAll`	List of up to 5 filter types	Results fulfill all of the filtering expressions in the group
`orAll`	List of up to 5 filter types	Results fulfill at least one of the filtering expressions in the group

For examples, see Send a query and include filters (Retrieve) and Send a query and include filters (RetrieveAndGenerate).

When you query a knowledge base and request response generation, Amazon Bedrock uses a prompt template that combines instructions and context with the user query to construct the prompt that's sent to the model for response generation. You can engineer the prompt template with the following tools:

Prompt placeholders – Pre-defined variables in Knowledge bases for Amazon Bedrock that are dynamically filled in at runtime during knowledge base query. In the system prompt, you'll see these placeholders surrounded by the $ symbol. The following list describes the placeholders you can use:

Variable	Replaced by	Model	Required?
$query$	The user query sent to the knowledge base.	Anthropic Claude Instant, Anthropic Claude v2.x	Yes
$query$	The user query sent to the knowledge base.	Anthropic Claude 3 Sonnet	No (automatically included in model input)
$search_results$	The retrieved results for the user query.	All	Yes
$output_format_instructions$	Underlying instructions for formatting the response generation and citations. Differs by model. If you define your own formatting instructions, we suggest that you remove this placeholder. Without this placeholder, the response won't contain citations.	All	No
$current_time$	The current time.	All	No

XML tags – Anthropic models support the use of XML tags to structure and delineate your prompts. Use descriptive tag names for optimal results. For example, in the default system prompt, you'll see the <database> tag used to delineate a database of previously asked questions). For more information, see Use XML tags in the Anthropic user guide.

For general prompt engineering guidelines, see Prompt engineering guidelines.

Select the tab corresponding to your method of choice and follow the steps.

You can implement safeguards for your knowledge base for your use cases and responsible AI policies. You can create multiple guardrails tailored to different use cases and apply them across multiple request and response conditions, providing a consistent user experience and standardizing safety controls across your knowledge base. You can configure denied topics to disallow undesirable topics and content filters to block harmful content in model inputs and responses. For more information, see Guardrails for Amazon Bedrock.

For general prompt engineering guidelines, see Prompt engineering guidelines.

Select the tab corresponding to your method of choice and follow the steps.

Console

Follow the console steps at Query the knowledge base and return results or generate responses. In the test window, turn on Generate responses. Then, in the Configurations pane, expand the Guardrails section.

In the Guardrails section, choose the Name and the Version of your guardrail. If you would like to see the details for your chosen guardrail and version, choose View.

Alternatively, you can create a new one by choosing the Guardrail link.
When you're finished editing, choose Save changes. To exit without saving choose Discard changes.

API

When you make a RetrieveAndGenerate request, include the guardrailsConfiguration field within the generationConfiguration to use your guardrail with the request. To see the location of this field, refer to the RetrieveAndGenerate request body in the API reference.

The following JSON object shows the minimal fields required in the GenerationConfiguration to set the guardrailsConfiguration:


"generationConfiguration": {
    "guardrailConfiguration": {
        "guardrailIdentifier": "string",
        "guardrailVersion": "string"
    }
}

Specify the ARN or ID of the guardrail to use with your knowledge base and the version to use.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Query the knowledge base

Manage a data source

Query configurations

Note