Troubleshooting tool calls

You might see different errors when working with tools and Amazon Nova models. Here are a few examples and tips to help you troubleshoot.

Model Error Exception

An error occurred (ModelErrorException) when calling the Converse operation: The system encountered an unexpected error during processing. Try your request again.

Validate your inference parameters - Amazon Nova models have more success calling tools when using greedy decoding. To enable greedy decoding, set the top P, top K, and temperature parameters to 1.

inferenceConfig={
    "topP": 1, 
    "temperature": 1
}

additionalModelRequestFields={"inferenceConfig": {"topK": 1}}

Increase the maximum token count - It is common for tool outputs to require a large token output, ensure that the max tokens set is large enough to accommodate the expected return schema. If the model response is larger than your max token count, it will trigger an exception. You can set the maximum tokens in the inferenceConfig parameter:

inferenceConfig={
    "maxTokens": 3000
}

Review the system prompt - To improve the accuracy of tool calling, Amazon Nova uses chain of thought reasoning when calling a tool. You will see this output in the response in <thinking> tags. We do not recommended trying to remove this functionality. Instead, we recommend that you drop the output if you do not need it in your application.

Model Error Exception

An error occurred (ModelErrorException) when calling the Converse operation: The system encountered an unexpected error during processing. Try your request again.

Validate your inference parameters - Amazon Nova models have more success calling tools when using greedy decoding. To enable greedy decoding, set the top P, top K, and temperature parameters to 1.

inferenceConfig={
    "topP": 1, 
    "temperature": 1
}

additionalModelRequestFields={"inferenceConfig": {"topK": 1}}

Increase the maximum token count - It is common for tool outputs to require a large token output, ensure that the max tokens set is large enough to accommodate the expected return schema. If the model response is larger than your max token count, it will trigger an exception. You can set the maximum tokens in the inferenceConfig parameter:

inferenceConfig={
    "maxTokens": 3000
}

Review the system prompt - To improve the accuracy of tool calling, Amazon Nova uses chain of thought reasoning when calling a tool. You will see this output in the response in <thinking> tags. We do not recommended trying to remove this functionality. Instead, we recommend that you drop the output if you do not need it in your application.

Validation Exception

An error occurred (ValidationException) when calling the Converse operation

The passed tool configuration does not comply to the required conventions. Refer to the Converse API spec for specifications for all parameters.

Validation Exception

An error occurred (ValidationException) when calling the Converse operation

The passed tool configuration does not comply to the required conventions. Refer to the Converse API spec for specifications for all parameters.

A tool is not returned when expected

If it looks like tools are being ignored by the model, make sure you're following the correct JSON schema for the tool config:

• Top level schema must of of type Object

• Only three fields are supported in top-level Object - type (must be set to 'object'), properties, and required

• Common unsupported fields at the top level are: $schema, description, title, and additionalProperties

A tool is not returned when expected

If it looks like tools are being ignored by the model, make sure you're following the correct JSON schema for the tool config:

• Top level schema must of of type Object

• Only three fields are supported in top-level Object - type (must be set to 'object'), properties, and required

• Common unsupported fields at the top level are: $schema, description, title, and additionalProperties

A necessary tool isn't available

You might receive a successful response from Amazon Nova, but the model does not acknowledge that the needed tool is not present.

Validate your inference parameters - Amazon Nova models have more success calling tools when using greedy decoding. To enable greedy decoding, set the top P, top K, and temperature parameters to 1.

inferenceConfig={
    "topP": 1, 
    "temperature": 1
}

additionalModelRequestFields={"inferenceConfig": {"topK": 1}}

Review the system prompt - To improve the accuracy of tool calling, Amazon Nova uses chain of thought reasoning when calling a tool. You will see this output in the response in <thinking> tags. We do not recommended trying to remove this functionality. Instead, we recommend that you drop the output if you do not need it in your application.

A necessary tool isn't available

You might receive a successful response from Amazon Nova, but the model does not acknowledge that the needed tool is not present.

Validate your inference parameters - Amazon Nova models have more success calling tools when using greedy decoding. To enable greedy decoding, set the top P, top K, and temperature parameters to 1.

inferenceConfig={
    "topP": 1, 
    "temperature": 1
}

additionalModelRequestFields={"inferenceConfig": {"topK": 1}}

Review the system prompt - To improve the accuracy of tool calling, Amazon Nova uses chain of thought reasoning when calling a tool. You will see this output in the response in <thinking> tags. We do not recommended trying to remove this functionality. Instead, we recommend that you drop the output if you do not need it in your application.