Client-side tool use
If you use the Responses API, Chat Completions API, Converse API, or InvokeModel API to send the request, then the model uses client-side tool calling. This means that in your code, you call the tool on the model's behalf. In this scenario, assume the tool implementation is an API. The tool could just as easily be a database, Lambda function, or some other software. You decide how you want to implement the tool. You then continue the conversation with the model by supplying a message with the result from the tool. Finally, the model generates a response for the original message that includes the tool results that you sent to the model.
Let us define the tool we will use for tool use. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.
def get_most_popular_song(station_name: str) -> str: stations = { "Radio Free Mars": "Starman – David Bowie", "Neo Tokyo FM": "Plastic Love – Mariya Takeuchi", "Cloud Nine Radio": "Blinding Lights – The Weeknd", } return stations.get(station_name, "Unknown Station – No chart data available")
Using Responses API for client-side tooling
You can use the Function calling
from openai import OpenAI import json client = OpenAI() response = client.responses.create( model="oss-gpt-120b", input="What is the most popular song on Radio Free Mars?", tools=[ { "type": "function", "name": "get_most_popular_song", "description": "Returns the most popular song on a radio station", "parameters": { "type": "object", "properties": { "station_name": { "type": "string", "description": "Name of the radio station" } }, "required": ["station_name"] } } ] ) if response.output and response.output[0].content: tool_call = response.output[0].content[0] args = json.loads(tool_call["arguments"]) result = get_most_popular_song(args["station_name"]) final_response = client.responses.create( model="oss-gpt-120b", input=[ { "role": "tool", "tool_call_id": tool_call["id"], "content": result } ] ) print(final_response.output_text)
Using Chat Completions API for client-side tooling
You can use the Chat Completions API also. Here is the Python code for using Chat Completions:
from openai import OpenAI import json client = OpenAI() completion = client.chat.completions.create( model="oss-gpt-120b", messages=[{"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}], tools=[{ "type": "function", "function": { "name": "get_most_popular_song", "description": "Returns the most popular song on a radio station", "parameters": { "type": "object", "properties": { "station_name": {"type": "string", "description": "Name of the radio station"} }, "required": ["station_name"] } } }] ) message = completion.choices[0].message if message.tool_calls: tool_call = message.tool_calls[0] args = json.loads(tool_call.function.arguments) result = get_most_popular_song(args["station_name"]) followup = client.chat.completions.create( model="oss-gpt-120b", messages=[ {"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}, message, {"role": "tool", "tool_call_id": tool_call.id, "content": result} ] ) print(followup.choices[0].message.content)
For more details on the using Function Calling on Responses API and Chat Completions API, see Function Calling
Using Converse API for client-side tooling
You can use the Converse API to let a model use a tool in a conversation. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """Shows how to use tools with the Converse API and the Cohere Command R model.""" import logging import json import boto3 from botocore.exceptions import ClientError class StationNotFoundError(Exception): """Raised when a radio station isn't found.""" pass logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def get_top_song(call_sign): """Returns the most popular song for the requested station. Args: call_sign (str): The call sign for the station for which you want the most popular song. Returns: response (json): The most popular song and artist. """ song = "" artist = "" if call_sign == 'WZPZ': song = "Elemental Hotel" artist = "8 Storey Hike" else: raise StationNotFoundError(f"Station {call_sign} not found.") return song, artist def generate_text(bedrock_client, model_id, tool_config, input_text): """Generates text using the supplied Amazon Bedrock model. If necessary, the function handles tool use requests and sends the result to the model. Args: bedrock_client: The Boto3 Bedrock runtime client. model_id (str): The Amazon Bedrock model ID. tool_config (dict): The tool configuration. input_text (str): The input text. Returns: Nothing. """ logger.info("Generating text with model %s", model_id) # Create the initial message from the user input. messages = [{"role": "user", "content": [{"text": input_text}]}] response = bedrock_client.converse(modelId=model_id, messages=messages, toolConfig=tool_config) output_message = response['output']['message'] messages.append(output_message) stop_reason = response['stopReason'] if stop_reason == 'tool_use': # Tool use requested. Call the tool and send the result to the model. tool_requests = response['output']['message']['content'] for tool_request in tool_requests: if 'toolUse' in tool_request: tool = tool_request['toolUse'] logger.info("Requesting tool %s. Request: %s", tool['name'], tool['toolUseId']) if tool['name'] == 'top_song': tool_result = {} try: song, artist = get_top_song(tool['input']['sign']) tool_result = {"toolUseId": tool['toolUseId'], "content": [{"json": {"song": song, "artist": artist}}]} except StationNotFoundError as err: tool_result = {"toolUseId": tool['toolUseId'], "content": [{"text": err.args[0]}], "status": 'error'} tool_result_message = {"role": "user", "content": [{"toolResult": tool_result}]} messages.append(tool_result_message) # Send the tool result to the model. response = bedrock_client.converse(modelId=model_id, messages=messages, toolConfig=tool_config) output_message = response['output']['message'] # print the final response from the model. for content in output_message['content']: print(json.dumps(content, indent=4)) def main(): """Entrypoint for tool use example.""" logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = "cohere.command-r-v1:0" input_text = "What is the most popular song on WZPZ?" tool_config = { "tools": [ { "toolSpec": { "name": "top_song", "description": "Get the most popular song played on a radio station.", "inputSchema": { "json": { "type": "object", "properties": { "sign": { "type": "string", "description": "The call sign for the radio station for which you want the most popular song. Example calls signs are WZPZ, and WKRP." } }, "required": ["sign"] } } } } ] } bedrock_client = boto3.client(service_name='bedrock-runtime') try: print(f"Question: {input_text}") generate_text(bedrock_client, model_id, tool_config, input_text) except ClientError as err: message = err.response['Error']['Message'] logger.error("A client error occurred: %s", message) print(f"A client error occured: {message}") else: print(f"Finished generating text with model {model_id}.") if __name__ == "__main__": main()
Choosing which tool the model requests
The optional toolChoice field in toolConfig controls whether the model has to request a tool. Which settings you can use depends on the individual model that you call, not on the model provider or the model family:
auto(the default) is supported by every model that supports tool use. The model decides whether to request a tool or to generate text instead.any, which requires the model to request at least one tool, andtool, which requires the model to request one tool that you name, are supported by some models but not others. A model can support one of the two and not the other.
If a model doesn't support the setting that you send, the request fails with a ValidationException rather than falling back to auto. The message identifies the field to remove, for example This model doesn't support the toolConfig.toolChoice.tool field. Remove toolConfig.toolChoice.tool and try again. Some models instead return the error that the model provider reports, such as The model returned the following errors: tool_choice: type "tool" and "any" are not supported for this model. For the Anthropic Claude models that don't support any or tool, see Forced tool use.
A model that doesn't support tool use at all fails with This model doesn't support tool use. for every toolChoice setting, including auto.
Tool use with response streaming
Support for tool use and support for response streaming are independent capabilities. Most models that support tool use also accept a toolConfig field on ConverseStream, but the following models accept tools only on the non-streaming Converse operation and reject them on ConverseStream with This model doesn't support tool use in streaming mode.:
Llama 3.1 8B Instruct, Llama 3.1 70B Instruct, and Llama 3.3 70B Instruct
Llama 4 Scout 17B Instruct and Llama 4 Maverick 17B Instruct
Mistral Large 1 (24.02), Mistral Large 2 (24.07), and Pixtral Large (25.02)
Writer Palmyra X4 and Writer Palmyra X5
Response streaming itself works with all of these models. Only the combination of streaming and tool use is unsupported. If you need tool use with one of them, call Converse instead.
About the tool use ID
Each tool request in a response carries a toolUseId that identifies that request. You echo the same value back in the corresponding toolResult block so that the model can match the result to the request. How Amazon Bedrock produces this value depends on the kind of tool and on the model:
For tools that you define in
toolConfig, Amazon Bedrock generates the value. It is a randomly generated identifier prefixed withtooluse_, and it is not produced by, or derived from, the model's output.For Amazon Bedrock server-side tools and MCP tools, Amazon Bedrock computes the value deterministically from the tool request rather than generating it randomly. The value still carries the
tooluse_prefix.For some models, the identifier that the model provider returns is passed through unchanged. Such a value uses the provider's own format, does not carry the
tooluse_prefix, and can contain text derived from your request, such as the name of the tool.
To tell the two apart at runtime, check the prefix. Every identifier that Amazon Bedrock generates begins with tooluse_ followed by 22 characters. An identifier that does not begin with tooluse_ came from the model provider. This behavior can differ between models in the same family, so test the specific model that you call.
Important
If your application redacts model-generated content, for example to meet compliance obligations for protected health information, do not assume that toolUseId is generated by Amazon Bedrock for every model. Treat the value as model-derived unless you have confirmed the behavior for the specific model that you call.
Using Invoke APIs for client-side tool use
It is possible to use tools with the base inference operations (InvokeModel or InvokeModelWithResponseStream). To find the inference parameters that you pass in the request body, see the inference parameters for the model that you want to use.
