

# Client-side tool use

If you use the Responses API, Chat Completions API, Converse API, or InvokeModel API to send the request, then the model uses client-side tool calling. This means that in your code, you call the tool on the model's behalf. In this scenario, assume the tool implementation is an API. The tool could just as easily be a database, Lambda function, or some other software. You decide how you want to implement the tool. You then continue the conversation with the model by supplying a message with the result from the tool. Finally, the model generates a response for the original message that includes the tool results that you sent to the model.

Let us define the tool we will use for tool use. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.

```
def get_most_popular_song(station_name: str) -> str:
    stations = {
        "Radio Free Mars": "Starman – David Bowie",
        "Neo Tokyo FM": "Plastic Love – Mariya Takeuchi",
        "Cloud Nine Radio": "Blinding Lights – The Weeknd",
    }
    return stations.get(station_name, "Unknown Station – No chart data available")
```

**Using Responses API for client-side tooling**

You can use the [Function calling](https://platform.openai.com/docs/guides/function-calling) feature provided by OpenAI to call this tool. Responses API is OpenAI's preferred API. Here is the Python code for Responses API for client-side tooling:

```
from openai import OpenAI
import json

client = OpenAI()

response = client.responses.create(
    model="oss-gpt-120b",
    input="What is the most popular song on Radio Free Mars?",
    tools=[
        {
            "type": "function",
            "name": "get_most_popular_song",
            "description": "Returns the most popular song on a radio station",
            "parameters": {
                "type": "object",
                "properties": {
                    "station_name": {
                        "type": "string",
                        "description": "Name of the radio station"
                    }
                },
                "required": ["station_name"]
            }
        }
    ]
)

if response.output and response.output[0].content:
    tool_call = response.output[0].content[0]
    args = json.loads(tool_call["arguments"])
    result = get_most_popular_song(args["station_name"])
    
    final_response = client.responses.create(
        model="oss-gpt-120b",
        input=[
            {
                "role": "tool",
                "tool_call_id": tool_call["id"],
                "content": result
            }
        ]
    )
    
    print(final_response.output_text)
```

**Using Chat Completions API for client-side tooling**

You can use the Chat Completions API also. Here is the Python code for using Chat Completions:

```
    from openai import OpenAI
import json

client = OpenAI()

completion = client.chat.completions.create(
    model="oss-gpt-120b",
    messages=[{"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_most_popular_song",
            "description": "Returns the most popular song on a radio station",
            "parameters": {
                "type": "object",
                "properties": {
                   "station_name": {"type": "string", "description": "Name of the radio station"}
                },
                "required": ["station_name"]
            }
        }
    }]
)

message = completion.choices[0].message

if message.tool_calls:
    tool_call = message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    result = get_most_popular_song(args["station_name"])

    followup = client.chat.completions.create(
        model="oss-gpt-120b",
        messages=[
            {"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"},
            message,
            {"role": "tool", "tool_call_id": tool_call.id, "content": result}
        ]
    )

    print(followup.choices[0].message.content)
```

For more details on the using Function Calling on Responses API and Chat Completions API, see [Function Calling](https://platform.openai.com/docs/guides/function-calling) in OpenAI.

**Using Converse API for client-side tooling**

You can use the [Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) to let a model use a tool in a conversation. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.

```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""Shows how to use tools with the Converse API and the Cohere Command R model."""

import logging
import json
import boto3
from botocore.exceptions import ClientError


class StationNotFoundError(Exception):
    """Raised when a radio station isn't found."""
    pass


logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def get_top_song(call_sign):
    """Returns the most popular song for the requested station.

    Args:
        call_sign (str): The call sign for the station for which you want
            the most popular song.

    Returns:
        response (json): The most popular song and artist.
    """
    song = ""
    artist = ""

    if call_sign == 'WZPZ':
        song = "Elemental Hotel"
        artist = "8 Storey Hike"
    else:
        raise StationNotFoundError(f"Station {call_sign} not found.")

    return song, artist


def generate_text(bedrock_client, model_id, tool_config, input_text):
    """Generates text using the supplied Amazon Bedrock model. If necessary,
    the function handles tool use requests and sends the result to the model.

    Args:
        bedrock_client: The Boto3 Bedrock runtime client.
        model_id (str): The Amazon Bedrock model ID.
        tool_config (dict): The tool configuration.
        input_text (str): The input text.

    Returns:
        Nothing.
    """
    logger.info("Generating text with model %s", model_id)

    # Create the initial message from the user input.
    messages = [{"role": "user",
                 "content": [{"text": input_text}]}]

    response = bedrock_client.converse(modelId=model_id,
                                       messages=messages,
                                       toolConfig=tool_config)

    output_message = response['output']['message']
    messages.append(output_message)

    stop_reason = response['stopReason']

    if stop_reason == 'tool_use':
        # Tool use requested. Call the tool and send the result to the model.
        tool_requests = response['output']['message']['content']

        for tool_request in tool_requests:
            if 'toolUse' in tool_request:
                tool = tool_request['toolUse']
                logger.info("Requesting tool %s. Request: %s",
                            tool['name'], tool['toolUseId'])

                if tool['name'] == 'top_song':
                    tool_result = {}
                    try:
                        song, artist = get_top_song(tool['input']['sign'])
                        tool_result = {"toolUseId": tool['toolUseId'],
                                       "content": [{"json": {"song": song, "artist": artist}}]}
                    except StationNotFoundError as err:
                        tool_result = {"toolUseId": tool['toolUseId'],
                                       "content": [{"text": err.args[0]}],
                                       "status": 'error'}

                    tool_result_message = {"role": "user",
                                           "content": [{"toolResult": tool_result}]}
                    messages.append(tool_result_message)

        # Send the tool result to the model.
        response = bedrock_client.converse(modelId=model_id,
                                           messages=messages,
                                           toolConfig=tool_config)

        output_message = response['output']['message']

    # print the final response from the model.
    for content in output_message['content']:
        print(json.dumps(content, indent=4))


def main():
    """Entrypoint for tool use example."""
    logging.basicConfig(level=logging.INFO,
                        format="%(levelname)s: %(message)s")

    model_id = "cohere.command-r-v1:0"
    input_text = "What is the most popular song on WZPZ?"

    tool_config = {
        "tools": [
            {
                "toolSpec": {
                    "name": "top_song",
                    "description": "Get the most popular song played on a radio station.",
                    "inputSchema": {
                        "json": {
                            "type": "object",
                            "properties": {
                                "sign": {
                                    "type": "string",
                                    "description": "The call sign for the radio station for which you want the most popular song. Example calls signs are WZPZ, and WKRP."
                                }
                            },
                            "required": ["sign"]
                        }
                    }
                }
            }
        ]
    }

    bedrock_client = boto3.client(service_name='bedrock-runtime')

    try:
        print(f"Question: {input_text}")
        generate_text(bedrock_client, model_id, tool_config, input_text)
    except ClientError as err:
        message = err.response['Error']['Message']
        logger.error("A client error occurred: %s", message)
        print(f"A client error occured: {message}")
    else:
        print(f"Finished generating text with model {model_id}.")


if __name__ == "__main__":
    main()
```

**Choosing which tool the model requests**

The optional `toolChoice` field in `toolConfig` controls whether the model has to request a tool. Which settings you can use depends on the individual model that you call, not on the model provider or the model family:
+ `auto` (the default) is supported by every model that supports tool use. The model decides whether to request a tool or to generate text instead.
+ `any`, which requires the model to request at least one tool, and `tool`, which requires the model to request one tool that you name, are supported by some models but not others. A model can support one of the two and not the other.

If a model doesn't support the setting that you send, the request fails with a `ValidationException` rather than falling back to `auto`. The message identifies the field to remove, for example `This model doesn't support the toolConfig.toolChoice.tool field. Remove toolConfig.toolChoice.tool and try again.` Some models instead return the error that the model provider reports, such as `The model returned the following errors: tool_choice: type "tool" and "any" are not supported for this model.` For the Anthropic Claude models that don't support `any` or `tool`, see [Forced tool use](model-parameters-anthropic-claude-messages-tool-use.md#model-parameters-anthropic-claude-forced-tool-use).

A model that doesn't support tool use at all fails with `This model doesn't support tool use.` for every `toolChoice` setting, including `auto`.

**Tool use with response streaming**

Support for tool use and support for response streaming are independent capabilities. Most models that support tool use also accept a `toolConfig` field on `ConverseStream`, but the following models accept tools only on the non-streaming `Converse` operation and reject them on `ConverseStream` with `This model doesn't support tool use in streaming mode.`:
+ Llama 3.1 8B Instruct, Llama 3.1 70B Instruct, and Llama 3.3 70B Instruct
+ Llama 4 Scout 17B Instruct and Llama 4 Maverick 17B Instruct
+ Mistral Large 1 (24.02), Mistral Large 2 (24.07), and Pixtral Large (25.02)
+ Writer Palmyra X4 and Writer Palmyra X5

Response streaming itself works with all of these models. Only the combination of streaming and tool use is unsupported. If you need tool use with one of them, call `Converse` instead.

**About the tool use ID**

Each tool request in a response carries a `toolUseId` that identifies that request. You echo the same value back in the corresponding `toolResult` block so that the model can match the result to the request. How Amazon Bedrock produces this value depends on the kind of tool and on the model:
+ For tools that you define in `toolConfig`, Amazon Bedrock generates the value. It is a randomly generated identifier prefixed with `tooluse_`, and it is not produced by, or derived from, the model's output.
+ For Amazon Bedrock server-side tools and MCP tools, Amazon Bedrock computes the value deterministically from the tool request rather than generating it randomly. The value still carries the `tooluse_` prefix.
+ For some models, the identifier that the model provider returns is passed through unchanged. Such a value uses the provider's own format, does not carry the `tooluse_` prefix, and can contain text derived from your request, such as the name of the tool.

To tell the two apart at runtime, check the prefix. Every identifier that Amazon Bedrock generates begins with `tooluse_` followed by 22 characters. An identifier that does not begin with `tooluse_` came from the model provider. This behavior can differ between models in the same family, so test the specific model that you call.

**Important**  
If your application redacts model-generated content, for example to meet compliance obligations for protected health information, do not assume that `toolUseId` is generated by Amazon Bedrock for every model. Treat the value as model-derived unless you have confirmed the behavior for the specific model that you call.

**Using Invoke APIs for client-side tool use**

It is possible to use tools with the base inference operations ([InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html)). To find the inference parameters that you pass in the request body, see the [inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html) for the model that you want to use.