Use a tool to complete an Amazon Bedrock model response

You can use the Amazon Bedrock API to give a model access to tools that can help it generate responses for messages that you send to the model. For example, you might have a chat application that lets users find out the most popular song played on a radio station. To answer a request for the most popular song, a model needs a tool that can query and return the song information.

Note

You can now use structured outputs with tool use. See Get validated JSON results from models for more details.

In Amazon Bedrock, the model doesn't directly call the tool. Rather, when you send a message to a model, you also supply a definition for one or more tools that could potentially help the model generate a response. In this example, you would supply a definition for a tool that returns the most popular song for a specified radio station. If the model determines that it needs the tool to generate a response for the message, the model, depending on the API used to invoke the model, can perform either client-side calling or ask Bedrock to call the tool using server-side tool calling. Let us discuss these two options in more detail.

Client-side tool calling

If you use the Responses API, Chat Completions API, Converse API, or InvokeModel API to send the request, then the model uses client-side tool calling. This means that in your code, you call the tool on the model's behalf. In this scenario, assume the tool implementation is an API. The tool could just as easily be a database, Lambda function, or some other software. You decide how you want to implement the tool. You then continue the conversation with the model by supplying a message with the result from the tool. Finally, the model generates a response for the original message that includes the tool results that you sent to the model.

Let us define the tool we will use for tool use. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.


def get_most_popular_song(station_name: str) -> str:
    stations = {
        "Radio Free Mars": "Starman – David Bowie",
        "Neo Tokyo FM": "Plastic Love – Mariya Takeuchi",
        "Cloud Nine Radio": "Blinding Lights – The Weeknd",
    }
    return stations.get(station_name, "Unknown Station – No chart data available")

Using Responses API for client-side tooling

You can use the Function calling feature provided by OpenAI to call this tool. Responses API is OpenAI's preferred API. Here is the Python code for Responses API for client-side tooling:


from openai import OpenAI
import json

client = OpenAI()

response = client.responses.create(
    model="oss-gpt-120b",
    input="What is the most popular song on Radio Free Mars?",
    tools=[
        {
            "type": "function",
            "name": "get_most_popular_song",
            "description": "Returns the most popular song on a radio station",
            "parameters": {
                "type": "object",
                "properties": {
                    "station_name": {
                        "type": "string",
                        "description": "Name of the radio station"
                    }
                },
                "required": ["station_name"]
            }
        }
    ]
)

if response.output and response.output[0].content:
    tool_call = response.output[0].content[0]
    args = json.loads(tool_call["arguments"])
    result = get_most_popular_song(args["station_name"])
    
    final_response = client.responses.create(
        model="oss-gpt-120b",
        input=[
            {
                "role": "tool",
                "tool_call_id": tool_call["id"],
                "content": result
            }
        ]
    )
    
    print(final_response.output_text)

Using Chat Completions API for client-side tooling

You can use the Chat Completions API also. Here is the Python code for using Chat Completions:



    from openai import OpenAI
import json

client = OpenAI()

completion = client.chat.completions.create(
    model="oss-gpt-120b",
    messages=[{"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_most_popular_song",
            "description": "Returns the most popular song on a radio station",
            "parameters": {
                "type": "object",
                "properties": {
                   "station_name": {"type": "string", "description": "Name of the radio station"}
                },
                "required": ["station_name"]
            }
        }
    }]
)

message = completion.choices[0].message

if message.tool_calls:
    tool_call = message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    result = get_most_popular_song(args["station_name"])

    followup = client.chat.completions.create(
        model="oss-gpt-120b",
        messages=[
            {"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"},
            message,
            {"role": "tool", "tool_call_id": tool_call.id, "content": result}
        ]
    )

    print(followup.choices[0].message.content)

For more details on the using Function Calling on Responses API and Chat Completions API, see Function Calling in OpenAI.

Using Converse API for client-side tooling

You can use the Converse API to let a model use a tool in a conversation. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station.


# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""Shows how to use tools with the Converse API and the Cohere Command R model."""

import logging
import json
import boto3
from botocore.exceptions import ClientError


class StationNotFoundError(Exception):
    """Raised when a radio station isn't found."""
    pass


logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)


def get_top_song(call_sign):
    """Returns the most popular song for the requested station.

    Args:
        call_sign (str): The call sign for the station for which you want
            the most popular song.

    Returns:
        response (json): The most popular song and artist.
    """
    song = ""
    artist = ""

    if call_sign == 'WZPZ':
        song = "Elemental Hotel"
        artist = "8 Storey Hike"
    else:
        raise StationNotFoundError(f"Station {call_sign} not found.")

    return song, artist


def generate_text(bedrock_client, model_id, tool_config, input_text):
    """Generates text using the supplied Amazon Bedrock model. If necessary,
    the function handles tool use requests and sends the result to the model.

    Args:
        bedrock_client: The Boto3 Bedrock runtime client.
        model_id (str): The Amazon Bedrock model ID.
        tool_config (dict): The tool configuration.
        input_text (str): The input text.

    Returns:
        Nothing.
    """
    logger.info("Generating text with model %s", model_id)

    # Create the initial message from the user input.
    messages = [{"role": "user",
                 "content": [{"text": input_text}]}]

    response = bedrock_client.converse(modelId=model_id,
                                       messages=messages,
                                       toolConfig=tool_config)

    output_message = response['output']['message']
    messages.append(output_message)

    stop_reason = response['stopReason']

    if stop_reason == 'tool_use':
        # Tool use requested. Call the tool and send the result to the model.
        tool_requests = response['output']['message']['content']

        for tool_request in tool_requests:
            if 'toolUse' in tool_request:
                tool = tool_request['toolUse']
                logger.info("Requesting tool %s. Request: %s",
                            tool['name'], tool['toolUseId'])

                if tool['name'] == 'top_song':
                    tool_result = {}
                    try:
                        song, artist = get_top_song(tool['input']['sign'])
                        tool_result = {"toolUseId": tool['toolUseId'],
                                       "content": [{"json": {"song": song, "artist": artist}}]}
                    except StationNotFoundError as err:
                        tool_result = {"toolUseId": tool['toolUseId'],
                                       "content": [{"text": err.args[0]}],
                                       "status": 'error'}

                    tool_result_message = {"role": "user",
                                           "content": [{"toolResult": tool_result}]}
                    messages.append(tool_result_message)

        # Send the tool result to the model.
        response = bedrock_client.converse(modelId=model_id,
                                           messages=messages,
                                           toolConfig=tool_config)

        output_message = response['output']['message']

    # print the final response from the model.
    for content in output_message['content']:
        print(json.dumps(content, indent=4))


def main():
    """Entrypoint for tool use example."""
    logging.basicConfig(level=logging.INFO,
                        format="%(levelname)s: %(message)s")

    model_id = "cohere.command-r-v1:0"
    input_text = "What is the most popular song on WZPZ?"

    tool_config = {
        "tools": [
            {
                "toolSpec": {
                    "name": "top_song",
                    "description": "Get the most popular song played on a radio station.",
                    "inputSchema": {
                        "json": {
                            "type": "object",
                            "properties": {
                                "sign": {
                                    "type": "string",
                                    "description": "The call sign for the radio station for which you want the most popular song. Example calls signs are WZPZ, and WKRP."
                                }
                            },
                            "required": ["sign"]
                        }
                    }
                }
            }
        ]
    }

    bedrock_client = boto3.client(service_name='bedrock-runtime')

    try:
        print(f"Question: {input_text}")
        generate_text(bedrock_client, model_id, tool_config, input_text)
    except ClientError as err:
        message = err.response['Error']['Message']
        logger.error("A client error occurred: %s", message)
        print(f"A client error occured: {message}")
    else:
        print(f"Finished generating text with model {model_id}.")


if __name__ == "__main__":
    main()

Using Invoke APIs for client-side tool use

It is possible to use tools with the base inference operations (InvokeModel or InvokeModelWithResponseStream). To find the inference parameters that you pass in the request body, see the inference parameters for the model that you want to use.

Server-side tool calling

If you use the Responses API to invoke the model, then it can use server-side tool calling, in addition to the client-side tool calling we discussed before. Server-side tool calling is a mechanism where tools (APIs, functions, workflows) are executed in a trusted, backend environment, not on the client. This improves the security, reliability, and governance posture of the application. Before Amazon Bedrock executes the Lambda function which implements the tool use, it ensures that the Lambda function has the same IAM policy as that of the application that calls it. As Amazon Bedrock is driving the execution of the tools against, clients can focus on implementing their business logic, rather than adding tool functionality. Amazon Bedrock also supports the highest governance standards such as ISO, SOC, and HIPAA eligible. Customers can either submit their own custom Lambda function to run the tool or use existing pre-defined tools, such as notes and tasks. Server-side tools using the Responses API is available starting with OpenAI's GPT OSS 20B/120B models, with support for other models coming soon. You can use the Models API to discover available models that you can use with the Responses API. For more details on the Responses API, see Generate responses using OpenAI APIs.

There are two types of tools you can use with Amazon Bedrock: Custom tools using Lambda, or pre-defined tools supported by Bedrock. In this section we will review how to create a custom Lambda tool with the Responses API. Let's discuss both in details.

Custom tools using Lambda in Responses API

By using a Lambda function as a custom tool in Bedrock, you can extend the capabilities of the agent by integrating custom AWS Lambda functions as tools. This enables you to create serverless, scalable tools that can be called by AI assistants and other applications through the Model Context Protocol (MCP). Here are the advantages of this feature:

Extend Functionality: Add custom business logic, API integrations, or data processing capabilities.
Run tools securely: Lambda allows tools to access resources inside a VPC without having to grant full VPC access.
Serverless Architecture: No infrastructure management, Lambda handles scaling automatically.
Cost Effective: Pay only for execution time, not idle resources.
Easy Integration: Lambda functions appear alongside built-in tools seamlessly.

To let a model in Amazon Bedrock use a tool to complete a response for a message, you send the message and the definitions for one or more tools to the model. Based on your application's prompt, if the model determines that one of the tools can help generate a response, it returns a request for Bedrock to use the tool and send the tool results back to the model. The model then uses the results to generate a response to the original message.

The following steps show how to use a tool with the Responses API.

How it works

Lambda function: Create your Lambda function that implements the MCP protocol
Tool Discovery: Bedrock calls your Lambda function to discover available tools
Tool Registration: Your tools are registered with Bedrock
Tool Execution: When agent request your tool, Bedrock invokes your Lambda function
Response Handling: Results are returned to the agent through the standard interface

Step 1: Define the Lambda function to get the most popular song

Create a Lambda function that implements the MCP protocol. Here's a simple Python example:


import json

def lambda_handler(event, context):
    # Parse JSON-RPC request
    method = event.get('method')
    params = event.get('params', {})
    request_id = event.get('id')
    
    if method == 'tools/list':
        return {
            "jsonrpc": "2.0",
            "id": request_id,
            "result": {
                "tools": [
                    {
                        "name": "my_custom_tool",
                        "description": "My custom business logic tool",
                        "inputSchema": {
                            "type": "object",
                            "properties": {
                                "input": {
                                    "type": "string",
                                    "description": "Input text to process"
                                }
                            },
                            "required": ["input"]
                        }
                    }
                ]
            }
        }
    elif method == 'tools/call':
        tool_name = params.get('name')
        arguments = params.get('arguments', {})
        
        if tool_name == 'my_custom_tool':
            # Your custom logic here
            result = f"Processed: {arguments.get('input', '')}"
            return {
                "jsonrpc": "2.0",
                "id": request_id,
                "result": {
                    "content": [
                        {
                            "type": "text",
                            "text": result
                        }
                    ]
                }
            }
    
    # Error response for unsupported methods
    return {
        "jsonrpc": "2.0",
        "id": request_id,
        "error": {
            "code": -32601,
            "message": "Method not found"
        }
    }

Step 2: Deploy the Lambda function

Next, use your IAM role to deploy this Lambda function to get an ARN. You can read more on deploying a Lambda function here.


# Example using AWS CLI
aws lambda create-function \
  --function-name my-custom-tool \
  --runtime python3.14 \
  --role arn:aws:iam::YOUR-ACCOUNT:role/lambda-execution-role \
  --handler lambda_function.lambda_handler \
  --zip-file fileb://function.zip

Let's say that your ARN is: arn:aws:lambda:us-west-2:123456789012:function:my-custom-tool

Step 3: Define the message and tool definition in your inference request

To send the message and tool definition, you use the Responses API operations. Amazon Bedrock uses the connectors and remote MCP servers functionality of the Responses API to provide tool-use capability. The definition of the tool is a JSON schema that you pass in the mcp request parameter to the Create operation. In the connector_id field in the Responses connectors API, you can pass in the Lambda ARN that you have created in the previous step. You do not need to provide authorization credentials since Bedrock uses the same IAM roles and policy that is used for your application that invokes the model. The following is an example schema for a tool that gets the most popular song played on a radio station.


from openai import OpenAI

client = OpenAI()

resp = client.responses.create(
    model="oss-gpt-120b",
    tools=[
        {
            "type": "mcp",
            "server_label": "xamzn_arn",
            "connector_id": "arn:aws:lambda:us-west-2:123456789012:function:my-custom-tool",
            "require_approval": "never",
        },
    ],
    input="My custom prompt.",
)

print(resp.output_text)

Step 4: Bedrock calls the tool and passes the response back to the model

The ability to use the connector tool is available in the models that support the Responses API. Check which tools support your model here. When you're using tools using the Responses API, you only pay for tokens used when importing tool definitions or making tool calls. There are no additional fees involved per tool call.

When you specify a Lambda function in the tools parameter, the API will attempt to get a list of tools from the server. If successful in retrieving the list of tools, a new mcp_list_tools output item will appear in the model response output. The tools property of this object will show the tools that were successfully imported. Once the model has access to these tool definitions, it may choose to call them depending on what's in the model's context. When the model decides to call a Lambda tool, the API will make a request to the Lambda function to call the tool and put its output into the model's context. You can read more on the list tools and calling tools in the OpenAI documentation. Note that your Lambda function must have the same IAM roles and policy attached to it as that of the application calling the model in Bedrock, otherwise the Lambda function will fail. The following is the error definition.


{
    "jsonrpc": "2.0",
    "id": 1,
    "error": {
        "code": -32000,
        "message": "Tool execution failed",
        "data": "Additional error details"
    }
}

Using AWS provided tools in Responses API

There are two AWS provided tools that are available in Bedrock: Note-taking functionality (notes tool) and Task management (tasks tool). Let us go over them both in details.

Notes Tool Overview

The notes tool allows you to store and retrieve key-value pairs within the same conversation session. This provides a simple memory mechanism for maintaining context across multiple interactions. Keys are case-sensitive strings and there is no limit on key length or naming conventions. The system overwrites previous values for the same key. Values are stored as strings (JSON, URLs etc.) and no size limits enforced at tool level. Values are persisted through the entire conversation session. Memory is scoped to the current conversation only.

Parameters

Parameter	Type	Required	Description
`operation`	string	Yes	The operation to perform: `"store"` or `"recall"`
`key`	string	Yes	The key identifier for the memory item
`value`	string	Conditional	The value to store (required only for `"store"` operation)

Valid Operations

store: Save a key-value pair to memory
recall: Retrieve a value by its key

You can use either natural language (e.g. "Remember that my favorite color is blue", "What did I tell you about my favorite color?", "Store the fact that I prefer morning meetings", "Recall what I said about meeting preferences") or you can use direct tools calls in your prompt ("Use the notes tool to store my email as john@example.com", "Check the notes for my email address").

Tasks Tool Overview

The tasks tool provides a Last-In-First-Out (LIFO) stack for managing tasks within a conversation session. This allows you to push tasks onto a stack and pop them off in reverse order, making it perfect for managing nested workflows, temporary reminders, or hierarchical task management. Tasks persist throughout the entire conversation session. Stack state is maintained across multiple interactions. Memory is scoped to the current conversation only.

Parameter	Type	Required	Description
`operation`	string	Yes	The operation to perform: `"push"` or `"pop"`
`description`	string	Conditional	The description of the task item (required only for `"push"` operation)
`summary`	string	No	Optional summary about the task item (only for `"push"` operation)

You can call the Tasks tool by either using natural languages (e.g. "Add a task to review the budget", "Push a reminder to call the client", "What's the next task I need to do?", "Pop the most recent task", "Get the latest task from my stack") or you can call the tool directly in your prompt ("Use the tasks tool to push 'finish presentation'", "Pop a task from the stack", "Add 'schedule meeting' to my task list").

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

API restrictions

Structured Outputs