The AWS SDK for JavaScript V3 API Reference Guide describes in detail all the API operations for the AWS SDK for JavaScript version 3 (V3).
Amazon Bedrock Runtime examples using SDK for JavaScript (v3)
The following code examples show you how to perform actions and implement common scenarios by using the AWS SDK for JavaScript (v3) with Amazon Bedrock Runtime.
Scenarios are code examples that show you how to accomplish specific tasks by calling multiple functions within a service or combined with other AWS services.
Each example includes a link to the complete source code, where you can find instructions on how to set up and run the code in context.
Get started
The following code examples show how to get started using Amazon Bedrock.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. /** * @typedef {Object} Content * @property {string} text * * @typedef {Object} Usage * @property {number} input_tokens * @property {number} output_tokens * * @typedef {Object} ResponseBody * @property {Content[]} content * @property {Usage} usage */ import { fileURLToPath } from "url"; import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; const AWS_REGION = "us-east-1"; const MODEL_ID = "anthropic.claude-3-haiku-20240307-v1:0"; const PROMPT = "Hi. In a short paragraph, explain what you can do."; const hello = async () => { console.log("=".repeat(35)); console.log("Welcome to the Amazon Bedrock demo!"); console.log("=".repeat(35)); console.log("Model: Anthropic Claude 3 Haiku"); console.log(`Prompt: ${PROMPT}\n`); console.log("Invoking model...\n"); // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: AWS_REGION }); // Prepare the payload for the model. const payload = { anthropic_version: "bedrock-2023-05-31", max_tokens: 1000, messages: [{ role: "user", content: [{ type: "text", text: PROMPT }] }], }; // Invoke Claude with the payload and wait for the response. const apiResponse = await client.send( new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId: MODEL_ID, }), ); // Decode and return the response(s) const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {ResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); const responses = responseBody.content; if (responses.length === 1) { console.log(`Response: ${responses[0].text}`); } else { console.log("Haiku returned multiple responses:"); console.log(responses); } console.log(`\nNumber of input tokens: ${responseBody.usage.input_tokens}`); console.log(`Number of output tokens: ${responseBody.usage.output_tokens}`); }; if (process.argv[1] === fileURLToPath(import.meta.url)) { await hello(); }
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
Topics
Scenarios
The following code example shows how to prepare and send a prompt to a variety of large-language models (LLMs) on Amazon Bedrock
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. import { fileURLToPath } from "url"; import { Scenario, ScenarioAction, ScenarioInput, ScenarioOutput, } from "@aws-doc-sdk-examples/lib/scenario/index.js"; import { FoundationModels } from "../config/foundation_models.js"; /** * @typedef {Object} ModelConfig * @property {Function} module * @property {Function} invoker * @property {string} modelId * @property {string} modelName */ const greeting = new ScenarioOutput( "greeting", "Welcome to the Amazon Bedrock Runtime client demo!", { header: true }, ); const selectModel = new ScenarioInput("model", "First, select a model:", { type: "select", choices: Object.values(FoundationModels).map((model) => ({ name: model.modelName, value: model, })), }); const enterPrompt = new ScenarioInput("prompt", "Now, enter your prompt:", { type: "input", }); const printDetails = new ScenarioOutput( "print details", /** * @param {{ model: ModelConfig, prompt: string }} c */ (c) => console.log(`Invoking ${c.model.modelName} with '${c.prompt}'...`), { slow: false }, ); const invokeModel = new ScenarioAction( "invoke model", /** * @param {{ model: ModelConfig, prompt: string, response: string }} c */ async (c) => { const modelModule = await c.model.module(); const invoker = c.model.invoker(modelModule); c.response = await invoker(c.prompt, c.model.modelId); }, ); const printResponse = new ScenarioOutput( "print response", /** * @param {{ response: string }} c */ (c) => c.response, { slow: false }, ); const scenario = new Scenario("Amazon Bedrock Runtime Demo", [ greeting, selectModel, enterPrompt, printDetails, invokeModel, printResponse, ]); if (process.argv[1] === fileURLToPath(import.meta.url)) { scenario.run(); }
-
For API details, see the following topics in AWS SDK for JavaScript API Reference.
-
AI21 Labs Jurassic-2
The following code example shows how to send a text message to AI21 Labs Jurassic-2, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to AI21 Labs Jurassic-2, using Bedrock's Converse API.
// Use the Conversation API to send a text message to AI21 Labs Jurassic-2. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Jurassic-2 Mid. const modelId = "ai21.j2-mid-v1"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to AI21 Labs Jurassic-2, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
import { fileURLToPath } from "url"; import { FoundationModels } from "../../config/foundation_models.js"; import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; /** * @typedef {Object} Data * @property {string} text * * @typedef {Object} Completion * @property {Data} data * * @typedef {Object} ResponseBody * @property {Completion[]} completions */ /** * Invokes an AI21 Labs Jurassic-2 model. * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "ai21.j2-mid-v1". */ export const invokeModel = async (prompt, modelId = "ai21.j2-mid-v1") => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { prompt, maxTokens: 500, temperature: 0.5, }; // Invoke the model with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); // Decode and return the response(s). const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {ResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); return responseBody.completions[0].data.text; }; // Invoke the function if this file was run directly. if (process.argv[1] === fileURLToPath(import.meta.url)) { const prompt = 'Complete the following in one sentence: "Once upon a time..."'; const modelId = FoundationModels.JURASSIC2_MID.modelId; console.log(`Prompt: ${prompt}`); console.log(`Model ID: ${modelId}`); try { console.log("-".repeat(53)); const response = await invokeModel(prompt, modelId); console.log(response); } catch (err) { console.log(err); } }
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
Amazon Titan Text
The following code example shows how to send a text message to Amazon Titan Text, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Amazon Titan Text, using Bedrock's Converse API.
// Use the Conversation API to send a text message to Amazon Titan Text. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Titan Text Premier. const modelId = "amazon.titan-text-premier-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Amazon Titan Text, using Bedrock's Converse API and process the response stream in real-time.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Amazon Titan Text, using Bedrock's Converse API and process the response stream in real-time.
// Use the Conversation API to send a text message to Amazon Titan Text. import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Titan Text Premier. const modelId = "amazon.titan-text-premier-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseStreamCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the streamed response text in real-time. for await (const item of response.stream) { if (item.contentBlockDelta) { process.stdout.write(item.contentBlockDelta.delta?.text); } } } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see ConverseStream in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Amazon Titan Text, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
import { fileURLToPath } from "url"; import { FoundationModels } from "../../config/foundation_models.js"; import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; /** * @typedef {Object} ResponseBody * @property {Object[]} results */ /** * Invokes an Amazon Titan Text generation model. * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "amazon.titan-text-express-v1". */ export const invokeModel = async ( prompt, modelId = "amazon.titan-text-express-v1", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { inputText: prompt, textGenerationConfig: { maxTokenCount: 4096, stopSequences: [], temperature: 0, topP: 1, }, }; // Invoke the model with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); // Decode and return the response. const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {ResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); return responseBody.results[0].outputText; }; // Invoke the function if this file was run directly. if (process.argv[1] === fileURLToPath(import.meta.url)) { const prompt = 'Complete the following in one sentence: "Once upon a time..."'; const modelId = FoundationModels.TITAN_TEXT_G1_EXPRESS.modelId; console.log(`Prompt: ${prompt}`); console.log(`Model ID: ${modelId}`); try { console.log("-".repeat(53)); const response = await invokeModel(prompt, modelId); console.log(response); } catch (err) { console.log(err); } }
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
Anthropic Claude
The following code example shows how to send a text message to Anthropic Claude, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Anthropic Claude, using Bedrock's Converse API.
// Use the Conversation API to send a text message to Anthropic Claude. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Claude 3 Haiku. const modelId = "anthropic.claude-3-haiku-20240307-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Anthropic Claude, using Bedrock's Converse API and process the response stream in real-time.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Anthropic Claude, using Bedrock's Converse API and process the response stream in real-time.
// Use the Conversation API to send a text message to Anthropic Claude. import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Claude 3 Haiku. const modelId = "anthropic.claude-3-haiku-20240307-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseStreamCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the streamed response text in real-time. for await (const item of response.stream) { if (item.contentBlockDelta) { process.stdout.write(item.contentBlockDelta.delta?.text); } } } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see ConverseStream in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Anthropic Claude, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
import { fileURLToPath } from "url"; import { FoundationModels } from "../../config/foundation_models.js"; import { BedrockRuntimeClient, InvokeModelCommand, InvokeModelWithResponseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; /** * @typedef {Object} ResponseContent * @property {string} text * * @typedef {Object} MessagesResponseBody * @property {ResponseContent[]} content * * @typedef {Object} Delta * @property {string} text * * @typedef {Object} Message * @property {string} role * * @typedef {Object} Chunk * @property {string} type * @property {Delta} delta * @property {Message} message */ /** * Invokes Anthropic Claude 3 using the Messages API. * * To learn more about the Anthropic Messages API, go to: * https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-3-haiku-20240307-v1:0". */ export const invokeModel = async ( prompt, modelId = "anthropic.claude-3-haiku-20240307-v1:0", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { anthropic_version: "bedrock-2023-05-31", max_tokens: 1000, messages: [ { role: "user", content: [{ type: "text", text: prompt }], }, ], }; // Invoke Claude with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); // Decode and return the response(s) const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {MessagesResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); return responseBody.content[0].text; }; /** * Invokes Anthropic Claude 3 and processes the response stream. * * To learn more about the Anthropic Messages API, go to: * https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-3-haiku-20240307-v1:0". */ export const invokeModelWithResponseStream = async ( prompt, modelId = "anthropic.claude-3-haiku-20240307-v1:0", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { anthropic_version: "bedrock-2023-05-31", max_tokens: 1000, messages: [ { role: "user", content: [{ type: "text", text: prompt }], }, ], }; // Invoke Claude with the payload and wait for the API to respond. const command = new InvokeModelWithResponseStreamCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); let completeMessage = ""; // Decode and process the response stream for await (const item of apiResponse.body) { /** @type Chunk */ const chunk = JSON.parse(new TextDecoder().decode(item.chunk.bytes)); const chunk_type = chunk.type; if (chunk_type === "content_block_delta") { const text = chunk.delta.text; completeMessage = completeMessage + text; process.stdout.write(text); } } // Return the final response return completeMessage; }; // Invoke the function if this file was run directly. if (process.argv[1] === fileURLToPath(import.meta.url)) { const prompt = 'Write a paragraph starting with: "Once upon a time..."'; const modelId = FoundationModels.CLAUDE_3_HAIKU.modelId; console.log(`Prompt: ${prompt}`); console.log(`Model ID: ${modelId}`); try { console.log("-".repeat(53)); const response = await invokeModel(prompt, modelId); console.log("\n" + "-".repeat(53)); console.log("Final structured response:"); console.log(response); } catch (err) { console.log(`\n${err}`); } }
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Anthropic Claude models, using the Invoke Model API, and print the response stream.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message and process the response stream in real-time.
import { fileURLToPath } from "url"; import { FoundationModels } from "../../config/foundation_models.js"; import { BedrockRuntimeClient, InvokeModelCommand, InvokeModelWithResponseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; /** * @typedef {Object} ResponseContent * @property {string} text * * @typedef {Object} MessagesResponseBody * @property {ResponseContent[]} content * * @typedef {Object} Delta * @property {string} text * * @typedef {Object} Message * @property {string} role * * @typedef {Object} Chunk * @property {string} type * @property {Delta} delta * @property {Message} message */ /** * Invokes Anthropic Claude 3 using the Messages API. * * To learn more about the Anthropic Messages API, go to: * https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-3-haiku-20240307-v1:0". */ export const invokeModel = async ( prompt, modelId = "anthropic.claude-3-haiku-20240307-v1:0", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { anthropic_version: "bedrock-2023-05-31", max_tokens: 1000, messages: [ { role: "user", content: [{ type: "text", text: prompt }], }, ], }; // Invoke Claude with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); // Decode and return the response(s) const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {MessagesResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); return responseBody.content[0].text; }; /** * Invokes Anthropic Claude 3 and processes the response stream. * * To learn more about the Anthropic Messages API, go to: * https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-3-haiku-20240307-v1:0". */ export const invokeModelWithResponseStream = async ( prompt, modelId = "anthropic.claude-3-haiku-20240307-v1:0", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Prepare the payload for the model. const payload = { anthropic_version: "bedrock-2023-05-31", max_tokens: 1000, messages: [ { role: "user", content: [{ type: "text", text: prompt }], }, ], }; // Invoke Claude with the payload and wait for the API to respond. const command = new InvokeModelWithResponseStreamCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); let completeMessage = ""; // Decode and process the response stream for await (const item of apiResponse.body) { /** @type Chunk */ const chunk = JSON.parse(new TextDecoder().decode(item.chunk.bytes)); const chunk_type = chunk.type; if (chunk_type === "content_block_delta") { const text = chunk.delta.text; completeMessage = completeMessage + text; process.stdout.write(text); } } // Return the final response return completeMessage; }; // Invoke the function if this file was run directly. if (process.argv[1] === fileURLToPath(import.meta.url)) { const prompt = 'Write a paragraph starting with: "Once upon a time..."'; const modelId = FoundationModels.CLAUDE_3_HAIKU.modelId; console.log(`Prompt: ${prompt}`); console.log(`Model ID: ${modelId}`); try { console.log("-".repeat(53)); const response = await invokeModel(prompt, modelId); console.log("\n" + "-".repeat(53)); console.log("Final structured response:"); console.log(response); } catch (err) { console.log(`\n${err}`); } }
-
For API details, see InvokeModelWithResponseStream in AWS SDK for JavaScript API Reference.
-
Cohere Command
The following code example shows how to send a text message to Cohere Command, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Cohere Command, using Bedrock's Converse API.
// Use the Conversation API to send a text message to Cohere Command. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Command R. const modelId = "cohere.command-r-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Cohere Command, using Bedrock's Converse API and process the response stream in real-time.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Cohere Command, using Bedrock's Converse API and process the response stream in real-time.
// Use the Conversation API to send a text message to Cohere Command. import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Command R. const modelId = "cohere.command-r-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseStreamCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the streamed response text in real-time. for await (const item of response.stream) { if (item.contentBlockDelta) { process.stdout.write(item.contentBlockDelta.delta?.text); } } } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see ConverseStream in AWS SDK for JavaScript API Reference.
-
Meta Llama
The following code example shows how to send a text message to Meta Llama, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Meta Llama, using Bedrock's Converse API.
// Use the Conversation API to send a text message to Meta Llama. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Llama 3 8b Instruct. const modelId = "meta.llama3-8b-instruct-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Meta Llama, using Bedrock's Converse API and process the response stream in real-time.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Meta Llama, using Bedrock's Converse API and process the response stream in real-time.
// Use the Conversation API to send a text message to Meta Llama. import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Llama 3 8b Instruct. const modelId = "meta.llama3-8b-instruct-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseStreamCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the streamed response text in real-time. for await (const item of response.stream) { if (item.contentBlockDelta) { process.stdout.write(item.contentBlockDelta.delta?.text); } } } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see ConverseStream in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Meta Llama 2, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
// Send a prompt to Meta Llama 2 and print the response. import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region of your choice. const client = new BedrockRuntimeClient({ region: "us-west-2" }); // Set the model ID, e.g., Llama 2 Chat 13B. const modelId = "meta.llama2-13b-chat-v1"; // Define the user message to send. const userMessage = "Describe the purpose of a 'hello world' program in one sentence."; // Embed the message in Llama 2's prompt format. const prompt = `<s>[INST] ${userMessage} [/INST]`; // Format the request payload using the model's native structure. const request = { prompt, // Optional inference parameters: max_gen_len: 512, temperature: 0.5, top_p: 0.9, }; // Encode and send the request. const response = await client.send( new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(request), modelId, }), ); // Decode the native response body. /** @type {{ generation: string }} */ const nativeResponse = JSON.parse(new TextDecoder().decode(response.body)); // Extract and print the generated text. const responseText = nativeResponse.generation; console.log(responseText); // Learn more about the Llama 2 prompt format at: // https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-2
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Meta Llama 3, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
// Send a prompt to Meta Llama 3 and print the response. import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region of your choice. const client = new BedrockRuntimeClient({ region: "us-west-2" }); // Set the model ID, e.g., Llama 3 70B Instruct. const modelId = "meta.llama3-70b-instruct-v1:0"; // Define the user message to send. const userMessage = "Describe the purpose of a 'hello world' program in one sentence."; // Embed the message in Llama 3's prompt format. const prompt = ` <|begin_of_text|><|start_header_id|>user<|end_header_id|> ${userMessage} <|eot_id|> <|start_header_id|>assistant<|end_header_id|> `; // Format the request payload using the model's native structure. const request = { prompt, // Optional inference parameters: max_gen_len: 512, temperature: 0.5, top_p: 0.9, }; // Encode and send the request. const response = await client.send( new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(request), modelId, }), ); // Decode the native response body. /** @type {{ generation: string }} */ const nativeResponse = JSON.parse(new TextDecoder().decode(response.body)); // Extract and print the generated text. const responseText = nativeResponse.generation; console.log(responseText); // Learn more about the Llama 3 prompt format at: // https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/#special-tokens-used-with-meta-llama-3
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Meta Llama 2, using the Invoke Model API, and print the response stream.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message and process the response stream in real-time.
// Send a prompt to Meta Llama 2 and print the response stream in real-time. import { BedrockRuntimeClient, InvokeModelWithResponseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region of your choice. const client = new BedrockRuntimeClient({ region: "us-west-2" }); // Set the model ID, e.g., Llama 2 Chat 13B. const modelId = "meta.llama2-13b-chat-v1"; // Define the user message to send. const userMessage = "Describe the purpose of a 'hello world' program in one sentence."; // Embed the message in Llama 2's prompt format. const prompt = `<s>[INST] ${userMessage} [/INST]`; // Format the request payload using the model's native structure. const request = { prompt, // Optional inference parameters: max_gen_len: 512, temperature: 0.5, top_p: 0.9, }; // Encode and send the request. const responseStream = await client.send( new InvokeModelWithResponseStreamCommand({ contentType: "application/json", body: JSON.stringify(request), modelId, }), ); // Extract and print the response stream in real-time. for await (const event of responseStream.body) { /** @type {{ generation: string }} */ const chunk = JSON.parse(new TextDecoder().decode(event.chunk.bytes)); if (chunk.generation) { process.stdout.write(chunk.generation); } } // Learn more about the Llama 3 prompt format at: // https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/#special-tokens-used-with-meta-llama-3
-
For API details, see InvokeModelWithResponseStream in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Meta Llama 3, using the Invoke Model API, and print the response stream.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message and process the response stream in real-time.
// Send a prompt to Meta Llama 3 and print the response stream in real-time. import { BedrockRuntimeClient, InvokeModelWithResponseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region of your choice. const client = new BedrockRuntimeClient({ region: "us-west-2" }); // Set the model ID, e.g., Llama 3 70B Instruct. const modelId = "meta.llama3-70b-instruct-v1:0"; // Define the user message to send. const userMessage = "Describe the purpose of a 'hello world' program in one sentence."; // Embed the message in Llama 3's prompt format. const prompt = ` <|begin_of_text|><|start_header_id|>user<|end_header_id|> ${userMessage} <|eot_id|> <|start_header_id|>assistant<|end_header_id|> `; // Format the request payload using the model's native structure. const request = { prompt, // Optional inference parameters: max_gen_len: 512, temperature: 0.5, top_p: 0.9, }; // Encode and send the request. const responseStream = await client.send( new InvokeModelWithResponseStreamCommand({ contentType: "application/json", body: JSON.stringify(request), modelId, }), ); // Extract and print the response stream in real-time. for await (const event of responseStream.body) { /** @type {{ generation: string }} */ const chunk = JSON.parse(new TextDecoder().decode(event.chunk.bytes)); if (chunk.generation) { process.stdout.write(chunk.generation); } } // Learn more about the Llama 3 prompt format at: // https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/#special-tokens-used-with-meta-llama-3
-
For API details, see InvokeModelWithResponseStream in AWS SDK for JavaScript API Reference.
-
Mistral AI
The following code example shows how to send a text message to Mistral, using Bedrock's Converse API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Mistral, using Bedrock's Converse API.
// Use the Conversation API to send a text message to Mistral. import { BedrockRuntimeClient, ConverseCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Mistral Large. const modelId = "mistral.mistral-large-2402-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the response text. const responseText = response.output.message.content[0].text; console.log(responseText); } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see Converse in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Mistral, using Bedrock's Converse API and process the response stream in real-time.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Send a text message to Mistral, using Bedrock's Converse API and process the response stream in real-time.
// Use the Conversation API to send a text message to Mistral. import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Create a Bedrock Runtime client in the AWS Region you want to use. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Set the model ID, e.g., Mistral Large. const modelId = "mistral.mistral-large-2402-v1:0"; // Start a conversation with the user message. const userMessage = "Describe the purpose of a 'hello world' program in one line."; const conversation = [ { role: "user", content: [{ text: userMessage }], }, ]; // Create a command with the model ID, the message, and a basic configuration. const command = new ConverseStreamCommand({ modelId, messages: conversation, inferenceConfig: { maxTokens: 512, temperature: 0.5, topP: 0.9 }, }); try { // Send the command to the model and wait for the response const response = await client.send(command); // Extract and print the streamed response text in real-time. for await (const item of response.stream) { if (item.contentBlockDelta) { process.stdout.write(item.contentBlockDelta.delta?.text); } } } catch (err) { console.log(`ERROR: Can't invoke '${modelId}'. Reason: ${err}`); process.exit(1); }
-
For API details, see ConverseStream in AWS SDK for JavaScript API Reference.
-
The following code example shows how to send a text message to Mistral models, using the Invoke Model API.
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the AWS Code Examples Repository
. Use the Invoke Model API to send a text message.
import { fileURLToPath } from "url"; import { FoundationModels } from "../../config/foundation_models.js"; import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; /** * @typedef {Object} Output * @property {string} text * * @typedef {Object} ResponseBody * @property {Output[]} outputs */ /** * Invokes a Mistral 7B Instruct model. * * @param {string} prompt - The input text prompt for the model to complete. * @param {string} [modelId] - The ID of the model to use. Defaults to "mistral.mistral-7b-instruct-v0:2". */ export const invokeModel = async ( prompt, modelId = "mistral.mistral-7b-instruct-v0:2", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "us-east-1" }); // Mistral instruct models provide optimal results when embedding // the prompt into the following template: const instruction = `<s>[INST] ${prompt} [/INST]`; // Prepare the payload. const payload = { prompt: instruction, max_tokens: 500, temperature: 0.5, }; // Invoke the model with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command); // Decode and return the response. const decodedResponseBody = new TextDecoder().decode(apiResponse.body); /** @type {ResponseBody} */ const responseBody = JSON.parse(decodedResponseBody); return responseBody.outputs[0].text; }; // Invoke the function if this file was run directly. if (process.argv[1] === fileURLToPath(import.meta.url)) { const prompt = 'Complete the following in one sentence: "Once upon a time..."'; const modelId = FoundationModels.MISTRAL_7B.modelId; console.log(`Prompt: ${prompt}`); console.log(`Model ID: ${modelId}`); try { console.log("-".repeat(53)); const response = await invokeModel(prompt, modelId); console.log(response); } catch (err) { console.log(err); } }
-
For API details, see InvokeModel in AWS SDK for JavaScript API Reference.
-