Track agent's step-by-step reasoning process using trace

Each response from an Amazon Bedrock agent is accompanied by a trace that details the steps being orchestrated by the agent. The trace helps you follow the agent's reasoning process that leads it to the response it gives at that point in the conversation.

Use the trace to track the agent's path from the user input to the response it returns. The trace provides information about the inputs to the action groups that the agent invokes and the knowledge bases that it queries to respond to the user. In addition, the trace provides information about the outputs that the action groups and knowledge bases return. You can view the reasoning that the agent uses to determine the action that it takes or the query that it makes to a knowledge base. If a step in the trace fails, the trace returns a reason for the failure. Use the detailed information in the trace to troubleshoot your agent. You can identify steps at which the agent has trouble or at which it yields unexpected behavior. Then, you can use this information to consider ways in which you can improve the agent's behavior.

Structure of the trace

The trace is shown as a JSON object in both the console and the API. Each Step in the console or Trace in the API can be one of the following traces:

• PreProcessingTrace – Traces the input and output of the pre-processing step, in which the agent contextualizes and categorizes user input and determines if it is valid.

• OrchestrationTrace – Traces the input and output of the orchestration step, in which the agent interprets the input, invokes action groups, and queries knowledge bases. Then the agent returns output to either continue orchestration or to respond to the user.

• PostProcessingTrace – Traces the input and output of the post-processing step, in which the agent handles the final output of the orchestration and determines how to return the response to the user.

• FailureTrace – Traces the reason that a step failed.

• GuardrailTrace – Traces the actions of the Guardrail.

Each of the traces (except FailureTrace) contains a ModelInvocationInput object. The ModelInvocationInput object contains configurations set in the prompt template for the step, alongside the prompt provided to the agent at this step. For more information about how to modify prompt templates, see Enhance agent's accuracy using advanced prompt templates in Amazon Bedrock. The structure of the ModelInvocationInput object is as follows:

{
    "traceId": "string",
    "text": "string",
    "type": "PRE_PROCESSING | ORCHESTRATION | KNOWLEDGE_BASE_RESPONSE_GENERATION | POST_PROCESSING",
    "inferenceConfiguration": {
        "maximumLength": number,
        "stopSequences": ["string"],
        "temperature": float,
        "topK": float,
        "topP": float
    },
    "promptCreationMode": "DEFAULT | OVERRIDDEN",
    "parserMode": "DEFAULT | OVERRIDDEN",
    "overrideLambda": "string"
}

The following list describes the fields of the ModelInvocationInput object:

• traceId – The unique identifier of the trace.

• text – The text from the prompt provided to the agent at this step.

• type – The current step in the agent's process.

• inferenceConfiguration – Inference parameters that influence response generation. For more information, see Influence response generation with inference parameters.

• promptCreationMode – Whether the agent's default base prompt template was overridden for this step. For more information, see Enhance agent's accuracy using advanced prompt templates in Amazon Bedrock.

• parserMode – Whether the agent's default response parser was overridden for this step. For more information, see Enhance agent's accuracy using advanced prompt templates in Amazon Bedrock.

• overrideLambda – The Amazon Resource Name (ARN) of the parser Lambda function used to parse the response, if the default parser was overridden. For more information, see Enhance agent's accuracy using advanced prompt templates in Amazon Bedrock.

For more information about each trace type, see the following sections:

PreProcessingTrace

{
    "modelInvocationInput": { // see above for details }
    "modelInvocationOutput": {
        "metadata": {
             "usage": {
                  "inputToken":: int,
                  "outputToken":: int
           },
         "rawResponse": {
              "content": "string"
          }
        "parsedResponse": {
            "isValid": boolean,
            "rationale": "string"
        },
        "traceId": "string"
    }
}

The PreProcessingTrace consists of a ModelInvocationInput object and a PreProcessingModelInvocationOutput object. The PreProcessingModelInvocationOutput contains the following fields.

• metadata – Contains the following information about the foundation model output.

  • usage – Contains the following information of the usage of the foundation model.

    • inputTokens – Contains the information about the input tokens from the foundation model usage.

    • outputTokens – Contains the information about the output tokens from the foundation model usage.

• rawResponse – Contains the raw output from the foundation model.

  • content – The foundation model's raw output content.

• parsedResponse – Contains the following details about the parsed user prompt.

  • isValid – Specifies whether the user prompt is valid.

  • rationale – Specifies the agent's reasoning for the next steps to take.

• traceId – The unique identifier of the trace.

OrchestrationTrace

The OrchestrationTrace consists of the ModelInvocationInput object, OrchestrationModelInvocationOutput object, and any combination of the Rationale, InvocationInput, and Observation objects. The OrchestrationModelInvocationOutput contains the following fields. For more information about Rationale, InvocationInput, and Observation objects, select from the following tabs.

{
    "modelInvocationInput": { // see above for details },
     "modelInvocationOutput": {
        "metadata": {
             "usage": {
                  "inputToken":: int,
                  "outputToken":: int
           },
         "rawResponse": {
              "content": "string"
          },
    "rationale": { ... },
    "invocationInput": { ... },
    "observation": { ... }
}

Rationale

The Rationale object contains the reasoning of the agent given the user input. Following is the structure:

{
       "traceId": "string",
       "text": "string"
    }

The following list describes the fields of the Rationale object:

• traceId – The unique identifier of the trace step.

• text – The reasoning process of the agent, based on the input prompt.

InvocationInput

The InvocationInput object contains information that will be input to the action group or knowledge base that is to be invoked or queried. Following is the structure:

{
    "traceId": "string",
    "invocationType": "ACTION_GROUP | KNOWLEDGE_BASE | FINISH",
    "actionGroupInvocationInput": {
        // see below for details
    },
    "knowledgeBaseLookupInput": {
        "knowledgeBaseId": "string",
        "text": "string"
    }
}

The following list describes the fields of the InvocationInput object:

• traceId – The unique identifier of the trace.

• invocationType – Specifies whether the agent is invoking an action group or a knowledge base, or ending the session.

• actionGroupInvocationInput – Appears if the type is ACTION_GROUP. For more information, see Define actions in the action group. Can be one of the following structures:

  • If the action group is defined by an API schema, the structure is as follows:

    {
        "actionGroupName": "string",
        "apiPath": "string",
        "verb": "string",
        "parameters": [
            {
                "name": "string",
                "type": "string",
                "value": "string"
            },
            ...
        ],
        "requestBody": {
            "content": {
                "<content-type>": [
                    {
                        "name": "string",
                        "type": "string",
                        "value": "string"
                    }   
                ]
            }
        },
        "executionType": "LAMBDA | RETURN_CONTROL",
        "invocationId": "string"
    }

    Following are descriptions of the fields:

    • actionGroupName – The name of the action group that the agent predicts should be invoked.

    • apiPath – The path to the API operation to call, according to the API schema.

    • verb – The API method being used, according to the API schema.

    • parameters – Contains a list of objects. Each object contains the name, type, and value of a parameter in the API operation, as defined in the API schema.

    • requestBody – Contains the request body and its properties, as defined in the API schema.

    • executionType – Whether fulfillment of the action is passed to a Lambda function (LAMBDA) or control is returned through the InvokeAgent response (RETURN_CONTROL). For more information, see Handle fulfillment of the action.

    • invocationId – The unique identifier of the invocation. Only returned if the executionType is RETURN_CONTROL.

  • If the action group is defined by function details, the structure is as follows:

    {
        "actionGroupName": "string",
        "function": "string",
        "parameters": [
            {
                "name": "string",
                "type": "string",
                "value": "string"
            },
            ...
        ],
        "executionType": "LAMBDA | RETURN_CONTROL",
        "invocationId": "string"
    }

    Following are descriptions of the fields:

    • actionGroupName – The name of the action group that the agent predicts should be invoked.

    • function – The name of the function that the agent predicts should be called.

    • parameters – The parameters of the function.

    • executionType – Whether fulfillment of the action is passed to a Lambda function (LAMBDA) or control is returned through the InvokeAgent response (RETURN_CONTROL). For more information, see Handle fulfillment of the action.

    • invocationId – The unique identifier of the invocation. Only returned if the executionType is RETURN_CONTROL.

• knowledgeBaseLookupInput – Appears if the type is KNOWLEDGE_BASE. For more information, see Retrieve data and generate AI responses with knowledge bases. Contains the following information about the knowledge base and the search query for the knowledge base:

  • knowledgeBaseId – The unique identifier of the knowledge base that the agent will look up.

  • text – The query to be made to the knowledge base.

Observation

The Observation object contains the result or output of an action group or knowledge base, or the response to the user. Following is the structure:

{
    "traceId": "string",
    "type": "ACTION_GROUP | KNOWLEDGE_BASE | REPROMPT | ASK_USER | FINISH",
    "actionGroupInvocation": {
        "text": "JSON-formatted string"
    },
    "knowledgeBaseLookupOutput": {
        "retrievedReferences": [
            {
                "content": {
                    "text": "string"
                },
                "location": {
                    "type": "S3",
                    "s3Location": {
                        "uri": "string"
                    }
                }
            },
            ...
        ]
    },
    "repromptResponse": {
        "source": "ACTION_GROUP | KNOWLEDGE_BASE | PARSER",
        "text": "string"
    },
    "finalResponse": {
        "text"
    }
}

The following list describes the fields of the Observation object:

• traceId – The unique identifier of the trace.

• type – Specifies whether the agent's observation is returned from the result of an action group or a knowledge base, if the agent is reprompting the user, requesting more information, or ending the conversation.

• actionGroupInvocationOutput – Contains the JSON-formatted string returned by the API operation that was invoked by the action group. Appears if the type is ACTION_GROUP. For more information, see Define OpenAPI schemas for your agent's action groups in Amazon Bedrock.

• knowledgeBaseLookupOutput – Contains text retrieved from the knowledge base that is relevant to responding to the prompt, alongside the Amazon S3 location of the data source. Appears if the type is KNOWLEDGE_BASE. For more information, see Retrieve data and generate AI responses with knowledge bases. Each object in the list of retrievedReferences contains the following fields:

  • content – Contains text from the knowledge base that is returned from the knowledge base query.

  • location – Contains the Amazon S3 URI of the data source from which the returned text was found.

• repromptResponse – Appears if the type is REPROMPT. Contains the text that asks for a prompt again alongside the source of why the agent needs to reprompt.

• finalResponse – Appears if the type is ASK_USER or FINISH. Contains the text that asks the user for more information or is a response to the user.

PostProcessingTrace

{
    "modelInvocationInput": { // see above for details }
    "modelInvocationOutput": {
        "rawResponse": {
            "content": "string"
        },
        "metadata": {
            "usage": {
                "inputToken": int,
                "outputToken": int    
             }     
         },
        "parsedResponse": {
            "text": "string"
        },
        "traceId": "string"
    }
}

The PostProcessingTrace consists of a ModelInvocationInput object and a PostProcessingModelInvocationOutput object. The PostProcessingModelInvocationOutput contains the following fields:

• rawResponse – Contains the raw output from the foundation model.

  • content – The foundation model's raw output content.

• metadata – Contains the following information about the foundation model output.

  • usage – Contains the following information of the usage of the foundation model.

    • inputTokens – Contains the information about the input tokens from the foundation model usage.

    • outputTokens – Contains the information about the output tokens from the foundation model usage.

• parsedResponse – Contains the text to return to the user after the text is processed by the parser function.

• traceId – The unique identifier of the trace.

FailureTrace

{
    "failureReason": "string",
    "traceId": "string"
}

The following list describes the fields of the FailureTrace object:

• failureReason – The reason that the step failed.

• traceId – The unique identifier of the trace.

GuardrailTrace

{
    "action": "GUARDRAIL_INTERVENED" | "NONE",
    "inputAssessments": [GuardrailAssessment],
    "outputAssessments": [GuardrailAssessment]
}

The following list describes the fields of the GuardrailAssessment object:

• action – indicates whether guardrails intervened or not on the input data. Options are GUARDRAIL_INTERVENED or NONE.

• inputAssessments – The details of the Guardrail assessment on the user input.

• outputAssessments – The details of the Guardrail assessment on the response.

For more details on the GuardrailAssessment object and testing a Guardrail, see Test a guardrail.

GuardrailAssessment example:

{
    "topicPolicy": {
        "topics": [{
            "name": "string",
            "type": "string",
            "action": "string"
        }]
    },
    "contentPolicy": {
        "filters": [{
            "type": "string",
            "confidence": "string",
            "action": "string"
        }]
    },
    "wordPolicy": {
        "customWords": [{
            "match": "string",
            "action": "string"
        }],
        "managedWordLists": [{
            "match": "string",
            "type": "string",
            "action": "string"
        }]
    },
    "sensitiveInformationPolicy": {
        "piiEntities": [{
            "type": "string",
            "match": "string",
            "action": "string"
        }],
        "regexes": [{
            "name": "string",
            "regex": "string",
            "match": "string",
            "action": "string"
        }]
    }
}