Python agent support TypeScript agent support Best practices for LangGraph agents

Set up LangGraph telemetry for AgentCore Evaluations

This page explains how to instrument a LangGraph agent, how spans are identified, and how evaluation fields are extracted. AgentCore Evaluations supports LangGraph agents built in Python and TypeScript; this page covers each language separately, in Python agent support and TypeScript agent support. It closes with best practices for structuring a LangGraph agent so that it can be evaluated reliably.

Topics

Python agent support
TypeScript agent support
Best practices for LangGraph agents

Python agent support

A Python LangGraph agent emits spans under the scope name opentelemetry.instrumentation.langchain (OpenTelemetry) or openinference.instrumentation.langchain (OpenInference).

Instrument your agent

You can instrument a LangGraph agent with either of two instrumentation libraries: OpenTelemetry (opentelemetry-instrumentation-langchain) or OpenInference (openinference-instrumentation-langchain). Amazon Bedrock AgentCore Evaluations supports both libraries. The libraries emit different scope names and use different span attributes. The evaluation service extracts the same values from each.

When your agent runs with the AWS Distro for OpenTelemetry (ADOT), such as on Amazon Bedrock AgentCore Runtime, you do not need to add explicit instrumentation code. Adding the instrumentation library to your project’s dependencies is enough. ADOT discovers it at startup and activates it automatically.

Add the instrumentation library for the path you want to your dependencies. The following examples pin a minimum version; use the latest available version unless you have a reason to pin.

Example

Note

Instrumentation is one step in setting up observability. To export telemetry for evaluation, complete the full setup in Set up observability.

How spans are identified

The attribute used to classify spans differs between the two instrumentation libraries.

Example

OpenTelemetry

The OpenTelemetry instrumentation library classifies spans using the traceloop.span.kind attribute, and recent versions also set gen_ai.operation.name.

Span type	Identifying attribute
Invoke agent	`traceloop.span.kind` = `workflow` (also `gen_ai.operation.name` = `invoke_agent`)
Execute tool	`traceloop.span.kind` = `tool` (also `gen_ai.operation.name` = `execute_tool`)
Inference	`gen_ai.operation.name` = `chat`

OpenInference

The OpenInference instrumentation library classifies spans using the openinference.span.kind attribute.

Span type	Identifying attribute
Invoke agent	`openinference.span.kind` = `CHAIN` or `AGENT`
Execute tool	`openinference.span.kind` = `TOOL`
Inference	`openinference.span.kind` = `LLM`

How evaluation fields are extracted

For the invoke agent span, the input and output do not contain a clean per-message list. Instead, the content is the serialized LangChain graph state: a JSON string that wraps the full state. The exact shape of this serialized state differs between the two instrumentation libraries. In both cases, the service parses it to find the user prompt (the human message) and the agent response (the AI message).

LangGraph also serializes message roles in more than one form. A role can appear as a lowercase value (human, ai, tool) or as a LangChain message class name (HumanMessage, AIMessage, ToolMessage). The service recognizes both forms.

The location of this content depends on how telemetry was collected. The identifying attribute (traceloop.span.kind or openinference.span.kind) is on the span in both cases. For more information, see Telemetry setup and delivery.

From event records

With split telemetry, the service reads content from the event record correlated to each span:

User prompt and agent response: from the invoke agent span’s event record, in body.input and body.output.
Tool call: the tool name from the execute tool span. The tool arguments and result come from that span’s event record, in body.input and body.output.

For more information, see Example spans in split telemetry.

From span attributes

With unified telemetry, the same content stays on the span as attributes. The attributes depend on the instrumentation library:

OpenTelemetry:
- User prompt and agent response: from gen_ai.task.input and gen_ai.task.output on the invoke agent span.
- Tool call: the tool name from gen_ai.tool.name, and the arguments and result from gen_ai.tool.call.arguments and gen_ai.tool.call.result, on the execute tool span.
OpenInference:
- User prompt and agent response: from input.value and output.value on the invoke agent span.
- Tool call: the tool name from tool.name, and the arguments and result from input.value and output.value, on the execute tool span.

For more information, see Example spans in unified telemetry.

Example spans in split telemetry

With split telemetry, the span carries the identifying attributes and the content lives in a correlated event record. The following examples are from a Python LangGraph travel-planning agent deployed on Amazon Bedrock AgentCore Runtime. The same agent is shown under each instrumentation library.

Note

These examples are not complete spans. They show representative data from a real agent interaction, with some fields omitted and long values truncated for readability.

OpenTelemetry

Example

Invoke agent span

The traceloop.span.kind attribute (workflow) identifies this as an invoke agent span; recent library versions also set gen_ai.operation.name = invoke_agent.


{
  "traceId": "6a01eef11066751d68f90def0da1f80a",
  "spanId": "ba1833fa7f097041",
  "parentSpanId": "836a5ccf9a2186cc",
  "name": "travel_agent.workflow",
  "kind": "INTERNAL",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain",
    "version": "0.60.0"
  },
  "startTimeUnixNano": 1778511607308521744,
  "endTimeUnixNano": 1778511610930280395,
  "durationNano": 3621758651,
  "attributes": {
    "traceloop.span.kind": "workflow",
    "gen_ai.operation.name": "invoke_agent",
    "gen_ai.agent.name": "travel_agent",
    "gen_ai.provider.name": "langgraph",
    "traceloop.workflow.name": "travel_agent",
    "session.id": "sea-nyc-trip-2-turns-adot_v17_opentelemetry_0_60_0"
  },
  "status": {
    "code": "OK"
  }
}

The correlated event record carries the conversation. Each message’s content is the serialized LangChain graph state. The input wraps the state under an inputs key. The output wraps it under an outputs key, with each message as a LangChain constructor object. The user prompt is the human message and the agent response is the AI message inside that serialized state.


{
  "spanId": "ba1833fa7f097041",
  "traceId": "6a01eef11066751d68f90def0da1f80a",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain"
  },
  "body": {
    "input": {
      "messages": [
        {
          "content": "{\"inputs\": {\"messages\": [{\"role\": \"user\", \"content\": \"Hey, how can you help me\"}]}, \"tags\": [], \"metadata\": {\"ls_integration\": \"langchain_create_agent\", \"lc_agent_name\": \"travel_agent\", \"thread_id\": \"sea-nyc-trip-2-turns-adot_v17_opentelemetry_0_60_0\"}, \"kwargs\": {\"name\": \"travel_agent\"}}",
          "role": "user"
        }
      ]
    },
    "output": {
      "messages": [
        {
          "content": "{\"outputs\": {\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain\", \"schema\", \"messages\", \"HumanMessage\"], \"kwargs\": {\"content\": \"Hey, how can you help me\", \"type\": \"human\", \"id\": \"12345678-1234-1234-1234-123456789012\"}}, {\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain\", \"schema\", \"messages\", \"AIMessage\"], \"kwargs\": {\"content\": \"Hello! I'm your travel planning assistant ...\", \"type\": \"ai\"}}]}, \"kwargs\": {\"tags\": []}}",
          "role": "assistant"
        }
      ]
    }
  }
}

Execute tool span

The traceloop.span.kind attribute (tool) identifies this as an execute tool span; gen_ai.tool.name holds the tool name and gen_ai.operation.name = execute_tool.


{
  "traceId": "6a01eefa5c52f3d86a35038f35f5ba30",
  "spanId": "5b332f3cd15ace04",
  "parentSpanId": "922a21edc04eba29",
  "name": "execute_tool search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain",
    "version": "0.60.0"
  },
  "startTimeUnixNano": 1778511614892698232,
  "endTimeUnixNano": 1778511614893399618,
  "durationNano": 701386,
  "attributes": {
    "traceloop.span.kind": "tool",
    "gen_ai.operation.name": "execute_tool",
    "gen_ai.tool.name": "search_flights",
    "gen_ai.tool.type": "function",
    "gen_ai.tool.description": "Search for available flights between cities.",
    "gen_ai.provider.name": "langgraph",
    "traceloop.workflow.name": "travel_agent",
    "session.id": "sea-nyc-trip-2-turns-adot_v17_opentelemetry_0_60_0"
  },
  "status": {
    "code": "OK"
  }
}

The correlated event record carries the tool input (arguments) and output (result, serialized as a LangChain ToolMessage).


{
  "spanId": "5b332f3cd15ace04",
  "traceId": "6a01eefa5c52f3d86a35038f35f5ba30",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain"
  },
  "body": {
    "input": {
      "messages": [
        { "content": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}" }
      ]
    },
    "output": {
      "messages": [
        {
          "role": "tool",
          "name": "search_flights",
          "content": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"flights\": [ ... ]}"
        }
      ]
    }
  }
}

OpenInference

With the OpenInference library, the span type is carried in the openinference.span.kind attribute, and the agent input and output are serialized in the correlated event record.

Example

Invoke agent span

The openinference.span.kind attribute (CHAIN, or AGENT when the graph is compiled with a name) identifies this as an invoke agent span.


{
  "traceId": "6a387ee61078243c1cc455ed45c6c313",
  "spanId": "0a7990d804132a9b",
  "parentSpanId": "29ae22014173881c",
  "name": "LangGraph",
  "kind": "INTERNAL",
  "scope": {
    "name": "openinference.instrumentation.langchain",
    "version": "0.1.66"
  },
  "startTimeUnixNano": 1782087405949310976,
  "endTimeUnixNano": 1782087408945828864,
  "durationNano": 2996517888,
  "attributes": {
    "openinference.span.kind": "CHAIN",
    "input.mime_type": "application/json",
    "output.mime_type": "application/json",
    "llm.input_messages.0.message.role": "user",
    "session.id": "sea-nyc-trip-2-turns-oi-0-1-66"
  },
  "status": {
    "code": "OK"
  }
}

The correlated event record carries the conversation. The user prompt is the human-role message and the agent response is the AI-role message in the serialized messages.


{
  "spanId": "0a7990d804132a9b",
  "traceId": "6a387ee61078243c1cc455ed45c6c313",
  "scope": {
    "name": "openinference.instrumentation.langchain"
  },
  "body": {
    "input": {
      "messages": [
        {
          "role": "user",
          "content": "{\"messages\": [{\"role\": \"user\", \"content\": \"Hey, how can you help me\"}]}"
        }
      ]
    },
    "output": {
      "messages": [
        {
          "content": "{\"messages\": [{\"type\": \"human\", \"data\": {\"content\": \"Hey, how can you help me\", ...}}, {\"type\": \"ai\", \"data\": {\"content\": \"Hello! I'm your travel planning assistant ...\", ...}}]}",
          "role": "assistant"
        }
      ]
    }
  }
}

Execute tool span

The openinference.span.kind attribute (TOOL) identifies this as an execute tool span; tool.name holds the tool name.


{
  "traceId": "6a387ef07b8f4f3732fab45d3c0b51ff",
  "spanId": "ab105c12cc40048f",
  "parentSpanId": "9b2d4e72760690b4",
  "name": "search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "openinference.instrumentation.langchain",
    "version": "0.1.66"
  },
  "startTimeUnixNano": 1782087411724620032,
  "endTimeUnixNano": 1782087411725306880,
  "durationNano": 686848,
  "attributes": {
    "openinference.span.kind": "TOOL",
    "tool.name": "search_flights",
    "tool.description": "Search for available flights between cities.",
    "input.mime_type": "application/json",
    "output.mime_type": "application/json",
    "session.id": "sea-nyc-trip-2-turns-oi-0-1-66"
  },
  "status": {
    "code": "OK"
  }
}

The correlated event record carries the tool input (arguments) and output (result, serialized as a LangChain ToolMessage).


{
  "spanId": "ab105c12cc40048f",
  "traceId": "6a387ef07b8f4f3732fab45d3c0b51ff",
  "scope": {
    "name": "openinference.instrumentation.langchain"
  },
  "body": {
    "input": {
      "messages": [
        {
          "role": "user",
          "content": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}"
        }
      ]
    },
    "output": {
      "messages": [
        {
          "content": "{\"type\": \"tool\", \"data\": {\"content\": \"{\\\"origin\\\": \\\"SEA\\\", \\\"destination\\\": \\\"NYC\\\", \\\"flights\\\": [ ... ]}\", \"type\": \"tool\", \"name\": \"search_flights\", \"tool_call_id\": \"toolu_bdrk_01LzXXJCfpfuS7Bpf7e1qLMg\", \"status\": \"success\"}}",
          "role": "assistant"
        }
      ]
    }
  }
}

Example spans in unified telemetry

With unified telemetry, the same content stays on the span attributes and no separate event record is produced. The following examples are from a Python LangGraph travel-planning agent. The same agent is shown under each instrumentation library.

Note

These examples are not complete spans. They show representative data from a real agent interaction, with some fields omitted and long values truncated for readability.

OpenTelemetry

Example

Invoke agent span

The gen_ai.task.input attribute holds the user prompt, and the gen_ai.task.output attribute holds the serialized state with the agent response. Both are the serialized LangChain graph state.


{
  "traceId": "6a4de7b85e61747e6b568a1f4768e89d",
  "spanId": "31ea3d5882dac680",
  "name": "LangGraph.workflow",
  "kind": "INTERNAL",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain",
    "version": "0.62.1"
  },
  "attributes": {
    "traceloop.span.kind": "workflow",
    "gen_ai.operation.name": "invoke_agent",
    "gen_ai.agent.name": "LangGraph",
    "gen_ai.task.input": "{\"inputs\": {\"messages\": [{\"role\": \"user\", \"content\": \"Hey, how can you help me\"}]}, \"tags\": [], \"metadata\": { ... }, \"kwargs\": {\"name\": \"LangGraph\"}}",
    "gen_ai.task.output": "{\"outputs\": {\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain\", \"schema\", \"messages\", \"HumanMessage\"], \"kwargs\": {\"content\": \"Hey, how can you help me\", \"type\": \"human\"}}, {\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain\", \"schema\", \"messages\", \"AIMessage\"], \"kwargs\": {\"content\": \"Hello! I'm your travel planning assistant ...\", \"type\": \"ai\"}}]}, \"kwargs\": {\"tags\": []}}",
    "session.id": "sea-nyc-trip-2-turns-unified"
  },
  "status": {
    "code": "OK"
  }
}

Execute tool span

The gen_ai.tool.call.arguments attribute holds the tool arguments, and the gen_ai.tool.call.result attribute holds the tool result, serialized as a LangChain ToolMessage.


{
  "traceId": "6a4de7c376913db82e6f0f336a16731d",
  "spanId": "b64c37adefae74f0",
  "name": "execute_tool search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "opentelemetry.instrumentation.langchain",
    "version": "0.62.1"
  },
  "attributes": {
    "traceloop.span.kind": "tool",
    "gen_ai.operation.name": "execute_tool",
    "gen_ai.tool.name": "search_flights",
    "gen_ai.tool.description": "Search for available flights between cities.",
    "gen_ai.tool.call.arguments": "{\"input_str\": \"{'origin': 'SEA', 'destination': 'NYC', 'date': '2025-03-15'}\", \"inputs\": {\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}, \"metadata\": { ... }}",
    "gen_ai.tool.call.result": "{\"output\": {\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain\", \"schema\", \"messages\", \"ToolMessage\"], \"kwargs\": {\"content\": \"{\\\"origin\\\": \\\"SEA\\\", \\\"destination\\\": \\\"NYC\\\", \\\"flights\\\": [ ... ]}\", \"type\": \"tool\", \"name\": \"search_flights\", \"status\": \"success\"}}}",
    "session.id": "sea-nyc-trip-2-turns-unified"
  },
  "status": {
    "code": "OK"
  }
}

OpenInference

Example

TypeScript agent support

A TypeScript LangGraph agent emits the same span types as a Python agent, so the evaluation service reads it the same way. There are three TypeScript instrumentation libraries, each with its own scope name and span-classification convention.

Instrument your agent

Add the instrumentation library for the path you want to your TypeScript dependencies. The following examples pin a minimum version; use the latest available version unless you have a reason to pin.

Example

Note

Instrumentation is one step in setting up observability. To export telemetry for evaluation, complete the full setup in Set up observability.

How spans are identified

Span identification depends on the instrumentation library:

ADOT (OpenTelemetry): the AWS Distro Node autoinstrumentation package (@aws/aws-distro-opentelemetry-node-autoinstrumentation), which emits the scope name @aws/aws-distro-opentelemetry-instrumentation-langchain, sets gen_ai.operation.name (invoke_agent, execute_tool, chat), the same as the other ADOT-native frameworks.
Traceloop (OpenTelemetry): the OpenTelemetry JS library from Traceloop (@traceloop/instrumentation-langchain) sets traceloop.span.kind (workflow for the invoke agent span, task for the tool span), matching the Python OpenTelemetry library. See How spans are identified under Python agent support.
OpenInference: the OpenInference JS library (@arizeai/openinference-instrumentation-langchain) sets openinference.span.kind (CHAIN or AGENT, TOOL, LLM), the same as the Python OpenInference library.

How evaluation fields are extracted

Field extraction depends on the instrumentation library:

ADOT (OpenTelemetry): the invoke agent span is a structural container, and the conversation content lives on the inference (chat) span, in the parts-format gen_ai.input.messages and gen_ai.output.messages attributes.
Traceloop (OpenTelemetry): with the OpenTelemetry JS library from Traceloop, the conversation is in the traceloop.entity.input and traceloop.entity.output attributes, as serialized LangChain state. This matches the Python OpenTelemetry library; see How evaluation fields are extracted under Python agent support.
OpenInference: with the OpenInference JS library, the conversation is in the input.value and output.value attributes, and inference messages also appear on the indexed llm.input_messages.* and llm.output_messages.* attributes. This matches the Python OpenInference library.

Example spans from a TypeScript agent

The following examples are from a TypeScript LangGraph travel-planning agent deployed on Amazon Bedrock AgentCore Runtime with unified telemetry. The same agent is shown under each of the three TypeScript instrumentation libraries.

Note

These examples are not complete spans. They show representative data from a real agent interaction, with some fields omitted and long values truncated for readability.

OpenTelemetry (ADOT native)

With the ADOT-native library (from the AWS Distro Node autoinstrumentation package @aws/aws-distro-opentelemetry-node-autoinstrumentation, emitting the scope name @aws/aws-distro-opentelemetry-instrumentation-langchain), the invoke agent span is a structural container and the conversation content lives on the inference (chat) span, in the parts-format gen_ai.input.messages and gen_ai.output.messages attributes.

Example

Invoke agent span

The gen_ai.operation.name attribute (invoke_agent) identifies this as an invoke agent span. The span carries the agent name and model but no conversation content.


{
  "traceId": "6a6bd0a1c8d91ed1e70a3906b551618",
  "spanId": "ba1833fa7f097041",
  "name": "invoke_agent LangGraph",
  "kind": "INTERNAL",
  "scope": {
    "name": "@aws/aws-distro-opentelemetry-instrumentation-langchain",
    "version": "0.12.0"
  },
  "attributes": {
    "gen_ai.operation.name": "invoke_agent",
    "gen_ai.agent.name": "LangGraph",
    "gen_ai.provider.name": "openai",
    "gen_ai.request.model": "gpt-4o-mini",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Execute tool span

The gen_ai.operation.name attribute (execute_tool) identifies this as an execute tool span; gen_ai.tool.name holds the tool name. The gen_ai.tool.call.arguments and gen_ai.tool.call.result attributes hold the tool arguments and result.


{
  "traceId": "6a6bd0a25c52f3d86a35038f35f5ba30",
  "spanId": "5b332f3cd15ace04",
  "name": "execute_tool search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "@aws/aws-distro-opentelemetry-instrumentation-langchain",
    "version": "0.12.0"
  },
  "attributes": {
    "gen_ai.operation.name": "execute_tool",
    "gen_ai.tool.name": "search_flights",
    "gen_ai.tool.type": "function",
    "gen_ai.tool.call.arguments": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}",
    "gen_ai.tool.call.result": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"flights\": [ ... ]}",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Inference span

The gen_ai.operation.name attribute (chat) identifies this as an inference span. The gen_ai.input.messages and gen_ai.output.messages attributes hold the conversation in the parts-format, and gen_ai.system_instructions holds the system prompt.


{
  "traceId": "6a6bd0a1c8d91ed1e70a3906b551618",
  "spanId": "7c1f9a2b4d6e8a03",
  "name": "chat gpt-4o-mini",
  "kind": "INTERNAL",
  "scope": {
    "name": "@aws/aws-distro-opentelemetry-instrumentation-langchain",
    "version": "0.12.0"
  },
  "attributes": {
    "gen_ai.operation.name": "chat",
    "gen_ai.provider.name": "openai",
    "gen_ai.request.model": "gpt-4o-mini",
    "gen_ai.input.messages": "[{\"role\": \"user\", \"parts\": [{\"type\": \"text\", \"content\": \"Hey, how can you help me\"}]}]",
    "gen_ai.output.messages": "[{\"role\": \"assistant\", \"parts\": [{\"type\": \"text\", \"content\": \"I can assist you with planning your trip ...\"}]}]",
    "gen_ai.system_instructions": "[{\"type\": \"text\", \"content\": \"You are a travel planning assistant ...\"}]",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

OpenTelemetry (Traceloop)

With the OpenTelemetry JS library from Traceloop (@traceloop/instrumentation-langchain), the span type is carried in the traceloop.span.kind attribute (workflow for the invoke agent span, task for the tool span), and gen_ai.operation.name = workflow on the invoke agent span. The conversation is in the traceloop.entity.input and traceloop.entity.output attributes, as serialized LangChain state.

Example

Invoke agent span

The traceloop.span.kind attribute (workflow) identifies this as an invoke agent span. The traceloop.entity.input and traceloop.entity.output attributes hold the serialized LangChain state, from which the user prompt (human message) and agent response (AI message) are parsed.


{
  "traceId": "6a6bd0b1c8d91ed1e70a3906b551618",
  "spanId": "ba1833fa7f097041",
  "name": "workflow RunnableSequence",
  "kind": "INTERNAL",
  "scope": {
    "name": "@traceloop/instrumentation-langchain",
    "version": "0.27.0"
  },
  "attributes": {
    "traceloop.span.kind": "workflow",
    "gen_ai.operation.name": "workflow",
    "gen_ai.provider.name": "langchain",
    "traceloop.workflow.name": "RunnableSequence",
    "traceloop.entity.input": "{\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"HumanMessage\"], \"kwargs\": {\"content\": \"Hey, how can you help me\"}}]}",
    "traceloop.entity.output": "{\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"AIMessage\"], \"kwargs\": {\"content\": \"I can assist you with planning your trip ...\"}}]}",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Execute tool span

The traceloop.span.kind attribute (task) identifies this as an execute tool span. The traceloop.entity.input and traceloop.entity.output attributes hold the tool arguments and result.


{
  "traceId": "6a6bd0b25c52f3d86a35038f35f5ba30",
  "spanId": "5b332f3cd15ace04",
  "name": "task search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "@traceloop/instrumentation-langchain",
    "version": "0.27.0"
  },
  "attributes": {
    "traceloop.span.kind": "task",
    "traceloop.entity.name": "search_flights",
    "traceloop.entity.input": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}",
    "traceloop.entity.output": "{\"output\": {\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"ToolMessage\"], \"kwargs\": {\"status\": \"success\", \"content\": \"{\\\"origin\\\": \\\"SEA\\\", \\\"destination\\\": \\\"NYC\\\", \\\"flights\\\": [ ... ]}\"}}}",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

OpenInference

With the OpenInference JS library (@arizeai/openinference-instrumentation-langchain), the span type is carried in the openinference.span.kind attribute. The conversation content is in the input.value and output.value attributes, and inference messages also appear on the indexed llm.input_messages.* and llm.output_messages.* attributes.

Example

Invoke agent span

The openinference.span.kind attribute (CHAIN) identifies this as an invoke agent span. The input.value and output.value attributes hold the serialized LangChain state.


{
  "traceId": "6a6bd0c1c8d91ed1e70a3906b551618",
  "spanId": "0a7990d804132a9b",
  "name": "LangGraph",
  "kind": "INTERNAL",
  "scope": {
    "name": "@arizeai/openinference-instrumentation-langchain",
    "version": "4.0.14"
  },
  "attributes": {
    "openinference.span.kind": "CHAIN",
    "input.value": "{\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"HumanMessage\"], \"kwargs\": {\"content\": \"Hey, how can you help me\"}}]}",
    "output.value": "{\"messages\": [{\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"AIMessage\"], \"kwargs\": {\"content\": \"I can assist you with planning your trip ...\"}}]}",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Execute tool span

The openinference.span.kind attribute (TOOL) identifies this as an execute tool span; tool.name holds the tool name. The input.value and output.value attributes hold the tool arguments and result (serialized as a LangChain ToolMessage).


{
  "traceId": "6a6bd0c25c52f3d86a35038f35f5ba30",
  "spanId": "ab105c12cc40048f",
  "name": "search_flights",
  "kind": "INTERNAL",
  "scope": {
    "name": "@arizeai/openinference-instrumentation-langchain",
    "version": "4.0.14"
  },
  "attributes": {
    "openinference.span.kind": "TOOL",
    "tool.name": "search_flights",
    "input.value": "{\"origin\": \"SEA\", \"destination\": \"NYC\", \"date\": \"2025-03-15\"}",
    "output.value": "{\"output\": {\"lc\": 1, \"type\": \"constructor\", \"id\": [\"langchain_core\", \"messages\", \"ToolMessage\"], \"kwargs\": {\"status\": \"success\", \"content\": \"{\\\"origin\\\": \\\"SEA\\\", \\\"destination\\\": \\\"NYC\\\", \\\"flights\\\": [ ... ]}\"}}}",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Inference span

The openinference.span.kind attribute (LLM) identifies this as an inference span. The llm.input_messages.* attributes hold the system prompt and user prompt, and the llm.output_messages.* attributes hold the agent response.


{
  "traceId": "6a6bd0c1c8d91ed1e70a3906b551618",
  "spanId": "1221a062c7f90a8e",
  "name": "ChatOpenAI",
  "kind": "INTERNAL",
  "scope": {
    "name": "@arizeai/openinference-instrumentation-langchain",
    "version": "4.0.14"
  },
  "attributes": {
    "openinference.span.kind": "LLM",
    "llm.model_name": "gpt-4o-mini",
    "llm.input_messages.0.message.role": "system",
    "llm.input_messages.0.message.content": "You are a travel planning assistant ...",
    "llm.input_messages.1.message.role": "user",
    "llm.input_messages.1.message.content": "Hey, how can you help me",
    "llm.output_messages.0.message.role": "assistant",
    "llm.output_messages.0.message.content": "I can assist you with planning your trip ...",
    "session.id": "sea-nyc-trip-2-turns"
  },
  "status": {
    "code": "OK"
  }
}

Best practices for LangGraph agents

How you build and invoke a LangGraph agent affects what appears in its telemetry, and therefore how reliably the agent can be evaluated. The following practices help ensure the user prompt, agent response, and tool activity are recoverable.

1. Choose an agent construction pattern

There are two common ways to build a LangGraph agent:

Prebuilt create_agent : the quickest way to get started. It produces a single invoke agent span per turn, with the conversation passed through LangGraph’s built-in execution loop. Use this when you want a standard reason-act agent without custom control flow.
```
from langchain.agents import create_agent

agent = create_agent(model=model, tools=[search_flights, book_flight])
```

Custom StateGraph : gives you full control over nodes, edges, and conditional routing. Each node execution becomes its own span, so traces are more granular. Use this when you need custom orchestration.


from langgraph.graph import StateGraph, START, END
from typing_extensions import TypedDict

class State(TypedDict):
    messages: list

graph = StateGraph(State)
graph.add_node("generate_response", generate_response)
graph.add_node("tools", run_tools)
graph.add_edge(START, "generate_response")
agent = graph.compile()

Both patterns are evaluated the same way; the difference is the granularity of the trace.

2. Use `messages` in your graph State (recommended)

The evaluation service reconstructs the conversation from the agent’s input and output messages. Using a messages field is not mandatory, but it enables the most reliable extraction. For a custom StateGraph, keep the conversation in a messages field in your State:

Include messages in your State (recommended). You can add other custom fields (such as user_id or metadata). When messages is present, the standard extraction finds the user prompt and agent response directly. If messages is absent, the service falls back to reconstructing the conversation from individual inference spans, which is less reliable.
Append, don’t replace. Follow the LangGraph convention of appending new messages to the list rather than overwriting it, so the full conversation history is preserved.
Use canonical LangChain message types (HumanMessage, AIMessage, ToolMessage, SystemMessage). The instrumentation serializes these correctly, and the service recognizes their roles.

3. Pass the user message in a supported format

When you invoke a LangGraph agent, you add the user message to the graph’s messages state. LangGraph accepts the message in three interchangeable formats, and AgentCore Evaluations supports all of them. Each produces spans and event records that the service can read.

Tuple: a (role, content) pair:


agent.invoke({"messages": [("user", user_message)]}, config=config)

LangChain message object: a HumanMessage (or other message class):


from langchain_core.messages import HumanMessage

agent.invoke({"messages": [HumanMessage(content=user_message)]}, config=config)

Dictionary: a {"role", "content"} dictionary:


agent.invoke({"messages": [{"role": "user", "content": user_message}]}, config=config)

All three formats result in the same messages state, so the user prompt and agent response are extracted identically regardless of which you choose.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Strands Agents

OpenAI Agents

Set up LangGraph telemetry for AgentCore Evaluations

Python agent support

Instrument your agent

Example

Note

How spans are identified

Example

How evaluation fields are extracted

From event records

From span attributes

Example spans in split telemetry

Note

OpenTelemetry

Example

OpenInference

Example

Example spans in unified telemetry

Note

OpenTelemetry

Example

OpenInference

Example

TypeScript agent support

Instrument your agent

Example

Note

How spans are identified

How evaluation fields are extracted

Example spans from a TypeScript agent

Note

OpenTelemetry (ADOT native)

Example

OpenTelemetry (Traceloop)

Example

OpenInference

Example

Best practices for LangGraph agents

1. Choose an agent construction pattern

2. Use messages in your graph State (recommended)

3. Pass the user message in a supported format

2. Use `messages` in your graph State (recommended)