Request Syntax URI Request Parameters Request Body Response Syntax Response Elements Errors See Also

CreateOnlineEvaluationConfig

Creates an online evaluation configuration for continuous monitoring of agent performance. Online evaluation automatically samples live traffic from CloudWatch logs at specified rates and applies evaluators to assess agent quality in production.

Request Syntax


POST /online-evaluation-configs/create HTTP/1.1
Content-type: application/json

{
   "clientToken": "string",
   "dataSourceConfig": { ... },
   "description": "string",
   "enableOnCreate": boolean,
   "evaluationExecutionRoleArn": "string",
   "evaluators": [ 
      { ... }
   ],
   "onlineEvaluationConfigName": "string",
   "rule": { 
      "filters": [ 
         { 
            "key": "string",
            "operator": "string",
            "value": { ... }
         }
      ],
      "samplingConfig": { 
         "samplingPercentage": number
      },
      "sessionConfig": { 
         "sessionTimeoutMinutes": number
      }
   }
}

URI Request Parameters

The request does not use any URI parameters.

Request Body

The request accepts the following data in JSON format.

clientToken

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.

Type: String

Length Constraints: Minimum length of 33. Maximum length of 256.

Pattern: [a-zA-Z0-9](-*[a-zA-Z0-9]){0,256}

Required: No

dataSourceConfig

The data source configuration that specifies CloudWatch log groups and service names to monitor for agent traces.

Type: DataSourceConfig object

Note: This object is a Union. Only one member of this object can be specified or returned.

Required: Yes

description

The description of the online evaluation configuration that explains its monitoring purpose and scope.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 200.

Pattern: .+

Required: No

enableOnCreate

Whether to enable the online evaluation configuration immediately upon creation. If true, evaluation begins automatically.

Type: Boolean

Required: Yes

evaluationExecutionRoleArn

The Amazon Resource Name (ARN) of the IAM role that grants permissions to read from CloudWatch logs, write evaluation results, and invoke Amazon Bedrock models for evaluation.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 2048.

Pattern: arn:aws(-[^:]+)?:iam::([0-9]{12})?:role/.+

Required: Yes

evaluators

The list of evaluators to apply during online evaluation. Can include both built-in evaluators and custom evaluators created with CreateEvaluator.

Type: Array of EvaluatorReference objects

Array Members: Minimum number of 1 item. Maximum number of 10 items.

Required: Yes

onlineEvaluationConfigName

The name of the online evaluation configuration. Must be unique within your account.

Type: String

Pattern: [a-zA-Z][a-zA-Z0-9_]{0,47}

Required: Yes

rule

The evaluation rule that defines sampling configuration, filters, and session detection settings for the online evaluation.

Type: Rule object

Required: Yes

Response Syntax


HTTP/1.1 202
Content-type: application/json

{
   "createdAt": number,
   "executionStatus": "string",
   "failureReason": "string",
   "onlineEvaluationConfigArn": "string",
   "onlineEvaluationConfigId": "string",
   "outputConfig": { 
      "cloudWatchConfig": { 
         "logGroupName": "string"
      }
   },
   "status": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 202 response.

The following data is returned in JSON format by the service.

createdAt

The timestamp when the online evaluation configuration was created.

Type: Timestamp

executionStatus

The execution status indicating whether the online evaluation is currently running.

Type: String

Valid Values: ENABLED | DISABLED

failureReason

The reason for failure if the online evaluation configuration creation or execution failed.

Type: String

onlineEvaluationConfigArn

The Amazon Resource Name (ARN) of the created online evaluation configuration.

Type: String

Pattern: arn:aws:bedrock-agentcore:[a-z0-9-]+:[0-9]{12}:online-evaluation-config\/[a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}

onlineEvaluationConfigId

The unique identifier of the created online evaluation configuration.

Type: String

Pattern: [a-zA-Z][a-zA-Z0-9-_]{0,99}-[a-zA-Z0-9]{10}

outputConfig

The configuration that specifies where evaluation results should be written for monitoring and analysis.

Type: OutputConfig object

status

The status of the online evaluation configuration.

Type: String

Errors

For information about the errors that are common to all actions, see Common Errors.

AccessDeniedException

This exception is thrown when a request is denied per access permissions

HTTP Status Code: 403

ConflictException

This exception is thrown when there is a conflict performing an operation

HTTP Status Code: 409

InternalServerException

This exception is thrown if there was an unexpected error during processing of request

HTTP Status Code: 500

ServiceQuotaExceededException

This exception is thrown when a request is made beyond the service quota

HTTP Status Code: 402

ThrottlingException

This exception is thrown when the number of requests exceeds the limit

HTTP Status Code: 429

ValidationException

The input fails to satisfy the constraints specified by the service.

HTTP Status Code: 400