Sign in

v1

latestOpenAPI 3.0.0Apache 2.0 License2026-07-141531,219979.8 KB

Creates a custom evaluator for agent quality assessment. Custom evaluators can use either LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings, or code-based configurations with customer-managed Lambda functions to evaluate agent performance at tool call, trace, or session levels.

post/evaluators/create

Request body

clientTokenstring

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see <a href="https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_Instance_Idempotency.html">Ensuring idempotency</a>.

evaluatorNamestring required

The name of the evaluator. Must be unique within your account.

descriptionstring password

The description of the evaluator that explains its purpose and evaluation criteria.

level'TOOL_CALL' | 'TRACE' | 'SESSION' required

The evaluation level that determines the scope of evaluation. Valid values are <code>TOOL_CALL</code> for individual tool invocations, <code>TRACE</code> for single request-response interactions, or <code>SESSION</code> for entire conversation sessions.

kmsKeyArnstring

The Amazon Resource Name (ARN) of a customer managed KMS key to use for encrypting sensitive evaluator data, including instructions and rating scale. If you don't specify a KMS key, the evaluator data is encrypted with an Amazon Web Services owned key. Only symmetric encryption KMS keys are supported. For more information, see <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/evaluations-encryption.html">Encryption at rest for AgentCore Evaluations</a>.

tagsobject

A map of tag keys and values to assign to an AgentCore Evaluator. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment.

Response

Success

evaluatorArnstring required

The Amazon Resource Name (ARN) of the created evaluator.

evaluatorIdstring required

The unique identifier of the created evaluator.

createdAtstring date-time required

The timestamp when the evaluator was created.

status'ACTIVE' | 'CREATING' | 'CREATE_FAILED' | 'UPDATING' | 'UPDATE_FAILED' | 'DELETING' required

The status of the evaluator creation operation.