--- title: "POST /evaluators/create" method: POST path: "/evaluators/create" --- # POST /evaluators/create `POST /evaluators/create` Creates a custom evaluator for agent quality assessment. Custom evaluators can use either LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings, or code-based configurations with customer-managed Lambda functions to evaluate agent performance at tool call, trace, or session levels. ## Request body - object - `clientToken` string — A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency. - `evaluatorName` string, required — The name of the evaluator. Must be unique within your account. - `description` string, password — The description of the evaluator that explains its purpose and evaluation criteria. - `evaluatorConfig` object, required — The configuration that defines how an evaluator assesses agent performance, including the evaluation method and parameters. - `llmAsAJudge` object — The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales. - `instructions` string, password, required — The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines. - `ratingScale` object, required — The rating scale that defines how the evaluator should score agent performance, either numerical or categorical. - `numerical` NumericalScaleDefinition[] — The numerical rating scale with defined score values and descriptions for quantitative evaluation. - `definition` string, required — The description that explains what this numerical rating represents and when it should be used. - `value` number, double, required — The numerical value for this rating scale option. - `label` string, required — The label or name that describes this numerical rating option. - `categorical` CategoricalScaleDefinition[] — The categorical rating scale with named categories and definitions for qualitative evaluation. - `definition` string, required — The description that explains what this categorical rating represents and when it should be used. - `label` string, required — The label or name of this categorical rating option. - `modelConfig` object, required — The model configuration that specifies which foundation model to use and how to configure it for evaluation. - `bedrockEvaluatorModelConfig` object — The Amazon Bedrock model configuration for evaluation. - `modelId` string, required — The identifier of the Amazon Bedrock model to use for evaluation. Must be a supported foundation model available in your region. - `inferenceConfig` object — The inference configuration parameters that control model behavior during evaluation, including temperature, token limits, and sampling settings. - `maxTokens` integer — The maximum number of tokens to generate in the model response during evaluation. - `temperature` number, float — The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs. - `topP` number, float — The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices. - `stopSequences` NonEmptyString[] — The list of sequences that will cause the model to stop generating tokens when encountered. - `additionalModelRequestFields` object — Additional model-specific request fields to customize model behavior beyond the standard inference configuration. - `codeBased` object — Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance. - `lambdaConfig` object — The Lambda function configuration for code-based evaluation. - `lambdaArn` string, required — The Amazon Resource Name (ARN) of the Lambda function that implements the evaluation logic. - `lambdaTimeoutInSeconds` integer — The timeout in seconds for the Lambda function invocation. Defaults to 60. Must be between 1 and 300. - `level` 'TOOL_CALL' | 'TRACE' | 'SESSION', required — The evaluation level that determines the scope of evaluation. Valid values are TOOL_CALL for individual tool invocations, TRACE for single request-response interactions, or SESSION for entire conversation sessions. - `kmsKeyArn` string — The Amazon Resource Name (ARN) of a customer managed KMS key to use for encrypting sensitive evaluator data, including instructions and rating scale. If you don't specify a KMS key, the evaluator data is encrypted with an Amazon Web Services owned key. Only symmetric encryption KMS keys are supported. For more information, see Encryption at rest for AgentCore Evaluations. - `tags` object — A map of tag keys and values to assign to an AgentCore Evaluator. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment. ## Response `202` Success - CreateEvaluatorResponse - `evaluatorArn` string, required — The Amazon Resource Name (ARN) of the created evaluator. - `evaluatorId` string, required — The unique identifier of the created evaluator. - `createdAt` string, date-time, required — The timestamp when the evaluator was created. - `status` 'ACTIVE' | 'CREATING' | 'CREATE_FAILED' | 'UPDATING' | 'UPDATE_FAILED' | 'DELETING', required — The status of the evaluator creation operation. ## Other responses - `480` — ServiceQuotaExceededException - `481` — ValidationException - `482` — AccessDeniedException - `483` — ConflictException - `484` — ThrottlingException - `485` — InternalServerException --- [API](https://skmtc.net/aws/apis/bedrock-agentcore-control.md) · [All operations](https://skmtc.net/aws/apis/bedrock-agentcore-control/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/aws/bedrock-agentcore-control/versions/0ebde766792b/schema)