Sign in

4e064cf81dae

latestOpenAPI 3.1.02026-08-131831,331960.0 KB
evaluators

Create an evaluator

Create a grader/evaluator. Current public evaluator types are llm, human, and code; legacy human-specific types remain readable for older evaluators. New clients should use the flat config fields (score_config, passing_conditions, llm_config, code_config) instead of relying on legacy configurations.

post/api/evaluators/

Headers

Authorizationstring required

Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.

Request body

namestring required
evaluator_slugstring

Organization-scoped evaluator slug.

type'llm' | 'human' | 'code'
score_value_type'numerical' | 'boolean' | 'percentage' | 'single_select' | 'multi_select' | 'text' | 'json' | 'comment' | 'categorical'
eval_class'keywordsai_custom_llm' | 'custom_code'

Optional pre-built evaluator template.

descriptionstring nullable
passing_conditionsApiEvaluatorsPostRequestBodyContentApplicationJsonSchemaPassingConditions

Passing conditions in the standard Respan filter format.

configurationsobject

Legacy user-facing configuration object. New clients should prefer llm_config, code_config, score_config, and passing_conditions.

starredboolean

Response

Created evaluator.

idstring required

Stable evaluator ID shared by all versions.

version_idstring

Unique ID for this evaluator version.

versioninteger

Version number. The first draft is version 0.

is_read_onlyboolean

Whether this version is committed and immutable.

version_descriptionstring

Commit message for this version.

namestring required
evaluator_slugstring

Organization-scoped evaluator slug.

type'llm' | 'human' | 'code' required
score_value_type'numerical' | 'boolean' | 'percentage' | 'single_select' | 'multi_select' | 'text' | 'json' | 'comment' | 'categorical' required
eval_class'keywordsai_custom_llm' | 'custom_code'

Optional pre-built evaluator template.

descriptionstring nullable
passing_conditionsApiEvaluatorsPostResponsesContentApplicationJsonSchemaPassingConditions

Passing conditions in the standard Respan filter format.

configurationsobject

Legacy user-facing configuration object. New clients should prefer llm_config, code_config, score_config, and passing_conditions.

starredboolean
created_atstring date-time
updated_atstring date-time
is_publicboolean

Whether this is a Respan-managed public evaluator.