Sign in
OpenAPI 3.1.0raw.githubusercontent.com2026-08-202766453.1 MB

2cd7e7f7bcb4

Evals

Invoke a Custom Evaluator

Runs an evaluator that already exists in the workspace. Accepts either a conversation or the structured input and output fields; when both are present the conversation wins.

post/v3/evaluators/{id}/invoke

Path parameters

idstring required

Accepts a bare id, id@version, or id@environment.

Request body

modelstring

Model to grade with, as a catalog id such as "openai/gpt-4o".

Only meaningful for a hub template of type llm_eval or ragas, which has no model of its own. A stored evaluator uses the model on its own definition and ignores this.

querystring

Latest user message. Folds into context.input.user_query.

outputstring

The generated response from the model. Folds into context.output.response.

referencestring

The reference used to compare the output. Folds into context.input.expected_output.

retrievalsstring[]

Knowledge base retrievals. Folds into context.input.retrievals.

messagesobject[]

The conversation that produced the output. Folds into context.messages.

variablesobject

Template variables for evaluator prompt substitution. Folds into context.variables.

Response

OK