v171

OpenAPI 3.1.0raw.githubusercontent.com2026-08-0726249111.7 MB
Evals

Invoke a Custom Evaluator

post/v2/evaluators/{id}/invoke

Path parameters

idstring required

Evaluator ID

Evaluator ID

Request body

querystring

Latest user message

outputstring

The generated response from the model

referencestring

The reference used to compare the output

retrievalsstring[]

Knowledge base retrievals

modelstring

Model to use for LLM-based evaluators (e.g. "openai/gpt-4o")

variablesEvaluatorVariables

Template variables for evaluator prompt substitution. Request values override evaluator defaults, including for nested arrays and objects.

Example request

{
  "messages": [
    {
      "content": [
        {
          "image_url": {
            "url": "https://picsum.photos/id/1/200/300"
          }
        }
      ]
    }
  ],
  "variables": {
    "locale": "en",
    "tags": [
      "alpha",
      "omega"
    ],
    "profile": {
      "tier": "gold",
      "active": true
    }
  }
}

Response

Returns the result of the evaluator run

OR
OR
OR
OR
OR
OR
OR
OR