latestOpenAPI 3.1.02026-08-197104501.1 MB

8fbaab4fc7c5

evals

Create Run

Create a new run for an evaluation.

Model-based routing (for multi-account support):

  • Pass model via header: x-litellm-model: gpt-4-account-1
  • Pass model via query: ?model=gpt-4-account-1
  • Pass model via body: {"model": "gpt-4-account-1"}
  • Pass model via completion.model: {"completion": {"model": "gpt-4-account-1"}}

Example usage:

curl -X POST "http://localhost:4000/v1/evals/eval_123/runs"       -H "Authorization: Bearer your-key"       -H "Content-Type: application/json"       -d '{
    "data_source": {"type": "dataset", "dataset_id": "dataset_123"},
    "completion": {"model": "gpt-4", "temperature": 0.7}
  }'

Returns: Run object with id, status, timestamps, etc.

post/v1/evals/{eval_id}/runs

Path parameters

eval_idstring required

Query parameters

custom_llm_providerstring nullable

Response

Successful Response

completed_atinteger nullable
created_atinteger required
data_sourceobject required
errorobject nullable
eval_idstring required
idstring required
metadataobject nullable
modelstring nullable
namestring nullable
objectstring
{"stackTrail":"components:schemas:Run:properties:per_model_usage:anyOf","oasType":"schema","type":"unknown","title":"Per Model Usage","nullable":true}
report_urlstring nullable
result_countsobject nullable
shared_with_openaiboolean nullable
started_atinteger nullable
status'queued' | 'running' | 'completed' | 'failed' | 'cancelled' required