latestOpenAPI 3.1.02026-08-226994571.1 MB
1e929292ddd5
OpenAI Evals API
Create Eval
Create a new evaluation.
Model-based routing (for multi-account support):
- Pass model via header: x-litellm-model: gpt-4-account-1
- Pass model via query: ?model=gpt-4-account-1
- Pass model via body: {"model": "gpt-4-account-1"}
Example usage:
curl -X POST "http://localhost:4000/v1/evals" -H "Authorization: Bearer your-key" -H "Content-Type: application/json" -d '{
"name": "Test Eval",
"data_source_config": {"type": "file", "file_id": "file-abc123"},
"testing_criteria": {"graders": [{"type": "llm_as_judge"}]}
}'
Returns: Eval object with id, status, timestamps, etc.
post/v1/evals
Query parameters
custom_llm_providerstring nullable
Response
Successful Response