v2

latestOpenAPI 3.1.0AGPL-3.0raw.githubusercontent.com2026-08-01312535.8 KB
model-eval

Start a ModelLab benchmark

Starts an asynchronous local benchmark. Candidate and judge calls use the configured OpenRouter API key and may incur provider costs. Progress is streamed through /api/ws/model-eval/{job_id}.

post/api/model-eval

Request body

modelsstring[] required
judgestring nullable
timeoutnumber
max_tokensinteger
runsinteger
concurrencyinteger
suite_id'quick-v2' | 'advanced-v1'

Response

Benchmark job accepted