---
title: "Start an eval run"
method: POST
path: "/v1/ai/eval/runs"
tags: ["AI Eval"]
---

# Start an eval run

`POST /v1/ai/eval/runs`

Create and start a new run against an existing prompt set. The run enqueues one agentic job per prompt and begins executing immediately. Returns the newly created run with its initial per-prompt result rows.

## Request body

- EvalRunsCreateBody
  - `description` string, nullable — Optional human-readable description for the run. Omit or pass `null` to leave it unset. Max 1024 characters.
  - `prompt_set_id` string, uuid, required — The prompt set to execute.
  - `run_config` object — Per-run configuration. Optional — omit if no overrides.
    - `branch_id` string, uuid — Optional branch ID to run against. Must be a branch of the prompt set's model.

## Response `201`

Run created and jobs enqueued.

- EvalRunsCreateResponse
  - `job_count` integer, required — Number of per-prompt agentic jobs created for this run (one per prompt that fanned out successfully). Enqueue onto the work queue happens after creation and is best-effort, so this count reflects jobs created, not necessarily those successfully enqueued.
  - `run` EvalRunDetail, required — The newly created run with its initial results.
    - `branch_id` string, uuid, nullable, required — Optional branch ID the run was executed against. Null when run against the main shared model.
    - `branch_name` string, nullable, required — Display name for the branch, if `branch_id` is set.
    - `completed_at` string, nullable, required — ISO 8601 timestamp when the run reached a terminal state.
    - `created_at` string, nullable, required — ISO 8601 timestamp when the run was created.
    - `description` string, nullable, required — Optional human-readable description for the run.
    - `id` string, uuid, required — Unique identifier for the run.
    - `is_archived` boolean, required — Whether the run has been archived.
    - `model_id` string, uuid, required — The shared model this run was executed against.
    - `prompt_set_id` string, uuid, required — The prompt set this run was created from.
    - `results` EvalRunResult[], required — Per-prompt results for this run, ordered by their creation order in the prompt set.
      - `agentic_job` EvalRunResultAgenticJob, required
        - `conversation_id` string, uuid, nullable, required — Conversation the agentic job belongs to.
        - `id` string, uuid, required — Agentic job identifier.
        - `state` 'CANCELLED' | 'COMPLETE' | 'DELIVERING' | 'EXECUTING' | 'FAILED' | 'QUEUED', required — Current state of the agentic job that ran this prompt.
      - `cost` number, nullable, required — Total LLM cost (USD) for this prompt, if available.
      - `error_reason` string, nullable, required — Failure reason string for prompts whose underlying job failed.
      - `id` string, uuid, required — Unique identifier for the run result row.
      - `prompt` string, required — The prompt text that was evaluated.
      - `score` number, nullable, required — Numeric judge score for this prompt result, if scoring ran.
      - `scoring_cost` number, nullable, required — Total LLM cost (USD) for scoring this prompt result.
      - `timing_ms` integer, nullable, required — Wall-clock duration of the underlying job in milliseconds.
    - `run_number` integer, required — Sequential, per-prompt-set run number.
    - `status` 'RUNNING' | 'COMPLETE' | 'CANCELLED', required — Run-level lifecycle. Flips to a terminal state (COMPLETE or CANCELLED) exactly once.

## Other responses

- `400` — Invalid request body.
- `401` — Missing or invalid API key.
- `403` — Insufficient permissions. The caller must have at least the Querier role on the prompt set's model.
- `404` — The prompt set was not found, or `run_config.branch_id` does not match an existing branch in the organization.
- `422` — `run_config.branch_id` does not belong to the prompt set's model.
- `429` — Per-user active-run cap reached.
- `503` — AI eval is paused for this organization.

---

[API](https://skmtc.net/omni/apis/omni-api.md) · [All operations](https://skmtc.net/omni/apis/omni-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/omni/omni-api/revisions/6b02f7349d0e/schema)