---
title: "Launch one run per target as a group (async)"
method: POST
path: "/projects/{projectId}/eval-run-groups"
tags: ["Eval runs"]
---

# Launch one run per target as a group (async)

`POST /projects/{projectId}/eval-run-groups`

Launches ONE run per target — attached project environments, or attached named hosts — under a single **server-minted** `runGroupId`, then detaches execution and responds `202` with a per-target receipt.

This is the only endpoint with grouped-launch semantics. `POST /eval-runs` also accepts a `runGroupId`, but purely as a display label: it groups sibling rows in the UI and gives N separate launches no group treatment. Here the server bounds the fan-out (max 10 targets), validates EVERY target before launching any of them (an unattached target, a heterogeneous target list, or a target whose host selects an unavailable harness returns `400` with zero runs started), and holds exactly ONE organization concurrency slot for the whole group — released only when the last sibling finishes.

Targets fan out along ONE axis: all `environmentId` or all `namedHostId`, never a mix. Duplicates are deduplicated by id, preserving order.

A per-target failure does NOT abort its siblings, so read `outcome` rather than treating the `202` as "everything started": `started` means every target launched, `partial` means some did, `failed` means none did.

Send `Idempotency-Key` (or `idempotencyKey`) to make the launch replayable: the group id is DERIVED from the key, and each target carries a run key derived from that, so a retry after a crash mid-launch returns the original run ids instead of double-launching the targets that already started.

## Path parameters

- `projectId` string, required

## Request body

- EvalRunGroupCreateRequest — Launch one run per target under a single server-minted group id. `targets` fan out along ONE axis: every entry is an `environmentId`, or every entry is a `namedHostId`. A mixed list returns `400` with `details.reason: "HETEROGENEOUS_TARGETS"` — mixing them would have to answer whether a host runs inside an environment or beside it, and every answer to that is a cross product nobody asked for. Duplicates are deduplicated by id, preserving order. The knobs below apply UNIFORMLY to every run in the group. `serverIds` and `refreshSnapshot` are deliberately absent: an environment supplies its own closed server set, a host derives one from its config, and last-writer-wins on a frozen suite snapshot is never what a fan-out means. Inline `tests` are absent for the same reason — a group runs a persisted suite.
  - `suiteId` string, required — The suite every target runs.
  - `targets` union[], required — One entry per run. Every target must be ATTACHED to the suite; an unattached one returns `400` with zero runs started.
    - union
      - object
        - `environmentId` string, required — A project environment in the suite's `environmentIds`.
        - `namedHostId` string — A host attached to the suite.
      - object
        - `environmentId` string — A project environment in the suite's `environmentIds`.
        - `namedHostId` string, required — A host attached to the suite.
  - `iterationOverride` integer — Override the per-case `runs` count for every run in the group.
  - `caseIds` string[] — Narrow every run in the group to these suite cases.
  - `matchOptionsOverride` object — Match options for every run in the group. Same public-or-internal vocabulary as `POST /eval-runs`.
  - `skillsOverride` 'exclude' — Run every target as the "without skills" arm.
  - `notes` string
  - `passCriteria` object
    - `minimumPassRate` number
  - `idempotencyKey` string — Makes the whole launch replayable. The `runGroupId` is DERIVED from this key and each target's run key is derived from that, so a retry after a crash mid-launch returns the original run ids rather than double-launching the targets that already started. The `Idempotency-Key` header carries the same value and wins over this field.

## Response `202`

Group accepted. Each target's outcome is in `targets`; execution continues in the background.

- EvalRunGroupCreated — One entry per target. A per-target failure does NOT abort its siblings, so read `outcome` rather than treating the `202` as "everything started".
  - `runGroupId` string, required — Server-minted. Every sibling run carries it, and the hosted UI collapses them into one row.
  - `suiteId` string, required
  - `outcome` 'started' | 'partial' | 'failed', required — `started` — every target launched; `partial` — some did and some did not; `failed` — none did (still a `202`: the group itself was valid, and the per-target reasons are in `targets`).
  - `startedCount` integer, required
  - `failedCount` integer, required
  - `targets` object[], required — Discriminated on `status`, so a target that failed can never be mistaken for one that started with an unread `runId`.
    - `target` object, required — Which target this entry is for. Exactly one id is set.
      - `environmentId` string
      - `namedHostId` string
      - `name` string
    - `status` 'started' | 'failed', required
    - `runId` string — Present when `status` is `started`.
    - `runStatus` string — The RUN's status (always `running` at launch). Named apart from the entry's own `status` on purpose — two fields called `status` in one object is how a reader ends up branching on the wrong one.
    - `servers` object[]
      - `id` string
      - `name` string
    - `environment` EvalRunEnvironment, nullable — The project environment a run is pinned to, at the revision resolved when it launched. `null` for a legacy run that used the suite's saved server selection.
      - `id` string, required
      - `name` string, nullable
      - `revision` integer, nullable — The environment revision the run executed against.
    - `caseUpsert` object
    - `error` object — Present when `status` is `failed`. The code is preserved so a caller can tell a retryable conflict from a genuine internal error.
      - `code` string
      - `message` string
  - `runId` string — Mirror of the FIRST started run, so readers written against the single-run receipt keep working. Absent when nothing started. Read `targets` instead — this describes one run out of several.
  - `status` string — See `runId`.
  - `servers` object[] — See `runId`.
    - `id` string
    - `name` string
  - `environment` EvalRunEnvironment, nullable — The project environment a run is pinned to, at the revision resolved when it launched. `null` for a legacy run that used the suite's saved server selection.
    - `id` string, required
    - `name` string, nullable
    - `revision` integer, nullable — The environment revision the run executed against.
  - `caseUpsert` object — See `runId`.

## Other responses

- `400` — Malformed body or parameters.
- `401` — Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is a property of the server, not your key.
- `403` — Key is valid but not allowed to do this.
- `404` — Unknown project, server, or resource.
- `429` — Per-key rate limit exceeded (60 requests/minute sustained, bursts up to 10). Honor `Retry-After` and back off with jitter.
- `500` — Something failed on MCPJam's side.
- `502` — Could not connect to the target MCP server.
- `504` — The target MCP server connected but didn't respond in time.

---

[API](https://skmtc.net/mcpjam/apis/mcpjam-api.md) · [All operations](https://skmtc.net/mcpjam/apis/mcpjam-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/mcpjam/mcpjam-api/revisions/ece7d99ceaf3/schema)