latestOpenAPI 3.0.42026-08-19355873993.7 KB

fdb89a6eb2fb

Benchmark

Queries a benchmark's image matchups — a page of compared image pairs with how the responses split between them.

An item is one image pair as one leaderboard saw it, so a pair compared on several leaderboards comes back once per leaderboard and the boards' verdicts on it can be read side by side. Ordering (extremeness by default) ranks the matchups themselves — extremeness being how far the split sits from a coin flip, in standard deviations — where the prompt-ratings query ranks prompts by their most decisive matchup. Filters use that query's field set, applied to the votes a matchup is tallied from, plus WinnerParticipantId and LooserParticipantId, which select matchups by which side won the whole matchup; those two are only accepted as top-level and leaves, and a tied matchup matches neither. Matchups of leaderboards excluded from the benchmark overall are never returned. sides holds exactly two images, the one that took more responses first, and their responses sum to the matchup's. extremeness and the winning side are always read off the raw counts; useWeightedScoring changes only each side's wins.

get/benchmark/{benchmarkId}/matchups/query

Path parameters

benchmarkIdstring required

The id of the benchmark.

Query parameters

useWeightedScoringboolean

Whether to report each side's wins weighted by annotator reliability.

groupBy'Image' | 'Participant'

What each rated entry of a prompt represents.

What a compared image stands for. Per image, every matchup is returned. Per participant, a matchup between two images of one participant is a self-matchup and leaves the page.

pageinteger

The 1-based page index.

page_sizeinteger

The number of items per page.

sortstring[]

Sort fields. Prefix with - for descending order (e.g. -created_at).

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by country.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by language.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by gender.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by age_bucket.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by occupation.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by tags.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by participant_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by leaderboard_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by run_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by prompt_identifier.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by voted_at.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by winner_participant_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by looser_participant_id.

logic'and' | 'or'

How to combine the field filters: "and" (default) requires every filter to match, "or" requires any of them to match.

Response

OK

totalinteger required

The number of matchups the filter leaves — the paging total.

pageinteger required

The 1-based page index.

pageSizeinteger required

The page size in matchups.