latestOpenAPI 3.0.42026-08-19355873993.7 KB

fdb89a6eb2fb

Benchmark

Queries a benchmark's prompt ratings — a page of prompts, each with its ranked entries and the pairwise win matrix over them.

Filters apply to the underlying votes and use the same field set as the standings query. Ordering (extremeness by default) selects which prompts a page fits; a rating is an entry's standing against the other entries on its prompt, so a participant filter narrows the entries but does not refit. A prompt with no matching votes yields no entries. groupBy decides what competes. Per image, every image is its own entry and a participant with several images on a prompt appears several times. Per participant, its images become one entry rated from their pooled wins over other participants; comparisons between two of its own images are excluded from the rating, from both matrices, and from responseCount. Grouping changes only how prompts are scored, never which prompts the page holds. matrix.wins and matrix.votes are square and aligned to entries by position: row i, column j holds entry i's count against entry j. Read the axes off entries — the matrix carries no labels of its own.

get/benchmark/{benchmarkId}/prompt-ratings/query

Path parameters

benchmarkIdstring required

The id of the benchmark.

Query parameters

useWeightedScoringboolean

Whether to weight ratings by annotator reliability.

groupBy'Image' | 'Participant'

What each rated entry of a prompt represents.

What each rated entry represents.

pageinteger

The 1-based page index.

page_sizeinteger

The number of items per page.

sortstring[]

Sort fields. Prefix with - for descending order (e.g. -created_at).

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by country.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by language.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by gender.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by age_bucket.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by occupation.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by tags.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by participant_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by leaderboard_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by run_id.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by prompt_identifier.

eqstring
neqstring
gtstring
gtestring
ltstring
ltestring
containsstring
starts_withstring
ends_withstring
instring[]
not_containsstring

Filter by voted_at.

logic'and' | 'or'

How to combine the field filters: "and" (default) requires every filter to match, "or" requires any of them to match.

Response

OK

totalinteger required

The number of prompts the filter leaves rankable — the paging total.

pageinteger required

The 1-based page index.

pageSizeinteger required

The page size in prompts.