Sign in

v2

latestOpenAPI 3.1.02026-07-2675166192.8 KB
evals

Create Evaluation

Create a new evaluation

This endpoint supports:

  • Environment evaluations: Provide environments
  • Prime RL evaluations: Provide run_id
  • Suite evaluations: Provide suite_id

Ownership:

  • If team_id is provided in request, the evaluation will be owned by the team
  • Otherwise, the evaluation will be owned by the authenticated user
post/api/v1/evaluations/

Request body

namestring required

Name of the evaluation

team_idstring nullable

Team ID if creating evaluation for a team

suite_idstring nullable

Suite ID if this evaluation is part of a suite

run_idstring nullable

Run ID for Prime RL runs (optional)

is_hostedboolean

Whether this is a hosted evaluation

inference_modelstring nullable

Prime Inference model ID

model_namestring nullable

Model name

datasetstring nullable

Dataset name

frameworkstring nullable

Framework used (e.g., 'prime-rl', 'openai/evals')

task_typestring nullable

Type of task (e.g., 'classification', 'generation')

descriptionstring nullable

Description of the evaluation

tagsstring[]

Tags for categorization

metadataobject nullable

Additional metadata

metricsobject nullable

High-level metrics summary

is_publicboolean

Whether this evaluation is publicly shareable by link

show_on_leaderboardboolean

Whether this public evaluation appears on environment leaderboards

Response

Successful Response

evaluation_idstring required

ID of the created evaluation

namestring required
status'PENDING' | 'RUNNING' | 'PROCESSING' | 'COMPLETED' | 'FAILED' | 'TIMEOUT' | 'CANCELLED' required

Evaluation status enum

eval_type'suite' | 'training' | 'environment' required

Evaluation type enum

environment_idsstring[] nullable
suite_idstring nullable
run_idstring nullable
version_idstring nullable
viewer_urlstring nullable
created_atstring date-time required