---
title: "Create Evaluation"
method: POST
path: "/api/v1/evaluations/"
tags: ["evals"]
---

# Create Evaluation

`POST /api/v1/evaluations/`

Create a new evaluation

This endpoint supports:
- Environment evaluations: Provide environments
- Prime RL evaluations: Provide run_id
- Suite evaluations: Provide suite_id

Ownership:
- If team_id is provided in request, the evaluation will be owned by the team
- Otherwise, the evaluation will be owned by the authenticated user

## Request body

- CreateEvaluationRequest — Request to create a new evaluation
  - `name` string, required — Name of the evaluation
  - `team_id` string, nullable — Team ID if creating evaluation for a team
  - `environments` EnvironmentReference[], nullable — List of environment references with optional version IDs
    - `id` string, required
    - `version_id` string, nullable
  - `suite_id` string, nullable — Suite ID if this evaluation is part of a suite
  - `run_id` string, nullable — Run ID for Prime RL runs (optional)
  - `is_hosted` boolean — Whether this is a hosted evaluation
  - `inference_model` string, nullable — Prime Inference model ID
  - `eval_config` HostedEvalConfig — Hosted evaluation configuration
    - `num_examples` integer, required — Number of examples to evaluate (-1 for all)
    - `rollouts_per_example` integer, required — Rollouts per example
    - `env_args` object, nullable — Optional environment arguments to pass to the evaluation
    - `allow_sandbox_access` boolean, nullable — Allow sandbox read/write access
    - `allow_instances_access` boolean, nullable — Allow instance creation and management access
    - `allow_tunnel_access` boolean, nullable — Allow tunnel creation and management access
    - `timeout_minutes` integer, nullable — Custom timeout in minutes for the hosted eval run. Defaults to 1440. Use any negative value for no sandbox lifetime deadline. Finite values must be between 60 and 5256000.
    - `custom_secrets` object, nullable — Custom secrets to set in the evaluation sandbox (e.g., API keys, tokens)
    - `sampling_args` object, nullable — Optional sampling arguments forwarded to `prime eval run --sampling-args`
    - `max_concurrent` integer, nullable — Optional max concurrency forwarded to `prime eval run --max-concurrent`
    - `auto_max_concurrent` boolean — Whether max_concurrent was computed from hosted eval runtime resources
    - `max_retries` integer, nullable — Optional max retries forwarded to `prime eval run --max-retries`
    - `state_columns` string[], nullable — Optional state columns forwarded to `prime eval run --state-columns`
    - `independent_scoring` boolean, nullable — Forward `--independent-scoring` to the hosted eval runner
    - `verbose` boolean, nullable — Forward `--verbose` to the hosted eval runner
    - `headers` string[], nullable — Optional repeated headers forwarded to `prime eval run --header`
    - `extra_env_kwargs` object, nullable — Optional environment constructor kwargs forwarded to `prime eval run --extra-env-kwargs`
    - `api_client_type` string, nullable — Optional API client type forwarded to `prime eval run --api-client-type`
    - `api_base_url` string, nullable — Optional inference base URL forwarded to `prime eval run --api-base-url`
    - `api_key_var` string, nullable — Optional API key env var forwarded to `prime eval run --api-key-var`
  - `model_name` string, nullable — Model name
  - `dataset` string, nullable — Dataset name
  - `framework` string, nullable — Framework used (e.g., 'prime-rl', 'openai/evals')
  - `task_type` string, nullable — Type of task (e.g., 'classification', 'generation')
  - `description` string, nullable — Description of the evaluation
  - `tags` string[] — Tags for categorization
  - `metadata` object, nullable — Additional metadata
  - `metrics` object, nullable — High-level metrics summary
  - `is_public` boolean — Whether this evaluation is publicly shareable by link
  - `show_on_leaderboard` boolean — Whether this public evaluation appears on environment leaderboards

## Response `201`

Successful Response

- CreateEvaluationResponse — Response after creating an evaluation
  - `evaluation_id` string, required — ID of the created evaluation
  - `name` string, required
  - `status` 'PENDING' | 'RUNNING' | 'PROCESSING' | 'COMPLETED' | 'FAILED' | 'TIMEOUT' | 'CANCELLED', required — Evaluation status enum
  - `eval_type` 'suite' | 'training' | 'environment', required — Evaluation type enum
  - `environment_ids` string[], nullable
  - `suite_id` string, nullable
  - `run_id` string, nullable
  - `version_id` string, nullable
  - `viewer_url` string, nullable
  - `created_at` string, date-time, required

## Other responses

- `401` — Authorization failed
- `422` — Invalid request data

---

[API](https://skmtc.net/primeintellect/apis/pi-api.md) · [All operations](https://skmtc.net/primeintellect/apis/pi-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/primeintellect/pi-api/revisions/3d3868828a78/schema)