Sign in

v3

latestOpenAPI 3.1.02026-07-311,4541,5202.3 MB
evaluations

Eval Sets Create

POST handler with superadmin-only field protection.

Strips superadmin-only fields from non-superadmin requests before delegating to OrganizationInjectionMixin.post() for org injection.

post/evaluations/eval-sets/{id}/

Path parameters

idstring required

Headers

Authorizationstring required

JWT access token or Respan API key

Request body

initial_log_filtersFilterParamDictPydantic

Pydantic model for FilterParamDict. A dictionary that maps metric names to their filter parameters.

Each key is a metric name (str), and each value can be:

  • A single MetricFilterParamPydantic (one condition)
  • A List[MetricFilterParamPydantic] (multiple conditions for same metric)
  • A FilterBundlePydantic (nested filter bundle with connector)

Note: Uses extra="allow" for dynamic metric name fields. The pydantic_extra annotation tells Pydantic what types to expect for extra fields, and generates typed additionalProperties in JSON Schema.

projectstring nullable
namestring required
type'llm' | 'human' | 'dataset' | 'sampling' | 'reference'
  • llm - Llm
  • human - Human
  • dataset - Dataset
  • sampling - Sampling
  • reference - Reference
descriptionstring
running_progressnumber double
running_status'draft' | 'pending' | 'paused' | 'running' | 'waiting_for_annotations' | 'completed' | 'failed'
  • draft - Draft
  • pending - Pending
  • paused - Paused
  • running - Running
  • waiting_for_annotations - Waiting For Annotations
  • completed - Completed
  • failed - Failed
running_atstring date-time nullable
unique_organization_idsstring[]
timestampsstring[]
ingest_workflow_idstring nullable
starredboolean
organizationinteger required
evaluatorstring nullable

Response

idstring required
completed_annotation_countinteger required
initial_log_filtersFilterParamDictPydantic

Pydantic model for FilterParamDict. A dictionary that maps metric names to their filter parameters.

Each key is a metric name (str), and each value can be:

  • A single MetricFilterParamPydantic (one condition)
  • A List[MetricFilterParamPydantic] (multiple conditions for same metric)
  • A FilterBundlePydantic (nested filter bundle with connector)

Note: Uses extra="allow" for dynamic metric name fields. The pydantic_extra annotation tells Pydantic what types to expect for extra fields, and generates typed additionalProperties in JSON Schema.

projectstring nullable
namestring required
type'llm' | 'human' | 'dataset' | 'sampling' | 'reference'
  • llm - Llm
  • human - Human
  • dataset - Dataset
  • sampling - Sampling
  • reference - Reference
granularity'logs' | 'traces' | 'threads' required
  • logs - Logs
  • traces - Traces
  • threads - Threads
descriptionstring
created_atstring date-time required
running_progressnumber double
running_status'draft' | 'pending' | 'paused' | 'running' | 'waiting_for_annotations' | 'completed' | 'failed'
  • draft - Draft
  • pending - Pending
  • paused - Paused
  • running - Running
  • waiting_for_annotations - Waiting For Annotations
  • completed - Completed
  • failed - Failed
running_atstring date-time nullable
updated_atstring date-time required
unique_organization_idsstring[]
timestampsstring[]
log_countinteger required
ingest_workflow_idstring nullable
status'initializing' | 'ready' | 'failed' | 'loading' required
  • initializing - Initializing
  • ready - Ready
  • failed - Failed
  • loading - Loading
starredboolean
organizationinteger required
evaluatorstring nullable