Sign in
latestOpenAPI 3.0.3Apache-2.02026-08-21144388693.7 KB

c26a00e9b318

Annotation Queues

Create an annotation queue

Create a new annotation queue.

Payload Requirements

  • The annotation queue name must be unique within the given space (among active queues).
  • At least one annotation_config_id is required, and all configs must belong to the specified space.
  • Do not include system-managed fields on input: id, created_at, updated_at.
  • If assignment_method is not provided, it defaults to "all".

Valid example

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["reviewer@example.com"],
  "assignment_method": "all"
}

Valid example with records

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "annotation_config_ids": ["ac_abc123"],
  "annotator_emails": ["reviewer@example.com"],
  "records": [
    {"record_type": "SPAN", "project_id": "prj_abc", "start_time": "2024-01-15T00:00:00Z", "end_time": "2024-01-16T00:00:00Z", "span_ids": ["span_001"]},
    {"record_type": "EXAMPLE", "dataset_id": "ds_xyz", "example_ids": ["ex_001", "ex_002"]}
  ]
}

Invalid example (missing required annotation_config_ids)

{
  "name": "My Queue",
  "space_id": "spc_xyz789"
}
post/v2/annotation-queues

Request body

namestring required

The name of the annotation queue. Must be unique within the space for active queues.

space_idstring required

The space ID that the annotation queue belongs to

instructionsstring

Instructions for annotators working on this queue

annotation_config_idsstring[] required

IDs of annotation configs to associate with this queue. All configs must belong to the same space.

annotator_emailsEmail[] required

Email addresses of annotators to assign to the queue. Emails are resolved to user IDs server-side.

assignment_method'ALL' | 'RANDOM'

How records are assigned to annotators. Defaults to "ALL".

  • ALL: Every annotator is assigned to every record.
  • RANDOM: Each record is randomly assigned to one annotator.
column_allowliststring[]

The record column names annotators assigned to this queue are allowed to see. Omit or send an empty list to leave the queue unrestricted, so annotators see every column. Because this is an allowlist, columns introduced by records added to the queue later stay hidden until an admin allows them.

Example request

{
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotation_config_ids": [
    "ac_abc123",
    "ac_def456"
  ],
  "annotator_emails": [
    "annotator1@example.com",
    "annotator2@example.com"
  ],
  "column_allowlist": [
    "context.trace_id",
    "attributes.llm.input_messages",
    "column with spaces"
  ]
}

Response

An annotation queue object

idstring required

The unique identifier for the annotation queue

namestring required

The name of the annotation queue

space_idstring required

The space id the annotation queue belongs to

instructionsstring nullable

The instructions for the annotation queue

created_atstring date-time required

The timestamp for when the annotation queue was created

updated_atstring date-time required

The timestamp for when the annotation queue was last updated

column_allowliststring[] nullable

The record column names annotators assigned to this queue are allowed to see. Absent or null means the queue is unrestricted and annotators see every column.

Example response

{
  "id": "aq_abc123",
  "name": "Quality Review Queue",
  "space_id": "spc_xyz789",
  "instructions": "Review each response for accuracy and helpfulness",
  "annotators": [
    {
      "email": "user@example.com"
    }
  ],
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-20T14:45:00Z",
  "column_allowlist": [
    "context.trace_id",
    "attributes.llm.input_messages",
    "column with spaces"
  ]
}