Sign in

v2

latestOpenAPI 3.1.02026-08-05267431678.1 KB
felix
datasets

Process Uploaded Dataset

Process dataset uploaded to S3 via presigned URL.

Prerequisites:

  1. Must have called /upload/url to get dataset_id
  2. Must have uploaded file to S3 using presigned URL

Response:

  • Returns 202 Accepted with dataset in "uploading" status
  • Poll GET /{name}/{version} to check processing status
  • Status will change to "ready" when complete or "failed" on error

Processing:

  • Validates file exists in S3
  • Converts to Parquet if needed (CSV/JSONL/JSON → Parquet)
  • Extracts metadata (row count, schema, etc.)
  • Validates data against expected schema if provided
  • Updates database status throughout
post/felix/datasets/upload/process

Request body

dataset_idstring required

Dataset ID from upload/url response (all metadata stored in DB)

Response

Successful Response

idstring required
user_idstring required
dataset_namestring required
dataset_pathstring required
dataset_typestring required
sizeinteger nullable
sample_sizeinteger nullable
train_rationumber nullable

Train split ratio for this dataset version. Left-to-right split with no shuffle; validation is the tail.

created_atstring required
updated_atstring required
version_numberstring
root_dataset_idstring nullable
project_idstring nullable
schemaobject nullable
schema_warningsstring[] nullable
validationobject nullable
annotation_status'none' | 'in_progress' | 'completed' nullable
annotation_configobject nullable
annotation_progressobject nullable
status'initialized' | 'uploading' | 'converting' | 'validating' | 'ready' | 'failed' | 'generating' | 'queued' nullable

Dataset status: initialized/uploading/converting/validating/ready/failed/generating/queued

processing_errorstring nullable

Error message if status is failed

typestring

Dataset purpose tag: 'training', 'evaluation', or 'benchmark'

visibilitystring

Dataset visibility: 'private' or 'public'

is_competitionboolean

Whether this dataset is a competition benchmark

labelsstring[] nullable

Label names (entity types for NER, class labels for classification)

generation_typestring nullable

How the dataset was created: synthesize, upload, auto_relabel, manual_relabel, grow, external

is_seedboolean nullable

Whether this dataset is a seed dataset (small set for review before full expansion)

synthesis_session_idstring nullable

UUID of the synthesis log session for this dataset, used to restore creation workflow on resume

column_mappingobject nullable

Column mapping from original to standard names