Sign in

v2

latestOpenAPI 3.1.02026-08-05267431678.1 KB
felix
datasets

Get Upload Url

Get presigned S3 URL for direct dataset upload (bypasses API Gateway limits).

Two-step upload flow:

  1. Call this endpoint to get presigned URL and dataset_id
  2. Upload file directly to S3 using the presigned URL (HTTP PUT)
  3. Call /upload/process with dataset_id to trigger validation/processing

Auto-versioning behavior:

  • If dataset_name is new: creates version 1
  • If dataset_name exists: creates next version (e.g., 2, 3, etc.)

Response:

  • presigned_url: S3 PUT URL for direct upload
  • dataset_id: Use this in /upload/process
  • version_number: Dataset version
  • expires_in: URL valid for this many seconds (default: 3600 = 1 hour)

Supports JSONL, CSV, TSV, JSON, and Parquet formats.

post/felix/datasets/upload/url

Request body

dataset_namestring required

Name for the dataset

dataset_type'classification' | 'ner' | 'custom' | 'decoder'

Type of dataset

format'jsonl' | 'csv' | 'tsv' | 'parquet' | 'json' nullable

File format (auto-detected from filename if not provided)

filenamestring nullable

Original filename (used for format detection if format not provided)

schemastring nullable

Expected schema as JSON object (e.g., {"age": "Int64", "name": "Utf8"}). Enforces column types during parsing. Valid types: Int8, Int16, Int32, Int64, UInt8-64, Float32, Float64, Utf8, Boolean, Date, Datetime, Time, Duration, Categorical, Binary

project_idstring nullable

Optional project ID (UUID) to assign this dataset to

type'training' | 'evaluation' | 'benchmark' nullable

Dataset purpose: 'training' (trainable), 'evaluation' (not trainable), 'benchmark' (system-managed, evaluation-only; cannot be trained on or directly accessed).

visibilitystring nullable

Dataset visibility: 'private' (owner only) or 'public' (anyone can see)

generation_type'synthesize' | 'upload' | 'auto_relabel' | 'manual_relabel' | 'grow' | 'external' nullable

How the dataset was created: synthesize, upload, auto_relabel, manual_relabel, grow, external

split_ratioobject nullable

Split ratio when type is 'split', e.g. {'training': 0.8, 'evaluation': 0.2}

column_mappingobject nullable

Column mapping from source to standard names (e.g., {"sentence": "text", "category": "label"}). Valid standard targets: text, label, labels, entities.

Response

Successful Response

presigned_urlstring required

S3 presigned URL for PUT upload

dataset_idstring required

Dataset ID for subsequent processing

dataset_namestring required

Dataset name (for polling status via GET /{name}/{version})

version_numberstring required

Version number for this dataset

expires_ininteger required

URL expiration time in seconds

upload_instructionsstring

Instructions for completing the upload