---
title: "Update Collection"
method: PATCH
path: "/v1/collections/{collection_identifier}"
tags: ["Collections"]
---

# Update Collection

`PATCH /v1/collections/{collection_identifier}`

Update mutable collection fields (collection_name, description, taxonomy_applications, enabled)

## Path parameters

- `collection_identifier` string, required — The ID or name of the collection to update

## Request body

- object

## Response `200`

Successful Response

- CollectionResponse — Response model for collection endpoints.
  - `collection_id` string — NOT REQUIRED (auto-generated). Unique identifier for this collection. Used for: API paths, document queries, pipeline references. Format: 'col_' prefix + 10 random alphanumeric characters. Stable after creation - use for all collection references.
  - `collection_name` string, required — REQUIRED. Human-readable name for the collection. Must be unique within the namespace. Used for: Display, lookups (can query by name or ID), organization. Format: Alphanumeric with underscores/hyphens, 3-100 characters. Examples: 'product_embeddings', 'video_frames', 'customer_documents'.
  - `description` string, nullable — NOT REQUIRED. Human-readable description of the collection's purpose. Use for: Documentation, team communication, UI display. Common pattern: Describe what the collection contains and what processing is applied.
  - `input_schema` BucketSchemaOutput — Schema definition for bucket objects. IMPORTANT: The bucket schema defines what fields your bucket objects will have. This schema is REQUIRED if you want to: 1. Create collections that use input_mappings to process your bucket data 2. Validate object structure before ingestion 3. Enable type-safe data pipelines The schema defines the custom fields that will be used in: - Blob properties (e.g., "content", "thumbnail", "transcript") - Object metadata structure - Blob data structures Example workflow: 1. Create bucket WITH schema defining your data structure 2. Upload objects that conform to that schema 3. Create collections that map schema fields to feature extractors Without a bucket_schema, collections cannot use input_mappings.
    - `properties` object, required — REQUIRED. Map of field names to their type definitions. Each field must have a 'type' from the supported types: metadata types (string, number, integer, float, boolean, object, array, date, datetime) or file/blob types (text, image, audio, video, pdf, excel). NOTE: Use Mixpeek types, NOT JSON Schema types — e.g. use 'string' not 'keyword', 'text' for text blobs, 'image' for image blobs. Example: {"title": {"type": "string"}, "photo": {"type": "image"}}
  - `output_schema` BucketSchemaOutput — Schema definition for bucket objects. IMPORTANT: The bucket schema defines what fields your bucket objects will have. This schema is REQUIRED if you want to: 1. Create collections that use input_mappings to process your bucket data 2. Validate object structure before ingestion 3. Enable type-safe data pipelines The schema defines the custom fields that will be used in: - Blob properties (e.g., "content", "thumbnail", "transcript") - Object metadata structure - Blob data structures Example workflow: 1. Create bucket WITH schema defining your data structure 2. Upload objects that conform to that schema 3. Create collections that map schema fields to feature extractors Without a bucket_schema, collections cannot use input_mappings.
    - `properties` object, required — REQUIRED. Map of field names to their type definitions. Each field must have a 'type' from the supported types: metadata types (string, number, integer, float, boolean, object, array, date, datetime) or file/blob types (text, image, audio, video, pdf, excel). NOTE: Use Mixpeek types, NOT JSON Schema types — e.g. use 'string' not 'keyword', 'text' for text blobs, 'image' for image blobs. Example: {"title": {"type": "string"}, "photo": {"type": "image"}}
  - `feature_extractor` SharedCollectionFeaturesExtractorsModelsFeatureExtractorConfigOutput, required — Configuration for a feature extractor with field passthrough support. A feature extractor processes source data (from buckets or collections) and produces features (embeddings, extracted text, detected objects, etc.). With field passthrough, you can also include selected source fields in the output documents alongside the computed features. Core Concepts: 1. **Feature Extraction**: Extractors compute features from input data (e.g., text → embeddings, image → detections, video → scenes) 2. **Field Passthrough**: Selectively preserve source fields in output (e.g., title, category, campaign_id from source → output documents) 3. **Output Schema**: Combination of passed-through fields + extractor outputs (e.g., {title, category, text_embedding} all in one document) How Field Passthrough Works: 1. Define which source fields to include via field_passthrough list 2. During processing, these fields are extracted from source 3. They appear in output documents at root level 4. Combine with extractor outputs for complete documents 5. Use target_path to rename fields for cleaner schemas Field Selection Modes: - **Explicit** (field_passthrough + include_all=False): Only listed fields pass through. Clean, controlled output. Example: passthrough=[title, category] → output has ONLY title, category, embedding - **Inclusive** (include_all=True): All source fields pass through, field_passthrough for renaming. Example: source has 10 fields → output has all 10 + embedding - **None** (no field_passthrough): Only extractor outputs in documents. Example: → output has ONLY embedding (no source fields) Use Cases: - **Preserve Identifiers**: Keep campaign_id, product_sku, order_id for tracking - **Enable Filtering**: Pass category, status, department for query filters - **Maintain Context**: Include title, description for display - **Track Metadata**: Preserve author, created_at, source for lineage - **Business Logic**: Keep priority, region, type for application logic Common Patterns: 1. **Minimal Passthrough** (recommended): field_passthrough=[{"source_path": "id"}], include_all=False → Clean output, only ID + extractor features 2. **Metadata Preservation**: field_passthrough=[ {"source_path": "title"}, {"source_path": "category"}, {"source_path": "created_at"} ] → Document has context for display and filtering 3. **Field Renaming**: field_passthrough=[ {"source_path": "doc_title", "target_path": "title"}, {"source_path": "metadata.author", "target_path": "author"} ] → Cleaner output schema with flattened fields 4. **Required Fields**: field_passthrough=[ {"source_path": "campaign_id", "required": True}, {"source_path": "priority", "default": 0} ] → Ensures critical fields always present Requirements: - feature_extractor_name: REQUIRED - name of the extractor - version: REQUIRED - extractor version (e.g., "v1") - parameters: NOT REQUIRED - extractor-specific config (model, thresholds, etc.) - input_mappings: NOT REQUIRED - maps extractor inputs to source fields - field_passthrough: NOT REQUIRED - which source fields to preserve (default: none) - include_all_source_fields: NOT REQUIRED - preserve all fields (default: false)
    - `feature_extractor_name` string, required — Name of the feature extractor
    - `version` string, required — Version of the feature extractor (e.g., 'v1', 'v2')
    - `params` object, nullable — Optional extractor parameters that affect vector index configuration. Parameters set here are locked at namespace creation and determine vector dimensions in Qdrant. Collections using this extractor must use compatible params. Example: {'model': 'siglip_base'}
    - `parameters` union — Parameters for the feature extractor. Each extractor type has specific parameters. See the schema for your chosen extractor (e.g., MultimodalExtractorParams for multimodal_extractor).
      - AudioFingerprintExtractorParams — Parameters for the Audio Fingerprint Extractor. Processes audio files (or audio extracted from video) through CLAP (Contrastive Language-Audio Pretraining) to produce 512-d embeddings suitable for audio fingerprint matching. Core Pipeline: 1. Audio extraction (if video input, via FFmpeg) 2. Segmentation into fixed-length windows 3. CLAP embedding (laion/clap-htsat-tiny, 512-d) 4. L2 normalization Use Cases: - Sound mark detection (IP safety) - Audio similarity search - Music/jingle identification - Audio deduplication
        - `extractor_type` 'audio_fingerprint_extractor' — Discriminator field. Must be 'audio_fingerprint_extractor'.
        - `segment_duration_sec` number — Duration of each audio segment in seconds. 5.0: Recommended for sound mark matching. Shorter segments increase recall but reduce per-segment context.
        - `segment_hop_sec` number — Hop size between segments in seconds. 2.5: 50% overlap (recommended). Set equal to segment_duration_sec for no overlap.
        - `sample_rate` integer — Target sample rate for audio. 48000: CLAP default (recommended). Audio is resampled to this rate before embedding.
        - `normalize_embeddings` boolean — L2-normalize embeddings to unit vectors (recommended for cosine similarity).
        - `max_audio_length_sec` number — Maximum audio length to process in seconds. 120: Default (2 minutes). Audio beyond this is truncated.
      - DocumentGraphExtractorParams — Parameters for the document graph extractor. This extractor decomposes PDFs into spatial blocks with layout classification, confidence scoring, and optional VLM correction for degraded documents. **When to Use**: - Historical/archival document processing (FBI files, old records) - Scanned documents with mixed quality - Documents requiring spatial understanding (forms, tables, multi-column) - When you need block-level granularity with bounding boxes - When confidence scoring is needed for downstream filtering **When NOT to Use**: - Simple text-only documents -> Use text_extractor instead - When page-level granularity is sufficient -> Use pdf_extractor instead - Real-time processing requirements -> VLM correction adds latency
        - `extractor_type` 'document_graph_extractor' — Discriminator field for parameter type identification. Must be 'document_graph_extractor'.
        - `use_layout_detection` boolean — Enable ML-based layout detection to find ALL document elements (text, images, tables, figures). When enabled, uses the configured layout_detector to detect and extract both text regions AND non-text elements (scanned images, figures, charts) as separate documents. **Recommended for**: Scanned documents, image-heavy PDFs, mixed content documents. **When disabled**: Falls back to text-only extraction (faster but misses images). Default: True (detects all elements including images).
        - `layout_detector` 'pymupdf' | 'docling' — Layout detection engine to use when use_layout_detection=True. 'pymupdf': Fast, rule-based detection using PyMuPDF heuristics (~15 pages/sec). 'docling': SOTA ML-based detection using IBM Docling with DiT model (~3-8 sec/doc). **Docling advantages**: Better semantic type detection (section_header vs paragraph), true table structure extraction (rows/cols), more accurate figure detection. **PyMuPDF advantages**: Much faster, lower memory usage, simpler dependencies. Default: 'pymupdf' for speed. Use 'docling' for accuracy-critical applications.
        - `vertical_threshold` number — Maximum vertical gap (in points) between lines to be grouped in same block. Increase for looser grouping, decrease for tighter blocks. Default 15pt works well for standard documents.
        - `horizontal_threshold` number — Maximum horizontal distance (in points) for overlap detection. Affects column detection and block merging. Increase for wider columns, decrease for narrow layouts.
        - `min_text_length` integer — Minimum text length (characters) to keep a block. Blocks with less text are filtered out. Helps remove noise and tiny fragments.
        - `base_confidence` number — Base confidence score for embedded (native) text. Penalties are subtracted for OCR artifacts, encoding issues, etc.
        - `min_confidence_for_vlm` number — Confidence threshold below which VLM correction is triggered. Blocks with confidence < this value get sent to VLM for correction. Only applies when use_vlm_correction=True.
        - `use_vlm_correction` boolean — Enable VLM (Vision Language Model) correction for low-confidence blocks. Uses Gemini/GPT-4V to correct OCR errors by analyzing the page image. Significantly slower (~1 page/sec) but improves accuracy for degraded docs.
        - `fast_mode` boolean — Skip VLM correction entirely for maximum throughput (~15 pages/sec). Overrides use_vlm_correction. Use when speed is more important than accuracy.
        - `vlm_provider` string — LLM provider for VLM correction. Options: 'google' (Gemini), 'openai' (GPT-4V), 'anthropic' (Claude). Google recommended for best vision quality.
        - `vlm_model` string — Specific model for VLM correction. Examples: 'gemini-2.5-flash', 'gpt-4o', 'claude-3-5-sonnet'.
        - `llm_api_key` string, nullable — API key for VLM correction (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
        - `run_text_embedding` boolean — Generate text embeddings for semantic search over block content. Uses E5-Large (1024-dim) for multilingual support.
        - `render_dpi` integer — DPI for page rendering (used for VLM correction). 72: Fast, lower quality. 150: Balanced (recommended). 300: High quality, slower.
        - `generate_thumbnails` boolean — Generate thumbnail images for blocks. Useful for visual previews and UI display.
        - `thumbnail_mode` string — Thumbnail generation mode. 'full_page': Low-res thumbnail of entire page. 'segment': Cropped thumbnail of just the block's bounding box. 'both': Generate both types (recommended for flexibility).
        - `thumbnail_dpi` integer — DPI for thumbnail generation. Lower DPI = smaller files. 72: Standard web quality. 36: Very small thumbnails.
      - FaceIdentityExtractorParams — Parameters for the Face Identity Extractor. The Face Identity Extractor processes images or video frames to detect, align, and embed faces using production-grade SOTA models (SCRFD + ArcFace). Core Pipeline: 1. SCRFD Detection → Bounding boxes + 5 landmarks 2. 5-Point Affine Alignment → 112×112 canonical face 3. ArcFace Embedding → 512-d L2-normalized vector 4. Optional Quality Scoring → Filter low-quality faces Use Cases: - Face verification (1:1 matching) - Face identification (1:N search) - Face clustering (group photos by person) - Duplicate face detection
        - `extractor_type` 'face_identity_extractor' — Discriminator field for parameter type identification. Must be 'face_identity_extractor'.
        - `detection_model` 'scrfd_500m' | 'scrfd_2.5g' | 'scrfd_10g' — SCRFD model for face detection. 'scrfd_500m': Fastest (2-3ms). 'scrfd_2.5g': Balanced (5-7ms), recommended. 'scrfd_10g': Highest accuracy (10-15ms).
        - `min_face_size` integer — Minimum face size in pixels to detect. 20px: Balanced. 40px: Higher quality. 10px: Maximum recall.
        - `detection_threshold` number — Confidence threshold for face detection (0.0-1.0).
        - `max_faces_per_image` integer, nullable — Maximum number of faces to process per image. None: Process all.
        - `normalize_embeddings` boolean — L2-normalize embeddings to unit vectors (recommended).
        - `enable_quality_scoring` boolean — Compute quality scores (blur, size, landmarks). Adds ~5ms per face.
        - `quality_threshold` number, nullable — Minimum quality score to index faces. None: Index all faces. 0.5: Moderate filtering. 0.7: High quality only.
        - `max_video_length` integer — Maximum video length in seconds. 60: Default. 10: Recommended for retrieval. 300: Maximum (extraction only).
        - `video_sampling_fps` number, nullable — Frames per second to sample from video. 1.0: One frame per second (recommended).
        - `video_deduplication` boolean — Remove duplicate faces across video frames (extraction only). Reduces 90-95% redundancy. NOT used in retrieval.
        - `video_deduplication_threshold` number — Cosine similarity threshold for deduplication. 0.8: Conservative (default).
        - `output_mode` 'per_face' | 'per_image' — 'per_face': One document per face (recommended). 'per_image': One doc per image with faces array.
        - `include_face_crops` boolean — Include aligned 112×112 face crops as base64. Adds ~5KB per face. Required for LLM cluster labeling to see actual faces instead of hallucinating.
        - `include_source_frame_thumbnail` boolean — Include resized source frame/image as base64 thumbnail (~15-30KB per face). Used for display with bounding box overlay.
        - `store_detection_metadata` boolean — Store bbox, landmarks, detection scores. Recommended for debugging.
      - GeminiMultifileExtractorParams — Parameters for the Gemini Multifile Extractor. Uses Gemini Embedding 2 (gemini-embedding-2) to embed all files of an object into a single 3072-d vector in one API call. Supports images, video, audio, PDF, and text blobs.
        - `extractor_type` 'gemini_multifile_extractor' — Discriminator field for parameter type identification.
        - `output_dimensionality` integer — Output embedding dimensions. Gemini Embedding 2 supports 3072 (default), 768, or 256 via truncation. Lower dimensions reduce storage cost at slight quality loss.
        - `task_type` string — Embedding intent used as a text instruction for Gemini Embedding 2. Common values: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY, CLASSIFICATION.
        - `input_key` string — The input_mappings key whose value is the list of blob fields to embed together. Must match the key used in input_mappings (e.g., 'files'). Default: 'files'.
      - ImageExtractorParams — Parameters for the Image Extractor.
        - `extractor_type` 'image_extractor' — Discriminator field for parameter type identification.
        - `enable_thumbnails` boolean — Whether to generate thumbnail images.
        - `use_cdn` boolean — Whether to use CloudFront CDN for thumbnail delivery.
      - MultimodalExtractorParams — Parameters for multimodal extractor v2. Same pipeline as v1 but uses Gemini Embedding 2 (3072D) for the multimodal embedding step. Supports configurable output dimensions via Matryoshka representation learning (3072/1536/768).
        - `extractor_type` 'multimodal_extractor' — Discriminator field. Must be 'multimodal_extractor'.
        - `split_method` 'time' | 'scene' | 'silence'
        - `description_prompt` string — Prompt for description generation.
        - `time_split_interval` integer, nullable — Interval in seconds for 'time' splitting.
        - `silence_db_threshold` integer, nullable — Decibel threshold for silence detection. Recommended: -40.
        - `scene_detection_threshold` number, nullable — Scene detection threshold (0.0-1.0). Recommended: 0.5.
        - `run_transcription` boolean — Run Whisper transcription on segments.
        - `transcription_language` string — Transcription language code.
        - `run_video_description` boolean — Generate Gemini descriptions for segments.
        - `run_transcription_embedding` boolean — Generate E5 embeddings for transcriptions (1024D).
        - `run_ocr_embedding` boolean — Generate E5 embeddings for OCR text (1024D). Requires run_ocr.
        - `run_description_embedding` boolean — Generate E5 embeddings for descriptions (1024D). Requires run_video_description.
        - `run_multimodal_embedding` boolean — Generate Gemini Embedding 2 multimodal embeddings (3072D). Creates unified embeddings across video, image, text, audio, and GIF content.
        - `run_ocr` boolean — Extract text from video frames via Gemini OCR.
        - `max_segment_duration` number, nullable — Maximum duration in seconds for any single segment. Scene/silence segments longer than this are subdivided. Set to None to disable. Default: 30s.
        - `sensitivity` string — Scene detection sensitivity.
        - `enable_thumbnails` boolean — Generate thumbnail images for segments.
        - `use_cdn` boolean — Use CloudFront CDN for thumbnail delivery.
        - `generation_config` GenerationConfig — Configuration for generative models.
          - `candidate_count` integer — Number of candidate responses to generate for video description.
          - `max_output_tokens` integer — Maximum number of tokens for the generated video description.
          - `temperature` number — Controls randomness for video description generation. Higher is more random.
          - `top_p` number — Nucleus sampling (top-p) for video description generation.
          - `response_mime_type` string, nullable — MIME type for response (e.g., 'application/json')
          - `response_schema` object, nullable — JSON schema for structured output
        - `output_dimensionality` integer — Output embedding dimensions. Gemini Embedding 2 supports Matryoshka dimension reduction: 3072 (full), 1536, or 768.
        - `task_type` string — Embedding task type hint. Options: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY, CLASSIFICATION.
        - `response_shape` union — Custom structured output schema for Gemini extraction. String for natural language prompt, dict for explicit JSON schema.
          - string
          - object
        - `embedding_task` string, nullable — Embedding task hint for instruction-aware models (E5). Prefer setting this at collection level (embedding_task on the collection) rather than here. Collection-level overrides this value. Defaults to 'retrieval_document'. Values: retrieval_document, retrieval_query, semantic_similarity, classification, clustering. Note: Vertex AI multimodal embeddings ignore this — only E5 transcription embeddings use it.
      - PassthroughExtractorParams — Parameters for passthrough extractor. Minimal configuration - just passes data through with canonicalization.
        - `extractor_type` 'passthrough_extractor' — Discriminator field for parameter type identification.
        - `preserve_metadata` boolean — Preserve source object metadata in output document.
      - ScrollingTextExtractorParams — Parameters for the on-screen text extractor. Extracts ALL on-screen text from video: STATIC overlays (title cards, disclaimers, captions, CTAs — the majority of ad-creative text) via keyframe-grid VLM OCR, and SCROLLING/marquee text via computer vision (phase-correlation band detection + panoramic stitching) + VLM OCR. **When to Use**: - Ad creative with static overlay text (disclaimers, CTAs, titles) - Video ads with scrolling promotional banners or tickers - News broadcasts with scrolling chyrons or tickers - Videos with terms & conditions or disclaimers (static or scrolling) - Social media content with text overlays - Credits sequences in film/TV content - Live event streams with scrolling info bars **When NOT to Use**: - Spoken content → use multimodal_extractor with run_transcription=True - Text documents/PDFs → use text_extractor **How It Works**: Static pass (static_ocr=True, default): 1. Select visually distinct keyframes (identical frames collapse to one) 2. Stitch them into a single grid image 3. OCR the grid with one vision-language-model call (Gemini) Scrolling pass: 1. Sample frames from video at configurable FPS 2. Split each frame into horizontal and vertical strips 3. Phase-correlate consecutive frames to measure per-strip pixel shift 4. Strips with consistent shift in one direction = scrolling band 5. Stitch the band across frames into a single wide/tall panorama image 6. OCR the panorama using a vision language model (Gemini) 7. Deduplicate repeated marquee loops (e.g. "SALE • SALE • SALE •" → "SALE") **Performance**: - Processing speed: ~2-5x realtime (depends on video resolution and FPS) - Accuracy: Best with consistent scroll speed; handles variable speed with degradation - Minimum video length: ~2 seconds (needs 3+ frames for correlation) **Supported Scroll Directions**: - Horizontal: Right-to-left (most common), Left-to-right - Vertical: Bottom-to-top (credits), Top-to-bottom
        - `extractor_type` 'scrolling_text_extractor' — Discriminator field. Must be 'scrolling_text_extractor'.
        - `fps` number — Frame sampling rate for analysis. Higher values improve detection accuracy for fast-scrolling text but increase processing time. 5 FPS works well for most video ads and tickers.
        - `strip_height` integer — Height (in pixels) of each scanning strip used for phase correlation. Should roughly match the height of the scrolling text band. Smaller values detect narrower text bands; larger values are more robust but may miss thin tickers. 40px works for most standard video ads.
        - `min_shift_px` number — Minimum pixel shift per frame to consider a strip as 'scrolling'. Lower values detect slower-moving text; higher values filter out noise. 2.0px is a good default for 5 FPS sampling.
        - `consistency_ratio` number — Fraction of frame pairs that must show consistent shift for a band to be classified as scrolling. 0.6 means 60% of frames must agree. Lower values detect intermittent scrolling; higher values reduce false positives.
        - `pad` integer — Pixel padding above/below detected band when cropping for stitching.
        - `static_ocr` boolean — Also OCR STATIC on-screen text (title cards, disclaimers, CTAs, captions) — the majority of text in ad creative. Distinct keyframes are stitched into one grid image and read with a single VLM call, so a fully static video costs one OCR call. Disable only if you strictly want scrolling-band text.
        - `max_static_frames` integer — Maximum number of visually distinct keyframes sampled for the static-text OCR grid. Identical frames collapse to one; scene changes contribute additional frames up to this cap.
      - TextExtractorParams — Parameters for the text extractor. The text extractor generates dense vector embeddings optimized for semantic similarity search. It uses the E5-Large multilingual model to convert text into 1024-dimensional vectors. When ``source_type`` is ``"youtube"``, the extractor first resolves YouTube URLs to caption text via yt-dlp before chunking and embedding. Use ``split_by="time_segments"`` with ``segment_length_seconds`` to segment captions by time window.
        - `extractor_type` 'text_extractor' — Discriminator field for parameter type identification.
        - `source_type` 'text' | 'youtube' — Source content type. Use 'youtube' to resolve YouTube URLs to caption text before embedding. Default: 'text' (plain text input).
        - `split_by` 'characters' | 'words' | 'sentences' | 'paragraphs' | 'pages' | 'time_segments' | 'none' — Strategy for splitting text into chunks.
        - `chunk_size` integer — Target chunk size, counted in the split_by strategy's own unit: characters for split_by='characters', words for 'words', sentences for 'sentences', paragraphs for 'paragraphs', pages for 'pages'. For example split_by='sentences' with chunk_size=5 produces chunks of 5 sentences. Not used for 'none' or 'time_segments' (use segment_length_seconds there).
        - `chunk_overlap` integer — Overlap between consecutive chunks, counted in the same unit as chunk_size (characters, words, sentences, paragraphs, or pages). For example split_by='sentences' with chunk_overlap=1 carries one sentence from the end of each chunk into the next.
        - `segment_length_seconds` integer — Length of each transcript segment in seconds (for time_segments split strategy). Shorter segments give more precise search results but more documents.
        - `language` string — Preferred language code for YouTube captions (when source_type='youtube').
        - `extract_captions` boolean — Extract auto-captions or manual subtitles from YouTube videos (when source_type='youtube'). Falls back to video description if False.
        - `response_shape` union — Define custom structured output using LLM extraction.
          - string
          - object
        - `llm_provider` string, nullable — LLM provider for structured extraction (openai, google, anthropic).
        - `llm_model` string, nullable — Specific LLM model for structured extraction.
        - `llm_api_key` string, nullable — API key for LLM operations (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
        - `embedding_model` 'laion_clip_vit_l_14_v1' | 'multilingual_e5_large_instruct_v1' | 'vertex_multimodal_embedding' | 'multimodalembedding@001' | 'gemini-embedding-2' | 'google_siglip_base_v1' | 'google_siglip_so400m_v1' | 'text-embedding-3-small' | 'text-embedding-3-large' | 'face_identity_arcface_r100_v1' | 'all_minilm_l6_v2_v1' — Embedding model identifiers. Format: {provider}_{model_name}_{version}
        - `embedding_task` string, nullable — Embedding task hint for instruction-aware models (E5, Gemini). Prefer setting this at collection level (embedding_task on the collection) rather than here. Collection-level overrides this value. Defaults to 'retrieval_document'. Values: retrieval_document, retrieval_query, semantic_similarity, classification, clustering.
      - UniversalExtractorParams — Parameters for the Universal Extractor.
        - `extractor_type` 'universal_extractor' — Discriminator field for parameter type identification.
        - `output_dimensionality` integer — Output embedding dimensions (Gemini Embedding 2 supports 256-3072).
        - `task_type` string — Embedding intent used as a text instruction for Gemini Embedding 2. Common values: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY.
        - `generate_description` boolean — Generate a text description of the content via Gemini vision/understanding.
        - `extract_text` boolean — Extract text content (OCR for images/docs, transcription for audio/video).
        - `max_video_segments` integer — Maximum number of 30s segments to process for video files.
        - `max_document_pages` integer — Maximum number of pages to process for document files.
        - `max_file_download_mb` integer — Maximum file download size in MB for Celery fast-path processing.
        - `max_concurrency` integer — Maximum per-task object concurrency for Celery fast-path processing.
      - WebScraperExtractorParams — Parameters for the web scraper extractor. The web scraper extractor crawls websites and extracts content with three types of embeddings for comprehensive multimodal search: **Embedding Types:** - Text (E5-Large): 1024D embeddings for page content - Code (Jina Code): 768D embeddings for code blocks - Images (SigLIP): 768D semantic embeddings for figures/screenshots - Images (DINOv2): 768D structure embeddings for visual layout comparison **Crawl Modes:** - DETERMINISTIC: BFS following all links (default, predictable) - SEMANTIC: LLM-guided, prioritizes pages matching crawl_goal **Rendering Strategies:** - STATIC: Fast HTTP fetch (default, works for most sites) - JAVASCRIPT: Playwright browser for SPAs (React/Vue/Angular) - AUTO: Tries static, falls back to JS if content too short **Use Cases:** - Documentation freshness: Crawl docs, compare against course content - Job board ingestion: Extract job listings with structured data - Knowledge base building: Convert websites to searchable collections - Code example indexing: Find API usage patterns across docs
        - `extractor_type` 'web_scraper' — Discriminator field for parameter type identification.
        - `max_depth` integer — Maximum link depth to crawl. 0=seed page only, 1=seed+direct links, etc. Default: 2. Max: 10.
        - `max_pages` integer — Maximum pages to crawl. Default: 50. Max: 500.
        - `crawl_timeout` integer — Maximum total time for crawling in seconds. Default: 300 (5 minutes). Increase for large sites with many pages. Max: 3600 (1 hour).
        - `crawl_mode` 'deterministic' | 'semantic' — Mode for crawling web pages. Values: DETERMINISTIC: BFS crawl following all links up to max_depth SEMANTIC: LLM-guided crawl prioritizing pages relevant to crawl_goal
        - `crawl_goal` string, nullable — Goal for semantic crawling. Only used when crawl_mode=SEMANTIC. Example: 'Find all S3 API documentation and examples'
        - `render_strategy` 'static' | 'javascript' | 'auto' — Strategy for rendering web pages. Values: STATIC: Fast HTTP fetch, works for most sites JAVASCRIPT: Browser rendering via Playwright for SPAs AUTO: Try static first, fall back to JS if content too short
        - `include_patterns` string[], nullable — Regex patterns for URLs to include. Example: ['/docs/', '/api/']
        - `exclude_patterns` string[], nullable — Regex patterns for URLs to exclude. Example: ['/blog/', '\.pdf$']
        - `chunk_strategy` 'none' | 'sentences' | 'paragraphs' | 'words' | 'characters' — Strategy for splitting page content into chunks.
        - `chunk_size` integer — Target size for each chunk (in units of chunk_strategy).
        - `chunk_overlap` integer — Overlap between chunks to preserve context.
        - `document_id_strategy` 'url' | 'position' | 'content' — Strategy for generating deterministic document IDs. Values: URL: hash(page_url + chunk_index) - stable across re-crawls POSITION: hash(seed_url + page_index + chunk_index) - order-based CONTENT: hash(content) - deduplicates identical content
        - `generate_text_embeddings` boolean — Generate E5 embeddings for text content.
        - `generate_code_embeddings` boolean — Generate Jina code embeddings for code blocks.
        - `generate_image_embeddings` boolean — Generate SigLIP embeddings for images/figures.
        - `generate_structure_embeddings` boolean — Generate DINOv2 visual structure embeddings for layout comparison.
        - `response_shape` union — Optional structured extraction schema. Natural language or JSON schema. Example: 'Extract API version, deprecated methods, and example code'
          - string
          - object
        - `llm_provider` string, nullable — LLM provider for structured extraction: openai, google, anthropic
        - `llm_model` string, nullable — LLM model for structured extraction.
        - `llm_api_key` string, nullable — API key for LLM operations (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
        - `max_retries` integer — Maximum retry attempts for failed HTTP requests. Uses exponential backoff with jitter. Default: 3.
        - `retry_base_delay` number — Base delay in seconds for retry backoff. Actual delay = base * 2^attempt + jitter. Default: 1.0.
        - `retry_max_delay` number — Maximum delay in seconds between retries. Default: 30.
        - `respect_retry_after` boolean — Respect Retry-After header from 429/503 responses. If False, uses exponential backoff instead. Default: True.
        - `proxies` string[], nullable — List of proxy URLs for rotation. Supports formats: 'http://host:port', 'http://user:pass@host:port', 'socks5://host:port'. Proxies rotate on errors or every N requests.
        - `rotate_proxy_on_error` boolean — Rotate to next proxy when request fails. Default: True.
        - `rotate_proxy_every_n_requests` integer — Rotate proxy every N requests (0 = disabled). Useful for avoiding IP-based rate limits. Default: 0 (disabled).
        - `captcha_service_provider` string, nullable — Captcha solving service provider: '2captcha', 'anti-captcha', 'capsolver'. If not set, captcha pages are skipped gracefully.
        - `captcha_service_api_key` string, nullable — API key for captcha solving service. Supports secret reference: '{{SECRET.captcha_api_key}}'. Required if captcha_service_provider is set.
        - `detect_captcha` boolean — Detect captcha challenges (Cloudflare, reCAPTCHA, hCaptcha). If detected and no solver configured, page is skipped. Default: True.
        - `persist_cookies` boolean — Persist cookies across requests within a crawl session. Useful for sites requiring authentication. Default: True.
        - `custom_headers` object, nullable — Custom HTTP headers to include in all requests. Example: {'Authorization': 'Bearer token', 'X-Custom': 'value'}
        - `youtube_mode` 'auto' | 'off' | 'force' — YouTube channel fast path. 'auto' (default): detect YouTube channel URLs and hand off to yt-dlp; other URLs run normal BFS. 'off': never hand off. 'force': treat every URL as a YouTube channel (useful for explicit channel buckets).
        - `youtube_max_videos` integer — Max videos to pull per channel enumeration. Default: 50.
        - `youtube_backfill_months` integer — Skip videos older than this many months. Default: 6.
        - `youtube_show_filter` string, nullable — Optional case-insensitive regex applied to video titles. Used when a channel hosts multiple shows and you only want one.
        - `youtube_download_videos` boolean — If true (default), download each video file so downstream processors can read it from the `video_path` field. If false, only metadata + captions are emitted.
        - `youtube_format_ladder` string — yt-dlp format selector. Defaults to a muxed-first ladder that avoids breaking on YouTube's DASH n-challenge when the current EJS signature solver is stale. See BUILD_LOG.md in the greenroom folder for the rationale.
        - `youtube_cookies_path` string, nullable — Optional path to a Netscape cookies file. Required for age-gated or members-only content.
        - `youtube_request_sleep` number — Seconds to sleep between yt-dlp requests. Default: 1.0.
        - `delay_between_requests` number — Delay in seconds between consecutive requests. Useful for polite crawling and avoiding rate limits. Default: 0 (no delay).
      - CustomPluginParams — Parameters for custom plugin extractors. This model accepts any extractor_type that doesn't match the builtin extractors, allowing custom plugins to define their own parameters.
        - `extractor_type` string, required — Custom plugin extractor type (plugin name)
    - `input_mappings` union — Mapping from extractor input names to source field paths. Tells the extractor which source fields to process. Single value: {'image': 'thumbnail_url'} maps one blob field to one extractor input. Array value: {'files': ['image', 'spec_pdf', 'description']} maps multiple blob fields to a single extractor input as a list — used by multi-file extractors like gemini_multifile_extractor that embed all blobs of an object into one embedding.
      - object
      - InputMapping[]
        - `input_key` string, required — Key used in the constructed inputs payload.
        - `source_type` 'payload' | 'literal' | 'vector' | 'blob' — Where the value for an input should be retrieved from.
        - `path` string, nullable — Dot-notation path when source_type is PAYLOAD or VECTOR. PAYLOAD paths resolve from the document's ROOT fields — each document/source_document dict IS the payload (e.g. path 'text' reads {'text': ...}); a {'payload': {...}} envelope is also accepted as a fallback.
        - `override` unknown
    - `field_passthrough` FieldPassthrough[] — NOT REQUIRED. List of specific fields to pass through from source to output documents. These fields are included alongside extractor-computed features (embeddings, detections, etc.). Empty list = only extractor outputs in documents (default behavior). With entries = specified fields + extractor outputs in documents. How It Works: 1. During processing, fields are extracted from source object/document 2. They appear in output documents at the root level 3. Field filtering happens automatically (only listed fields included) 4. Use target_path to rename fields for cleaner schemas Common Use Cases: - Preserve identifiers: campaign_id, product_sku, order_id - Keep metadata: category, tags, author, created_at - Enable filtering: department, status, priority, region - Maintain context: title, description, source_url Behavior: - Works with include_all_source_fields=False (default): ONLY these fields included - Works with include_all_source_fields=True: These configs used for renaming/defaults - Fields must exist in source bucket_schema or upstream collection output_schema - Missing optional fields are omitted (unless default provided) - Missing required fields cause processing errors Output Schema: output_schema = field_passthrough fields + extractor output fields Example: ['title', 'category', 'text_extractor_v1_embedding']
      - `source_path` string, required — REQUIRED. Path to the source field to copy. Simple fields: Use field name directly (e.g., 'title', 'campaign_id'). Nested fields: Use dot notation (e.g., 'metadata.author', 'config.model.version'). The field must exist in the source bucket schema or upstream collection schema. Without target_path, nested fields are flattened: 'metadata.author' becomes 'author' in output.
      - `target_path` string, nullable — OPTIONAL. Target field name in output document. If NOT PROVIDED: Uses source_path name (or last component for nested paths). - 'title' → 'title' - 'metadata.author' → 'author' If PROVIDED: Uses this exact name in output. - source_path='doc_title', target_path='title' → 'title' - source_path='metadata.author', target_path='contributor' → 'contributor' Use cases: - Rename fields for cleaner API schemas - Avoid name conflicts with extractor outputs - Standardize field names across different sources Constraints: - Must not conflict with system fields (document_id, collection_id, etc.) - Must not conflict with extractor output fields - Must be a valid field name (alphanumeric, underscores, hyphens)
      - `default` string, nullable — OPTIONAL. Default value if source field doesn't exist or is None. If NOT PROVIDED and field missing: Field is omitted from output document. If PROVIDED and field missing: Field is included with this default value. Type should match expected field type (string, int, list, dict, etc.).
      - `required` boolean — OPTIONAL. Whether this field MUST exist in source. If True and field missing: Raises validation error, processing fails. If False and field missing: Field omitted (or default used if provided). Use True for: Critical identifiers, required business fields. Use False for: Optional metadata, nice-to-have fields. Default: False (field is optional).
    - `include_all_source_fields` boolean — NOT REQUIRED. Whether to include ALL fields from source object/document in output. Default: False (only field_passthrough fields included). When False (RECOMMENDED): - Only fields listed in field_passthrough are included in output - Creates clean, predictable output schemas - Prevents data leakage of unwanted fields - Output = field_passthrough fields + extractor outputs When True (USE WITH CAUTION): - ALL source fields are included in output documents - field_passthrough still used for renaming/defaults/requirements - Can result in large documents if source has many fields - Can leak sensitive or unnecessary data - Output = all source fields + extractor outputs Use True When: - You want to preserve complete source data - Source has limited, well-defined fields - Downstream processing needs all context Use False When (MOST CASES): - You want clean, controlled output schemas - Source has many fields you don't need - You want explicit field selection - You're concerned about document size Examples: False: source={a,b,c,d} + passthrough=[a,b] → output={a,b,embedding} True: source={a,b,c,d} + passthrough=[a→x] → output={x,b,c,d,embedding}
    - `feature_extractor_id` string, required — Construct unique identifier for the feature extractor instance (name + version).
  - `source` SourceConfigOutput, required — Configuration for collection source (bucket(s) or collection). Collections can process data from two types of sources: 1. **Bucket Source**: Process raw objects from one or more buckets (first-stage processing) - Use this to create your initial collections from uploaded data - Can specify multiple buckets to consolidate data from different sources - All buckets must have compatible schemas (validated at creation) - Example: Videos from multiple regions → Frame extraction collection 2. **Collection Source**: Process documents from another collection (decomposition trees) - Use this to create multi-stage processing pipelines - Example: Frames collection → Scene detection collection Multi-Bucket Requirements: - All buckets must have compatible schemas (same fields, types, and required status) - Schema compatibility is validated when the collection is created - Documents track which specific bucket they came from via root_bucket_id - Useful for consolidating data from multiple regions, teams, or environments The source determines: - What data the feature extractor receives as input - The input_schema available for input_mappings and field_passthrough - The lineage tracking in output documents Examples: Single bucket: {"type": "bucket", "bucket_ids": ["bkt_products"]} Multi-bucket: {"type": "bucket", "bucket_ids": ["bkt_us", "bkt_eu", "bkt_asia"]} Collection: {"type": "collection", "collection_id": "col_frames"}
    - `type` 'bucket' | 'collection' | 'taxonomy' | 'cluster' | 'direct_upsert' | 'none', required — Source types for any document/point.
    - `bucket_ids` string[], nullable — List of bucket IDs when type='bucket'. REQUIRED when type='bucket'. NOT ALLOWED when type='collection'. Can specify one or more buckets to process. Single bucket: Use array with one element ['bkt_id']. Multiple buckets: All buckets MUST have compatible schemas. Schema compatibility validated at collection creation. Compatible schemas have: 1) Same field names, 2) Same field types, 3) Same required status. Documents will include root_bucket_id to track which bucket they came from. Use cases: multi-region data, multi-team consolidation, environment aggregation.
    - `source_namespace_id` string, nullable — Namespace ID where the source buckets reside. Use this to process buckets from a different namespace within the same organization. When omitted, buckets are looked up in the current (collection's) namespace. Only valid when type='bucket'.
    - `collection_id` string, nullable — Collection ID when type='collection' (single collection). Use this OR collection_ids (not both). REQUIRED when type='collection' and processing single collection. NOT ALLOWED when type='bucket'. The collection will process documents from this upstream collection. The upstream collection's output_schema becomes this collection's input_schema. This enables decomposition trees (multi-stage pipelines). Example: Process frames collection → create scenes collection.
    - `collection_ids` string[], nullable — List of collection IDs when type='collection' (multiple collections). Use this OR collection_id (not both). REQUIRED when type='collection' and processing multiple collections. NOT ALLOWED when type='bucket'. Used for operations that consolidate multiple upstream collections. Example: Clustering across multiple collections → cluster output collection. All collections must have compatible schemas for consolidation operations.
    - `inherited_bucket_ids` string[], nullable — List of original bucket IDs that source collections originated from. OPTIONAL. Only used when type='collection'. Tracks the complete lineage chain: buckets → collections → derived collections. Extracted from upstream collection metadata at collection creation time. Enables tracing derived collections (like cluster outputs) back to original data sources. Example: Cluster output collection inherits bucket IDs from its source collections. Format: List of bucket IDs with 'bkt_' prefix.
    - `source_filters` SourceFiltersOutput — Filters applied to source data when processing collections. Source filters determine which objects (from buckets) or documents (from collections) are processed by this collection. Filters use the same LogicalOperator model as list APIs throughout the system, supporting complex AND/OR/NOT logic. Use Cases: - Process only specific content types from mixed-content buckets - Filter by metadata fields (status, category, tags, dates) - Create specialized collections from broader sources - Exclude certain objects or documents from processing Examples: Process only video content: { "AND": [ {"field": "blobs.type", "operator": "eq", "value": "video"} ] } Process only active, published content: { "AND": [ {"field": "metadata.status", "operator": "eq", "value": "active"}, {"field": "metadata.published", "operator": "eq", "value": true} ] } Process content from last 30 days: { "AND": [ {"field": "created_at", "operator": "gte", "value": "2025-10-08T00:00:00Z"} ] } Process specific brands OR categories: { "OR": [ {"field": "brand_name", "operator": "in", "value": ["Acme", "TechCo"]}, {"field": "category", "operator": "eq", "value": "premium"} ] } Filter Operators: - eq (equals) - ne (not equals) - gt (greater than) - gte (greater than or equal) - lt (less than) - lte (less than or equal) - in (value in list) - nin (value not in list) - contains (string contains) - starts_with (string starts with) - ends_with (string ends with) Performance Considerations: - Filters are evaluated at batch creation time - Only matching objects/documents are included in processing - More selective filters = smaller batches = faster processing - Use indexed fields (metadata, timestamps) for better performance Relationship to Batch Filters: - Source filters: Applied at collection definition (consistent across all batches) - Batch filters: Applied at batch creation (ad-hoc, per-batch basis) - Both can be used together: source filters + batch filters = intersection
      - `filters` LogicalOperatorOutput — Represents a logical operation (AND, OR, NOT) on filter conditions. Allows nesting with a defined depth limit. Also supports shorthand syntax where field names can be passed directly as key-value pairs for equality filtering (e.g., {"metadata.title": "value"}).
        - `AND` union[], nullable — Logical AND operation - all conditions must be true
          - union
            - LogicalOperatorOutput — recursive
            - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
              - …
        - `OR` union[], nullable — Logical OR operation - at least one condition must be true
          - union
            - LogicalOperatorOutput — recursive
            - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
              - …
        - `NOT` union[], nullable — Logical NOT operation - all conditions must be false
          - union
            - LogicalOperatorOutput — recursive
            - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
              - …
        - `case_sensitive` boolean, nullable — Whether to perform case-sensitive matching
  - `source_bucket_schemas` object, nullable — NOT REQUIRED (auto-computed). Snapshot of bucket schemas at collection creation. Only populated for multi-bucket collections (source.type='bucket' with multiple bucket_ids). Key: bucket_id, Value: BucketSchema at time of collection creation. Used for: Schema compatibility validation, document lineage, debugging. Schema snapshot is immutable - bucket schema changes after collection creation do not affect this. Single-bucket collections may omit this field (schema in input_schema is sufficient).
  - `source_lineage` SingleLineageEntry[] — NOT REQUIRED (auto-computed). Lineage chain showing complete processing history. Each entry contains: source_config, feature_extractor, output_schema for one tier. Length indicates processing depth (1 = tier 1, 2 = tier 2, etc.). Use for: Understanding multi-tier pipelines, visualizing decomposition trees.
    - `source_config` SourceConfigOutput, required — Configuration for collection source (bucket(s) or collection). Collections can process data from two types of sources: 1. **Bucket Source**: Process raw objects from one or more buckets (first-stage processing) - Use this to create your initial collections from uploaded data - Can specify multiple buckets to consolidate data from different sources - All buckets must have compatible schemas (validated at creation) - Example: Videos from multiple regions → Frame extraction collection 2. **Collection Source**: Process documents from another collection (decomposition trees) - Use this to create multi-stage processing pipelines - Example: Frames collection → Scene detection collection Multi-Bucket Requirements: - All buckets must have compatible schemas (same fields, types, and required status) - Schema compatibility is validated when the collection is created - Documents track which specific bucket they came from via root_bucket_id - Useful for consolidating data from multiple regions, teams, or environments The source determines: - What data the feature extractor receives as input - The input_schema available for input_mappings and field_passthrough - The lineage tracking in output documents Examples: Single bucket: {"type": "bucket", "bucket_ids": ["bkt_products"]} Multi-bucket: {"type": "bucket", "bucket_ids": ["bkt_us", "bkt_eu", "bkt_asia"]} Collection: {"type": "collection", "collection_id": "col_frames"}
      - `type` 'bucket' | 'collection' | 'taxonomy' | 'cluster' | 'direct_upsert' | 'none', required — Source types for any document/point.
      - `bucket_ids` string[], nullable — List of bucket IDs when type='bucket'. REQUIRED when type='bucket'. NOT ALLOWED when type='collection'. Can specify one or more buckets to process. Single bucket: Use array with one element ['bkt_id']. Multiple buckets: All buckets MUST have compatible schemas. Schema compatibility validated at collection creation. Compatible schemas have: 1) Same field names, 2) Same field types, 3) Same required status. Documents will include root_bucket_id to track which bucket they came from. Use cases: multi-region data, multi-team consolidation, environment aggregation.
      - `source_namespace_id` string, nullable — Namespace ID where the source buckets reside. Use this to process buckets from a different namespace within the same organization. When omitted, buckets are looked up in the current (collection's) namespace. Only valid when type='bucket'.
      - `collection_id` string, nullable — Collection ID when type='collection' (single collection). Use this OR collection_ids (not both). REQUIRED when type='collection' and processing single collection. NOT ALLOWED when type='bucket'. The collection will process documents from this upstream collection. The upstream collection's output_schema becomes this collection's input_schema. This enables decomposition trees (multi-stage pipelines). Example: Process frames collection → create scenes collection.
      - `collection_ids` string[], nullable — List of collection IDs when type='collection' (multiple collections). Use this OR collection_id (not both). REQUIRED when type='collection' and processing multiple collections. NOT ALLOWED when type='bucket'. Used for operations that consolidate multiple upstream collections. Example: Clustering across multiple collections → cluster output collection. All collections must have compatible schemas for consolidation operations.
      - `inherited_bucket_ids` string[], nullable — List of original bucket IDs that source collections originated from. OPTIONAL. Only used when type='collection'. Tracks the complete lineage chain: buckets → collections → derived collections. Extracted from upstream collection metadata at collection creation time. Enables tracing derived collections (like cluster outputs) back to original data sources. Example: Cluster output collection inherits bucket IDs from its source collections. Format: List of bucket IDs with 'bkt_' prefix.
      - `source_filters` SourceFiltersOutput — Filters applied to source data when processing collections. Source filters determine which objects (from buckets) or documents (from collections) are processed by this collection. Filters use the same LogicalOperator model as list APIs throughout the system, supporting complex AND/OR/NOT logic. Use Cases: - Process only specific content types from mixed-content buckets - Filter by metadata fields (status, category, tags, dates) - Create specialized collections from broader sources - Exclude certain objects or documents from processing Examples: Process only video content: { "AND": [ {"field": "blobs.type", "operator": "eq", "value": "video"} ] } Process only active, published content: { "AND": [ {"field": "metadata.status", "operator": "eq", "value": "active"}, {"field": "metadata.published", "operator": "eq", "value": true} ] } Process content from last 30 days: { "AND": [ {"field": "created_at", "operator": "gte", "value": "2025-10-08T00:00:00Z"} ] } Process specific brands OR categories: { "OR": [ {"field": "brand_name", "operator": "in", "value": ["Acme", "TechCo"]}, {"field": "category", "operator": "eq", "value": "premium"} ] } Filter Operators: - eq (equals) - ne (not equals) - gt (greater than) - gte (greater than or equal) - lt (less than) - lte (less than or equal) - in (value in list) - nin (value not in list) - contains (string contains) - starts_with (string starts with) - ends_with (string ends with) Performance Considerations: - Filters are evaluated at batch creation time - Only matching objects/documents are included in processing - More selective filters = smaller batches = faster processing - Use indexed fields (metadata, timestamps) for better performance Relationship to Batch Filters: - Source filters: Applied at collection definition (consistent across all batches) - Batch filters: Applied at batch creation (ad-hoc, per-batch basis) - Both can be used together: source filters + batch filters = intersection
        - `filters` LogicalOperatorOutput — Represents a logical operation (AND, OR, NOT) on filter conditions. Allows nesting with a defined depth limit. Also supports shorthand syntax where field names can be passed directly as key-value pairs for equality filtering (e.g., {"metadata.title": "value"}).
          - `AND` union[], nullable — Logical AND operation - all conditions must be true
            - union
              - …
          - `OR` union[], nullable — Logical OR operation - at least one condition must be true
            - union
              - …
          - `NOT` union[], nullable — Logical NOT operation - all conditions must be false
            - union
              - …
          - `case_sensitive` boolean, nullable — Whether to perform case-sensitive matching
    - `feature_extractor` SharedCollectionFeaturesExtractorsModelsFeatureExtractorConfigOutput, required — Configuration for a feature extractor with field passthrough support. A feature extractor processes source data (from buckets or collections) and produces features (embeddings, extracted text, detected objects, etc.). With field passthrough, you can also include selected source fields in the output documents alongside the computed features. Core Concepts: 1. **Feature Extraction**: Extractors compute features from input data (e.g., text → embeddings, image → detections, video → scenes) 2. **Field Passthrough**: Selectively preserve source fields in output (e.g., title, category, campaign_id from source → output documents) 3. **Output Schema**: Combination of passed-through fields + extractor outputs (e.g., {title, category, text_embedding} all in one document) How Field Passthrough Works: 1. Define which source fields to include via field_passthrough list 2. During processing, these fields are extracted from source 3. They appear in output documents at root level 4. Combine with extractor outputs for complete documents 5. Use target_path to rename fields for cleaner schemas Field Selection Modes: - **Explicit** (field_passthrough + include_all=False): Only listed fields pass through. Clean, controlled output. Example: passthrough=[title, category] → output has ONLY title, category, embedding - **Inclusive** (include_all=True): All source fields pass through, field_passthrough for renaming. Example: source has 10 fields → output has all 10 + embedding - **None** (no field_passthrough): Only extractor outputs in documents. Example: → output has ONLY embedding (no source fields) Use Cases: - **Preserve Identifiers**: Keep campaign_id, product_sku, order_id for tracking - **Enable Filtering**: Pass category, status, department for query filters - **Maintain Context**: Include title, description for display - **Track Metadata**: Preserve author, created_at, source for lineage - **Business Logic**: Keep priority, region, type for application logic Common Patterns: 1. **Minimal Passthrough** (recommended): field_passthrough=[{"source_path": "id"}], include_all=False → Clean output, only ID + extractor features 2. **Metadata Preservation**: field_passthrough=[ {"source_path": "title"}, {"source_path": "category"}, {"source_path": "created_at"} ] → Document has context for display and filtering 3. **Field Renaming**: field_passthrough=[ {"source_path": "doc_title", "target_path": "title"}, {"source_path": "metadata.author", "target_path": "author"} ] → Cleaner output schema with flattened fields 4. **Required Fields**: field_passthrough=[ {"source_path": "campaign_id", "required": True}, {"source_path": "priority", "default": 0} ] → Ensures critical fields always present Requirements: - feature_extractor_name: REQUIRED - name of the extractor - version: REQUIRED - extractor version (e.g., "v1") - parameters: NOT REQUIRED - extractor-specific config (model, thresholds, etc.) - input_mappings: NOT REQUIRED - maps extractor inputs to source fields - field_passthrough: NOT REQUIRED - which source fields to preserve (default: none) - include_all_source_fields: NOT REQUIRED - preserve all fields (default: false)
      - `feature_extractor_name` string, required — Name of the feature extractor
      - `version` string, required — Version of the feature extractor (e.g., 'v1', 'v2')
      - `params` object, nullable — Optional extractor parameters that affect vector index configuration. Parameters set here are locked at namespace creation and determine vector dimensions in Qdrant. Collections using this extractor must use compatible params. Example: {'model': 'siglip_base'}
      - `parameters` union — Parameters for the feature extractor. Each extractor type has specific parameters. See the schema for your chosen extractor (e.g., MultimodalExtractorParams for multimodal_extractor).
        - AudioFingerprintExtractorParams — Parameters for the Audio Fingerprint Extractor. Processes audio files (or audio extracted from video) through CLAP (Contrastive Language-Audio Pretraining) to produce 512-d embeddings suitable for audio fingerprint matching. Core Pipeline: 1. Audio extraction (if video input, via FFmpeg) 2. Segmentation into fixed-length windows 3. CLAP embedding (laion/clap-htsat-tiny, 512-d) 4. L2 normalization Use Cases: - Sound mark detection (IP safety) - Audio similarity search - Music/jingle identification - Audio deduplication
          - `extractor_type` 'audio_fingerprint_extractor' — Discriminator field. Must be 'audio_fingerprint_extractor'.
          - `segment_duration_sec` number — Duration of each audio segment in seconds. 5.0: Recommended for sound mark matching. Shorter segments increase recall but reduce per-segment context.
          - `segment_hop_sec` number — Hop size between segments in seconds. 2.5: 50% overlap (recommended). Set equal to segment_duration_sec for no overlap.
          - `sample_rate` integer — Target sample rate for audio. 48000: CLAP default (recommended). Audio is resampled to this rate before embedding.
          - `normalize_embeddings` boolean — L2-normalize embeddings to unit vectors (recommended for cosine similarity).
          - `max_audio_length_sec` number — Maximum audio length to process in seconds. 120: Default (2 minutes). Audio beyond this is truncated.
        - DocumentGraphExtractorParams — Parameters for the document graph extractor. This extractor decomposes PDFs into spatial blocks with layout classification, confidence scoring, and optional VLM correction for degraded documents. **When to Use**: - Historical/archival document processing (FBI files, old records) - Scanned documents with mixed quality - Documents requiring spatial understanding (forms, tables, multi-column) - When you need block-level granularity with bounding boxes - When confidence scoring is needed for downstream filtering **When NOT to Use**: - Simple text-only documents -> Use text_extractor instead - When page-level granularity is sufficient -> Use pdf_extractor instead - Real-time processing requirements -> VLM correction adds latency
          - `extractor_type` 'document_graph_extractor' — Discriminator field for parameter type identification. Must be 'document_graph_extractor'.
          - `use_layout_detection` boolean — Enable ML-based layout detection to find ALL document elements (text, images, tables, figures). When enabled, uses the configured layout_detector to detect and extract both text regions AND non-text elements (scanned images, figures, charts) as separate documents. **Recommended for**: Scanned documents, image-heavy PDFs, mixed content documents. **When disabled**: Falls back to text-only extraction (faster but misses images). Default: True (detects all elements including images).
          - `layout_detector` 'pymupdf' | 'docling' — Layout detection engine to use when use_layout_detection=True. 'pymupdf': Fast, rule-based detection using PyMuPDF heuristics (~15 pages/sec). 'docling': SOTA ML-based detection using IBM Docling with DiT model (~3-8 sec/doc). **Docling advantages**: Better semantic type detection (section_header vs paragraph), true table structure extraction (rows/cols), more accurate figure detection. **PyMuPDF advantages**: Much faster, lower memory usage, simpler dependencies. Default: 'pymupdf' for speed. Use 'docling' for accuracy-critical applications.
          - `vertical_threshold` number — Maximum vertical gap (in points) between lines to be grouped in same block. Increase for looser grouping, decrease for tighter blocks. Default 15pt works well for standard documents.
          - `horizontal_threshold` number — Maximum horizontal distance (in points) for overlap detection. Affects column detection and block merging. Increase for wider columns, decrease for narrow layouts.
          - `min_text_length` integer — Minimum text length (characters) to keep a block. Blocks with less text are filtered out. Helps remove noise and tiny fragments.
          - `base_confidence` number — Base confidence score for embedded (native) text. Penalties are subtracted for OCR artifacts, encoding issues, etc.
          - `min_confidence_for_vlm` number — Confidence threshold below which VLM correction is triggered. Blocks with confidence < this value get sent to VLM for correction. Only applies when use_vlm_correction=True.
          - `use_vlm_correction` boolean — Enable VLM (Vision Language Model) correction for low-confidence blocks. Uses Gemini/GPT-4V to correct OCR errors by analyzing the page image. Significantly slower (~1 page/sec) but improves accuracy for degraded docs.
          - `fast_mode` boolean — Skip VLM correction entirely for maximum throughput (~15 pages/sec). Overrides use_vlm_correction. Use when speed is more important than accuracy.
          - `vlm_provider` string — LLM provider for VLM correction. Options: 'google' (Gemini), 'openai' (GPT-4V), 'anthropic' (Claude). Google recommended for best vision quality.
          - `vlm_model` string — Specific model for VLM correction. Examples: 'gemini-2.5-flash', 'gpt-4o', 'claude-3-5-sonnet'.
          - `llm_api_key` string, nullable — API key for VLM correction (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
          - `run_text_embedding` boolean — Generate text embeddings for semantic search over block content. Uses E5-Large (1024-dim) for multilingual support.
          - `render_dpi` integer — DPI for page rendering (used for VLM correction). 72: Fast, lower quality. 150: Balanced (recommended). 300: High quality, slower.
          - `generate_thumbnails` boolean — Generate thumbnail images for blocks. Useful for visual previews and UI display.
          - `thumbnail_mode` string — Thumbnail generation mode. 'full_page': Low-res thumbnail of entire page. 'segment': Cropped thumbnail of just the block's bounding box. 'both': Generate both types (recommended for flexibility).
          - `thumbnail_dpi` integer — DPI for thumbnail generation. Lower DPI = smaller files. 72: Standard web quality. 36: Very small thumbnails.
        - FaceIdentityExtractorParams — Parameters for the Face Identity Extractor. The Face Identity Extractor processes images or video frames to detect, align, and embed faces using production-grade SOTA models (SCRFD + ArcFace). Core Pipeline: 1. SCRFD Detection → Bounding boxes + 5 landmarks 2. 5-Point Affine Alignment → 112×112 canonical face 3. ArcFace Embedding → 512-d L2-normalized vector 4. Optional Quality Scoring → Filter low-quality faces Use Cases: - Face verification (1:1 matching) - Face identification (1:N search) - Face clustering (group photos by person) - Duplicate face detection
          - `extractor_type` 'face_identity_extractor' — Discriminator field for parameter type identification. Must be 'face_identity_extractor'.
          - `detection_model` 'scrfd_500m' | 'scrfd_2.5g' | 'scrfd_10g' — SCRFD model for face detection. 'scrfd_500m': Fastest (2-3ms). 'scrfd_2.5g': Balanced (5-7ms), recommended. 'scrfd_10g': Highest accuracy (10-15ms).
          - `min_face_size` integer — Minimum face size in pixels to detect. 20px: Balanced. 40px: Higher quality. 10px: Maximum recall.
          - `detection_threshold` number — Confidence threshold for face detection (0.0-1.0).
          - `max_faces_per_image` integer, nullable — Maximum number of faces to process per image. None: Process all.
          - `normalize_embeddings` boolean — L2-normalize embeddings to unit vectors (recommended).
          - `enable_quality_scoring` boolean — Compute quality scores (blur, size, landmarks). Adds ~5ms per face.
          - `quality_threshold` number, nullable — Minimum quality score to index faces. None: Index all faces. 0.5: Moderate filtering. 0.7: High quality only.
          - `max_video_length` integer — Maximum video length in seconds. 60: Default. 10: Recommended for retrieval. 300: Maximum (extraction only).
          - `video_sampling_fps` number, nullable — Frames per second to sample from video. 1.0: One frame per second (recommended).
          - `video_deduplication` boolean — Remove duplicate faces across video frames (extraction only). Reduces 90-95% redundancy. NOT used in retrieval.
          - `video_deduplication_threshold` number — Cosine similarity threshold for deduplication. 0.8: Conservative (default).
          - `output_mode` 'per_face' | 'per_image' — 'per_face': One document per face (recommended). 'per_image': One doc per image with faces array.
          - `include_face_crops` boolean — Include aligned 112×112 face crops as base64. Adds ~5KB per face. Required for LLM cluster labeling to see actual faces instead of hallucinating.
          - `include_source_frame_thumbnail` boolean — Include resized source frame/image as base64 thumbnail (~15-30KB per face). Used for display with bounding box overlay.
          - `store_detection_metadata` boolean — Store bbox, landmarks, detection scores. Recommended for debugging.
        - GeminiMultifileExtractorParams — Parameters for the Gemini Multifile Extractor. Uses Gemini Embedding 2 (gemini-embedding-2) to embed all files of an object into a single 3072-d vector in one API call. Supports images, video, audio, PDF, and text blobs.
          - `extractor_type` 'gemini_multifile_extractor' — Discriminator field for parameter type identification.
          - `output_dimensionality` integer — Output embedding dimensions. Gemini Embedding 2 supports 3072 (default), 768, or 256 via truncation. Lower dimensions reduce storage cost at slight quality loss.
          - `task_type` string — Embedding intent used as a text instruction for Gemini Embedding 2. Common values: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY, CLASSIFICATION.
          - `input_key` string — The input_mappings key whose value is the list of blob fields to embed together. Must match the key used in input_mappings (e.g., 'files'). Default: 'files'.
        - ImageExtractorParams — Parameters for the Image Extractor.
          - `extractor_type` 'image_extractor' — Discriminator field for parameter type identification.
          - `enable_thumbnails` boolean — Whether to generate thumbnail images.
          - `use_cdn` boolean — Whether to use CloudFront CDN for thumbnail delivery.
        - MultimodalExtractorParams — Parameters for multimodal extractor v2. Same pipeline as v1 but uses Gemini Embedding 2 (3072D) for the multimodal embedding step. Supports configurable output dimensions via Matryoshka representation learning (3072/1536/768).
          - `extractor_type` 'multimodal_extractor' — Discriminator field. Must be 'multimodal_extractor'.
          - `split_method` 'time' | 'scene' | 'silence'
          - `description_prompt` string — Prompt for description generation.
          - `time_split_interval` integer, nullable — Interval in seconds for 'time' splitting.
          - `silence_db_threshold` integer, nullable — Decibel threshold for silence detection. Recommended: -40.
          - `scene_detection_threshold` number, nullable — Scene detection threshold (0.0-1.0). Recommended: 0.5.
          - `run_transcription` boolean — Run Whisper transcription on segments.
          - `transcription_language` string — Transcription language code.
          - `run_video_description` boolean — Generate Gemini descriptions for segments.
          - `run_transcription_embedding` boolean — Generate E5 embeddings for transcriptions (1024D).
          - `run_ocr_embedding` boolean — Generate E5 embeddings for OCR text (1024D). Requires run_ocr.
          - `run_description_embedding` boolean — Generate E5 embeddings for descriptions (1024D). Requires run_video_description.
          - `run_multimodal_embedding` boolean — Generate Gemini Embedding 2 multimodal embeddings (3072D). Creates unified embeddings across video, image, text, audio, and GIF content.
          - `run_ocr` boolean — Extract text from video frames via Gemini OCR.
          - `max_segment_duration` number, nullable — Maximum duration in seconds for any single segment. Scene/silence segments longer than this are subdivided. Set to None to disable. Default: 30s.
          - `sensitivity` string — Scene detection sensitivity.
          - `enable_thumbnails` boolean — Generate thumbnail images for segments.
          - `use_cdn` boolean — Use CloudFront CDN for thumbnail delivery.
          - `generation_config` GenerationConfig — Configuration for generative models.
            - `candidate_count` integer — Number of candidate responses to generate for video description.
            - `max_output_tokens` integer — Maximum number of tokens for the generated video description.
            - `temperature` number — Controls randomness for video description generation. Higher is more random.
            - `top_p` number — Nucleus sampling (top-p) for video description generation.
            - `response_mime_type` string, nullable — MIME type for response (e.g., 'application/json')
            - `response_schema` object, nullable — JSON schema for structured output
          - `output_dimensionality` integer — Output embedding dimensions. Gemini Embedding 2 supports Matryoshka dimension reduction: 3072 (full), 1536, or 768.
          - `task_type` string — Embedding task type hint. Options: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY, CLASSIFICATION.
          - `response_shape` union — Custom structured output schema for Gemini extraction. String for natural language prompt, dict for explicit JSON schema.
            - string
            - object
          - `embedding_task` string, nullable — Embedding task hint for instruction-aware models (E5). Prefer setting this at collection level (embedding_task on the collection) rather than here. Collection-level overrides this value. Defaults to 'retrieval_document'. Values: retrieval_document, retrieval_query, semantic_similarity, classification, clustering. Note: Vertex AI multimodal embeddings ignore this — only E5 transcription embeddings use it.
        - PassthroughExtractorParams — Parameters for passthrough extractor. Minimal configuration - just passes data through with canonicalization.
          - `extractor_type` 'passthrough_extractor' — Discriminator field for parameter type identification.
          - `preserve_metadata` boolean — Preserve source object metadata in output document.
        - ScrollingTextExtractorParams — Parameters for the on-screen text extractor. Extracts ALL on-screen text from video: STATIC overlays (title cards, disclaimers, captions, CTAs — the majority of ad-creative text) via keyframe-grid VLM OCR, and SCROLLING/marquee text via computer vision (phase-correlation band detection + panoramic stitching) + VLM OCR. **When to Use**: - Ad creative with static overlay text (disclaimers, CTAs, titles) - Video ads with scrolling promotional banners or tickers - News broadcasts with scrolling chyrons or tickers - Videos with terms & conditions or disclaimers (static or scrolling) - Social media content with text overlays - Credits sequences in film/TV content - Live event streams with scrolling info bars **When NOT to Use**: - Spoken content → use multimodal_extractor with run_transcription=True - Text documents/PDFs → use text_extractor **How It Works**: Static pass (static_ocr=True, default): 1. Select visually distinct keyframes (identical frames collapse to one) 2. Stitch them into a single grid image 3. OCR the grid with one vision-language-model call (Gemini) Scrolling pass: 1. Sample frames from video at configurable FPS 2. Split each frame into horizontal and vertical strips 3. Phase-correlate consecutive frames to measure per-strip pixel shift 4. Strips with consistent shift in one direction = scrolling band 5. Stitch the band across frames into a single wide/tall panorama image 6. OCR the panorama using a vision language model (Gemini) 7. Deduplicate repeated marquee loops (e.g. "SALE • SALE • SALE •" → "SALE") **Performance**: - Processing speed: ~2-5x realtime (depends on video resolution and FPS) - Accuracy: Best with consistent scroll speed; handles variable speed with degradation - Minimum video length: ~2 seconds (needs 3+ frames for correlation) **Supported Scroll Directions**: - Horizontal: Right-to-left (most common), Left-to-right - Vertical: Bottom-to-top (credits), Top-to-bottom
          - `extractor_type` 'scrolling_text_extractor' — Discriminator field. Must be 'scrolling_text_extractor'.
          - `fps` number — Frame sampling rate for analysis. Higher values improve detection accuracy for fast-scrolling text but increase processing time. 5 FPS works well for most video ads and tickers.
          - `strip_height` integer — Height (in pixels) of each scanning strip used for phase correlation. Should roughly match the height of the scrolling text band. Smaller values detect narrower text bands; larger values are more robust but may miss thin tickers. 40px works for most standard video ads.
          - `min_shift_px` number — Minimum pixel shift per frame to consider a strip as 'scrolling'. Lower values detect slower-moving text; higher values filter out noise. 2.0px is a good default for 5 FPS sampling.
          - `consistency_ratio` number — Fraction of frame pairs that must show consistent shift for a band to be classified as scrolling. 0.6 means 60% of frames must agree. Lower values detect intermittent scrolling; higher values reduce false positives.
          - `pad` integer — Pixel padding above/below detected band when cropping for stitching.
          - `static_ocr` boolean — Also OCR STATIC on-screen text (title cards, disclaimers, CTAs, captions) — the majority of text in ad creative. Distinct keyframes are stitched into one grid image and read with a single VLM call, so a fully static video costs one OCR call. Disable only if you strictly want scrolling-band text.
          - `max_static_frames` integer — Maximum number of visually distinct keyframes sampled for the static-text OCR grid. Identical frames collapse to one; scene changes contribute additional frames up to this cap.
        - TextExtractorParams — Parameters for the text extractor. The text extractor generates dense vector embeddings optimized for semantic similarity search. It uses the E5-Large multilingual model to convert text into 1024-dimensional vectors. When ``source_type`` is ``"youtube"``, the extractor first resolves YouTube URLs to caption text via yt-dlp before chunking and embedding. Use ``split_by="time_segments"`` with ``segment_length_seconds`` to segment captions by time window.
          - `extractor_type` 'text_extractor' — Discriminator field for parameter type identification.
          - `source_type` 'text' | 'youtube' — Source content type. Use 'youtube' to resolve YouTube URLs to caption text before embedding. Default: 'text' (plain text input).
          - `split_by` 'characters' | 'words' | 'sentences' | 'paragraphs' | 'pages' | 'time_segments' | 'none' — Strategy for splitting text into chunks.
          - `chunk_size` integer — Target chunk size, counted in the split_by strategy's own unit: characters for split_by='characters', words for 'words', sentences for 'sentences', paragraphs for 'paragraphs', pages for 'pages'. For example split_by='sentences' with chunk_size=5 produces chunks of 5 sentences. Not used for 'none' or 'time_segments' (use segment_length_seconds there).
          - `chunk_overlap` integer — Overlap between consecutive chunks, counted in the same unit as chunk_size (characters, words, sentences, paragraphs, or pages). For example split_by='sentences' with chunk_overlap=1 carries one sentence from the end of each chunk into the next.
          - `segment_length_seconds` integer — Length of each transcript segment in seconds (for time_segments split strategy). Shorter segments give more precise search results but more documents.
          - `language` string — Preferred language code for YouTube captions (when source_type='youtube').
          - `extract_captions` boolean — Extract auto-captions or manual subtitles from YouTube videos (when source_type='youtube'). Falls back to video description if False.
          - `response_shape` union — Define custom structured output using LLM extraction.
            - string
            - object
          - `llm_provider` string, nullable — LLM provider for structured extraction (openai, google, anthropic).
          - `llm_model` string, nullable — Specific LLM model for structured extraction.
          - `llm_api_key` string, nullable — API key for LLM operations (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
          - `embedding_model` 'laion_clip_vit_l_14_v1' | 'multilingual_e5_large_instruct_v1' | 'vertex_multimodal_embedding' | 'multimodalembedding@001' | 'gemini-embedding-2' | 'google_siglip_base_v1' | 'google_siglip_so400m_v1' | 'text-embedding-3-small' | 'text-embedding-3-large' | 'face_identity_arcface_r100_v1' | 'all_minilm_l6_v2_v1' — Embedding model identifiers. Format: {provider}_{model_name}_{version}
          - `embedding_task` string, nullable — Embedding task hint for instruction-aware models (E5, Gemini). Prefer setting this at collection level (embedding_task on the collection) rather than here. Collection-level overrides this value. Defaults to 'retrieval_document'. Values: retrieval_document, retrieval_query, semantic_similarity, classification, clustering.
        - UniversalExtractorParams — Parameters for the Universal Extractor.
          - `extractor_type` 'universal_extractor' — Discriminator field for parameter type identification.
          - `output_dimensionality` integer — Output embedding dimensions (Gemini Embedding 2 supports 256-3072).
          - `task_type` string — Embedding intent used as a text instruction for Gemini Embedding 2. Common values: RETRIEVAL_DOCUMENT, RETRIEVAL_QUERY, SEMANTIC_SIMILARITY.
          - `generate_description` boolean — Generate a text description of the content via Gemini vision/understanding.
          - `extract_text` boolean — Extract text content (OCR for images/docs, transcription for audio/video).
          - `max_video_segments` integer — Maximum number of 30s segments to process for video files.
          - `max_document_pages` integer — Maximum number of pages to process for document files.
          - `max_file_download_mb` integer — Maximum file download size in MB for Celery fast-path processing.
          - `max_concurrency` integer — Maximum per-task object concurrency for Celery fast-path processing.
        - WebScraperExtractorParams — Parameters for the web scraper extractor. The web scraper extractor crawls websites and extracts content with three types of embeddings for comprehensive multimodal search: **Embedding Types:** - Text (E5-Large): 1024D embeddings for page content - Code (Jina Code): 768D embeddings for code blocks - Images (SigLIP): 768D semantic embeddings for figures/screenshots - Images (DINOv2): 768D structure embeddings for visual layout comparison **Crawl Modes:** - DETERMINISTIC: BFS following all links (default, predictable) - SEMANTIC: LLM-guided, prioritizes pages matching crawl_goal **Rendering Strategies:** - STATIC: Fast HTTP fetch (default, works for most sites) - JAVASCRIPT: Playwright browser for SPAs (React/Vue/Angular) - AUTO: Tries static, falls back to JS if content too short **Use Cases:** - Documentation freshness: Crawl docs, compare against course content - Job board ingestion: Extract job listings with structured data - Knowledge base building: Convert websites to searchable collections - Code example indexing: Find API usage patterns across docs
          - `extractor_type` 'web_scraper' — Discriminator field for parameter type identification.
          - `max_depth` integer — Maximum link depth to crawl. 0=seed page only, 1=seed+direct links, etc. Default: 2. Max: 10.
          - `max_pages` integer — Maximum pages to crawl. Default: 50. Max: 500.
          - `crawl_timeout` integer — Maximum total time for crawling in seconds. Default: 300 (5 minutes). Increase for large sites with many pages. Max: 3600 (1 hour).
          - `crawl_mode` 'deterministic' | 'semantic' — Mode for crawling web pages. Values: DETERMINISTIC: BFS crawl following all links up to max_depth SEMANTIC: LLM-guided crawl prioritizing pages relevant to crawl_goal
          - `crawl_goal` string, nullable — Goal for semantic crawling. Only used when crawl_mode=SEMANTIC. Example: 'Find all S3 API documentation and examples'
          - `render_strategy` 'static' | 'javascript' | 'auto' — Strategy for rendering web pages. Values: STATIC: Fast HTTP fetch, works for most sites JAVASCRIPT: Browser rendering via Playwright for SPAs AUTO: Try static first, fall back to JS if content too short
          - `include_patterns` string[], nullable — Regex patterns for URLs to include. Example: ['/docs/', '/api/']
          - `exclude_patterns` string[], nullable — Regex patterns for URLs to exclude. Example: ['/blog/', '\.pdf$']
          - `chunk_strategy` 'none' | 'sentences' | 'paragraphs' | 'words' | 'characters' — Strategy for splitting page content into chunks.
          - `chunk_size` integer — Target size for each chunk (in units of chunk_strategy).
          - `chunk_overlap` integer — Overlap between chunks to preserve context.
          - `document_id_strategy` 'url' | 'position' | 'content' — Strategy for generating deterministic document IDs. Values: URL: hash(page_url + chunk_index) - stable across re-crawls POSITION: hash(seed_url + page_index + chunk_index) - order-based CONTENT: hash(content) - deduplicates identical content
          - `generate_text_embeddings` boolean — Generate E5 embeddings for text content.
          - `generate_code_embeddings` boolean — Generate Jina code embeddings for code blocks.
          - `generate_image_embeddings` boolean — Generate SigLIP embeddings for images/figures.
          - `generate_structure_embeddings` boolean — Generate DINOv2 visual structure embeddings for layout comparison.
          - `response_shape` union — Optional structured extraction schema. Natural language or JSON schema. Example: 'Extract API version, deprecated methods, and example code'
            - string
            - object
          - `llm_provider` string, nullable — LLM provider for structured extraction: openai, google, anthropic
          - `llm_model` string, nullable — LLM model for structured extraction.
          - `llm_api_key` string, nullable — API key for LLM operations (BYOK - Bring Your Own Key). Supports: - Direct key: 'sk-proj-abc123...' - Secret reference: '{{SECRET.openai_api_key}}' When using secret reference, the key is loaded from your organization's secrets vault at runtime. Store secrets via POST /v1/organizations/secrets. If not provided, uses Mixpeek's default API keys.
          - `max_retries` integer — Maximum retry attempts for failed HTTP requests. Uses exponential backoff with jitter. Default: 3.
          - `retry_base_delay` number — Base delay in seconds for retry backoff. Actual delay = base * 2^attempt + jitter. Default: 1.0.
          - `retry_max_delay` number — Maximum delay in seconds between retries. Default: 30.
          - `respect_retry_after` boolean — Respect Retry-After header from 429/503 responses. If False, uses exponential backoff instead. Default: True.
          - `proxies` string[], nullable — List of proxy URLs for rotation. Supports formats: 'http://host:port', 'http://user:pass@host:port', 'socks5://host:port'. Proxies rotate on errors or every N requests.
          - `rotate_proxy_on_error` boolean — Rotate to next proxy when request fails. Default: True.
          - `rotate_proxy_every_n_requests` integer — Rotate proxy every N requests (0 = disabled). Useful for avoiding IP-based rate limits. Default: 0 (disabled).
          - `captcha_service_provider` string, nullable — Captcha solving service provider: '2captcha', 'anti-captcha', 'capsolver'. If not set, captcha pages are skipped gracefully.
          - `captcha_service_api_key` string, nullable — API key for captcha solving service. Supports secret reference: '{{SECRET.captcha_api_key}}'. Required if captcha_service_provider is set.
          - `detect_captcha` boolean — Detect captcha challenges (Cloudflare, reCAPTCHA, hCaptcha). If detected and no solver configured, page is skipped. Default: True.
          - `persist_cookies` boolean — Persist cookies across requests within a crawl session. Useful for sites requiring authentication. Default: True.
          - `custom_headers` object, nullable — Custom HTTP headers to include in all requests. Example: {'Authorization': 'Bearer token', 'X-Custom': 'value'}
          - `youtube_mode` 'auto' | 'off' | 'force' — YouTube channel fast path. 'auto' (default): detect YouTube channel URLs and hand off to yt-dlp; other URLs run normal BFS. 'off': never hand off. 'force': treat every URL as a YouTube channel (useful for explicit channel buckets).
          - `youtube_max_videos` integer — Max videos to pull per channel enumeration. Default: 50.
          - `youtube_backfill_months` integer — Skip videos older than this many months. Default: 6.
          - `youtube_show_filter` string, nullable — Optional case-insensitive regex applied to video titles. Used when a channel hosts multiple shows and you only want one.
          - `youtube_download_videos` boolean — If true (default), download each video file so downstream processors can read it from the `video_path` field. If false, only metadata + captions are emitted.
          - `youtube_format_ladder` string — yt-dlp format selector. Defaults to a muxed-first ladder that avoids breaking on YouTube's DASH n-challenge when the current EJS signature solver is stale. See BUILD_LOG.md in the greenroom folder for the rationale.
          - `youtube_cookies_path` string, nullable — Optional path to a Netscape cookies file. Required for age-gated or members-only content.
          - `youtube_request_sleep` number — Seconds to sleep between yt-dlp requests. Default: 1.0.
          - `delay_between_requests` number — Delay in seconds between consecutive requests. Useful for polite crawling and avoiding rate limits. Default: 0 (no delay).
        - CustomPluginParams — Parameters for custom plugin extractors. This model accepts any extractor_type that doesn't match the builtin extractors, allowing custom plugins to define their own parameters.
          - `extractor_type` string, required — Custom plugin extractor type (plugin name)
      - `input_mappings` union — Mapping from extractor input names to source field paths. Tells the extractor which source fields to process. Single value: {'image': 'thumbnail_url'} maps one blob field to one extractor input. Array value: {'files': ['image', 'spec_pdf', 'description']} maps multiple blob fields to a single extractor input as a list — used by multi-file extractors like gemini_multifile_extractor that embed all blobs of an object into one embedding.
        - object
        - InputMapping[]
          - `input_key` string, required — Key used in the constructed inputs payload.
          - `source_type` 'payload' | 'literal' | 'vector' | 'blob' — Where the value for an input should be retrieved from.
          - `path` string, nullable — Dot-notation path when source_type is PAYLOAD or VECTOR. PAYLOAD paths resolve from the document's ROOT fields — each document/source_document dict IS the payload (e.g. path 'text' reads {'text': ...}); a {'payload': {...}} envelope is also accepted as a fallback.
          - `override` unknown
      - `field_passthrough` FieldPassthrough[] — NOT REQUIRED. List of specific fields to pass through from source to output documents. These fields are included alongside extractor-computed features (embeddings, detections, etc.). Empty list = only extractor outputs in documents (default behavior). With entries = specified fields + extractor outputs in documents. How It Works: 1. During processing, fields are extracted from source object/document 2. They appear in output documents at the root level 3. Field filtering happens automatically (only listed fields included) 4. Use target_path to rename fields for cleaner schemas Common Use Cases: - Preserve identifiers: campaign_id, product_sku, order_id - Keep metadata: category, tags, author, created_at - Enable filtering: department, status, priority, region - Maintain context: title, description, source_url Behavior: - Works with include_all_source_fields=False (default): ONLY these fields included - Works with include_all_source_fields=True: These configs used for renaming/defaults - Fields must exist in source bucket_schema or upstream collection output_schema - Missing optional fields are omitted (unless default provided) - Missing required fields cause processing errors Output Schema: output_schema = field_passthrough fields + extractor output fields Example: ['title', 'category', 'text_extractor_v1_embedding']
        - `source_path` string, required — REQUIRED. Path to the source field to copy. Simple fields: Use field name directly (e.g., 'title', 'campaign_id'). Nested fields: Use dot notation (e.g., 'metadata.author', 'config.model.version'). The field must exist in the source bucket schema or upstream collection schema. Without target_path, nested fields are flattened: 'metadata.author' becomes 'author' in output.
        - `target_path` string, nullable — OPTIONAL. Target field name in output document. If NOT PROVIDED: Uses source_path name (or last component for nested paths). - 'title' → 'title' - 'metadata.author' → 'author' If PROVIDED: Uses this exact name in output. - source_path='doc_title', target_path='title' → 'title' - source_path='metadata.author', target_path='contributor' → 'contributor' Use cases: - Rename fields for cleaner API schemas - Avoid name conflicts with extractor outputs - Standardize field names across different sources Constraints: - Must not conflict with system fields (document_id, collection_id, etc.) - Must not conflict with extractor output fields - Must be a valid field name (alphanumeric, underscores, hyphens)
        - `default` string, nullable — OPTIONAL. Default value if source field doesn't exist or is None. If NOT PROVIDED and field missing: Field is omitted from output document. If PROVIDED and field missing: Field is included with this default value. Type should match expected field type (string, int, list, dict, etc.).
        - `required` boolean — OPTIONAL. Whether this field MUST exist in source. If True and field missing: Raises validation error, processing fails. If False and field missing: Field omitted (or default used if provided). Use True for: Critical identifiers, required business fields. Use False for: Optional metadata, nice-to-have fields. Default: False (field is optional).
      - `include_all_source_fields` boolean — NOT REQUIRED. Whether to include ALL fields from source object/document in output. Default: False (only field_passthrough fields included). When False (RECOMMENDED): - Only fields listed in field_passthrough are included in output - Creates clean, predictable output schemas - Prevents data leakage of unwanted fields - Output = field_passthrough fields + extractor outputs When True (USE WITH CAUTION): - ALL source fields are included in output documents - field_passthrough still used for renaming/defaults/requirements - Can result in large documents if source has many fields - Can leak sensitive or unnecessary data - Output = all source fields + extractor outputs Use True When: - You want to preserve complete source data - Source has limited, well-defined fields - Downstream processing needs all context Use False When (MOST CASES): - You want clean, controlled output schemas - Source has many fields you don't need - You want explicit field selection - You're concerned about document size Examples: False: source={a,b,c,d} + passthrough=[a,b] → output={a,b,embedding} True: source={a,b,c,d} + passthrough=[a→x] → output={x,b,c,d,embedding}
      - `feature_extractor_id` string, required — Construct unique identifier for the feature extractor instance (name + version).
    - `output_schema` BucketSchemaOutput, required — Schema definition for bucket objects. IMPORTANT: The bucket schema defines what fields your bucket objects will have. This schema is REQUIRED if you want to: 1. Create collections that use input_mappings to process your bucket data 2. Validate object structure before ingestion 3. Enable type-safe data pipelines The schema defines the custom fields that will be used in: - Blob properties (e.g., "content", "thumbnail", "transcript") - Object metadata structure - Blob data structures Example workflow: 1. Create bucket WITH schema defining your data structure 2. Upload objects that conform to that schema 3. Create collections that map schema fields to feature extractors Without a bucket_schema, collections cannot use input_mappings.
      - `properties` object, required — REQUIRED. Map of field names to their type definitions. Each field must have a 'type' from the supported types: metadata types (string, number, integer, float, boolean, object, array, date, datetime) or file/blob types (text, image, audio, video, pdf, excel). NOTE: Use Mixpeek types, NOT JSON Schema types — e.g. use 'string' not 'keyword', 'text' for text blobs, 'image' for image blobs. Example: {"title": {"type": "string"}, "photo": {"type": "image"}}
  - `vector_indexes` unknown[] — NOT REQUIRED (auto-computed from extractor). Vector indexes for semantic search. Populated from feature_extractor.required_vector_indexes. Defines: Which embeddings are indexed, dimensions, distance metrics. Use for: Understanding search capabilities, debugging vector queries.
    - unknown
  - `payload_indexes` unknown[] — NOT REQUIRED (auto-computed from extractor + namespace). Payload indexes for filtering. Enables efficient filtering on metadata fields, timestamps, IDs. Populated from: extractor requirements + namespace defaults. Use for: Understanding which fields support fast filtering.
    - unknown
  - `embedding_task` string, nullable — Override the embedding task hint for instruction-aware models (E5, Gemini). Defaults to 'retrieval_document' for indexing pipelines. Values: retrieval_document, retrieval_query, semantic_similarity, classification, clustering. Applied to all task-aware embedding models in this collection's extractor pipeline.
  - `enabled` boolean — NOT REQUIRED (defaults to True). Whether the collection accepts new documents. False: Collection exists but won't process new objects. True: Active and processing. Use for: Temporarily disabling collections without deletion.
  - `metadata` object, nullable — NOT REQUIRED. Additional user-defined metadata for the collection. Arbitrary key-value pairs for custom organization, tracking, configuration. Not used by the platform - purely for user purposes. Common uses: team ownership, project tags, deployment environment.
  - `schedule` CollectionScheduleConfig — Schedule configuration for automatic collection re-processing. Attaches a cron or interval schedule to a collection, which creates a COLLECTION_TRIGGER trigger behind the scenes. This is a DX convenience so users don't need to create triggers manually. Examples: Daily re-crawl at 2am UTC: {"trigger_type": "cron", "schedule_config": {"cron_expression": "0 2 * * *"}} Every 6 hours: {"trigger_type": "interval", "schedule_config": {"interval_seconds": 21600}}
    - `trigger_type` string, required — Schedule type: 'cron' or 'interval'
    - `schedule_config` object, required — Schedule configuration. For cron: {cron_expression, timezone}. For interval: {interval_seconds, start_immediately}.
    - `description` string, nullable — Human-readable description of the schedule.
  - `trigger_id` string, nullable — NOT REQUIRED. ID of the linked trigger for scheduled re-processing. Automatically set when a schedule is configured.
  - `created_at` string, date-time, nullable — Timestamp when the collection was created. Automatically set by the system when the collection is first saved to the database.
  - `updated_at` string, date-time, nullable — Timestamp when the collection was last updated. Automatically updated by the system whenever the collection is modified.
  - `lifecycle_state` string — Storage lifecycle state: 'active' (Qdrant + S3), 'cold' (S3 only), 'archived' (metadata only). Managed via lifecycle API.
  - `s3_vector_index` string, nullable — S3 Vectors index name for this collection (e.g. 'col_{collection_id}').
  - `last_lifecycle_transition` string, date-time, nullable — Timestamp of the most recent lifecycle state change.
  - `tiering_rules` TieringRule[], nullable — Automatic storage tiering rules (not enforced in V1).
    - `rule_type` 'auto_evict' | 'auto_archive' | 'auto_rehydrate', required
    - `enabled` boolean
    - `threshold_days` integer, nullable
  - `document_count` integer, nullable — Number of documents in the collection
  - `schema_version` integer — Version number for the output_schema. Increments automatically when schema is updated via document sampling. Used to track schema evolution and trigger downstream collection schema updates.
  - `last_schema_sync` string, date-time, nullable — Timestamp of last automatic schema sync from document sampling. Used to debounce schema updates (prevents thrashing).
  - `schema_sync_enabled` boolean — Whether automatic schema discovery and sync is enabled for this collection. When True, schema is periodically updated by sampling documents. When False, schema remains fixed at creation time.
  - `document_schema` object, nullable — NOT REQUIRED. JSON Schema for validating documents on create/update. When set with schema_validation='strict', non-conforming documents are rejected (422). When set with schema_validation='warn', violations are recorded but document is accepted. Schema follows JSON Schema draft-07 format. Only validates user-defined fields (system fields like _internal, collection_id, document_id are excluded from validation).
  - `schema_validation` 'strict' | 'warn' | 'off' — Document schema validation mode. 'strict': reject non-conforming documents with 422. 'warn': accept but attach _schema_violations field to the document. 'off': no validation (default, preserves current behavior).
  - `taxonomy_applications` TaxonomyApplicationConfigOutput[], nullable — NOT REQUIRED. List of taxonomies to apply to documents in this collection. Each entry specifies: taxonomy_id, optional target_collection_id, optional filters. Enrichments are materialized (persisted to documents) during ingestion. Empty/null if no taxonomies attached. Use for: Categorization, hierarchical classification.
    - `taxonomy_id` string, required — ID of the `TaxonomyModel` to execute.
    - `execution_mode` 'materialize' — How a taxonomy should be executed when attached to a collection.
    - `target_collection_id` string, nullable — Optional collection to persist results into when `execution_mode` is 'materialize'. If omitted, the source collection is updated in-place.
    - `scroll_filters` LogicalOperatorOutput — Represents a logical operation (AND, OR, NOT) on filter conditions. Allows nesting with a defined depth limit. Also supports shorthand syntax where field names can be passed directly as key-value pairs for equality filtering (e.g., {"metadata.title": "value"}).
      - `AND` union[], nullable — Logical AND operation - all conditions must be true
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `OR` union[], nullable — Logical OR operation - at least one condition must be true
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `NOT` union[], nullable — Logical NOT operation - all conditions must be false
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `case_sensitive` boolean, nullable — Whether to perform case-sensitive matching
    - `execution_phase` integer — Which phase this taxonomy runs in. Default: 1 (TAXONOMY phase, runs first). Valid values: 1=TAXONOMY, 2=CLUSTER, 3=ALERT. Lower phases run earlier.
    - `priority` integer — Priority within the execution phase (higher = runs first)
    - `hierarchical_enrichment_style` 'full_chain' | 'best_match' | 'combined' — How hierarchical taxonomy results should be structured in enriched documents. Controls the field naming pattern for multi-tier taxonomy enrichment.
  - `cluster_applications` ClusterApplicationConfig[], nullable — NOT REQUIRED. List of clusters to automatically execute when batch processing completes. Each entry specifies: cluster_id, auto_execute_on_batch, min_document_threshold, cooldown_seconds. Clusters enrich source documents with cluster assignments (cluster_id, cluster_label, etc.). Empty/null if no clusters attached. Use for: Segmentation, grouping, pattern discovery.
    - `cluster_id` string, required — ID of the cluster to execute (must exist and use this collection as input)
    - `auto_execute_on_batch` boolean — Automatically execute cluster when batch processing completes for this collection. If False, cluster must be executed manually via API.
    - `min_document_threshold` integer, nullable — Minimum number of documents required before executing cluster. If document_count < threshold, clustering is skipped. Useful to avoid clustering on small datasets.
    - `cooldown_seconds` integer — Minimum time (in seconds) between automatic cluster executions. Prevents excessive re-clustering on frequent batch completions. Default: 3600 seconds (1 hour).
    - `execution_phase` integer — Which phase this cluster runs in. Default: 2 (CLUSTER phase, after taxonomies). Valid values: 1=TAXONOMY, 2=CLUSTER, 3=ALERT. Lower phases run earlier.
    - `priority` integer — Priority within the execution phase (higher = runs first)
  - `alert_applications` AlertApplicationConfigOutput[], nullable — NOT REQUIRED. List of alerts to automatically execute when documents are ingested. Each entry specifies: alert_id, execution_mode, input_mappings, execution_phase, priority. Alerts run retrievers on ingested documents and send notifications when matches are found. Empty/null if no alerts attached. Use for: Content monitoring, safety detection, compliance alerts.
    - `alert_id` string, required — ID of the alert to execute
    - `execution_mode` 'on_ingest' | 'scheduled' | 'on_demand' — When the alert should execute.
    - `input_mappings` AlertInputMapping[], required — Map document fields or constants to retriever input parameters
      - `input_key` string, required — The retriever input parameter name
      - `source` InputMappingSource, required — Defines how to get a value for a retriever input. Can be a document field reference, a constant value, or a source blob URL. Attributes: source_type: Where the value comes from path: JSONPath to document field (when source_type='document_field'), or blob property name (when source_type='source_blob', e.g. 'image') value: Constant value to use (when source_type='constant')
        - `source_type` 'document_field' | 'constant' | 'source_blob', required — Where the value comes from
        - `path` string, nullable — JSONPath to document field (when source_type='document_field'), or blob property name (when source_type='source_blob', e.g. 'image')
        - `value` string, nullable — Constant value to use (when source_type='constant')
    - `execution_phase` 1 | 2 | 3 | 4 — Execution phases for post-processing applications. Applications execute in phase order (lower = earlier). Within a phase, applications execute by priority (higher = earlier). Phases: TAXONOMY (1): Classification and labeling operations CLUSTER (2): Grouping and clustering operations ALERT (3): Notifications and alerts (default for alerts) RETRIEVER_ENRICHMENT (4): Retriever-based enrichment operations The default phase for each application type: - TaxonomyApplicationConfig: TAXONOMY - ClusterApplicationConfig: CLUSTER - AlertApplicationConfig: ALERT - RetrieverEnrichmentConfig: RETRIEVER_ENRICHMENT Users can override the phase via the `execution_phase` field to run applications in non-default order. For example, an alert can be configured to run in Phase 1 alongside taxonomies if early notification is needed. Example: # Default: Alert runs after taxonomies and clusters AlertApplicationConfig(alert_id="alt_123", execution_phase=PostProcessingPhase.ALERT) # Override: Run alert early, in taxonomy phase AlertApplicationConfig(alert_id="alt_urgent", execution_phase=PostProcessingPhase.TAXONOMY)
    - `priority` integer — Priority within the execution phase (higher = runs first)
  - `retriever_enrichments` RetrieverEnrichmentConfigOutput[], nullable — NOT REQUIRED. List of retriever enrichments to run on documents during post-processing. Each entry specifies: retriever_id, input_mappings, write_back_fields, execution_phase, priority. Retriever enrichments execute a retriever pipeline and write selected result fields back to documents. Empty/null if no enrichments attached. Use for: LLM classification, cross-collection joins, enrichment.
    - `retriever_id` string, required — ID of the retriever to execute
    - `input_mappings` EnrichmentInputMapping[], required — Map document fields or constants to retriever input parameters
      - `input_key` string, required — The retriever input parameter name
      - `source` InputMappingSource, required — Defines how to get a value for a retriever input. Can be a document field reference, a constant value, or a source blob URL. Attributes: source_type: Where the value comes from path: JSONPath to document field (when source_type='document_field'), or blob property name (when source_type='source_blob', e.g. 'image') value: Constant value to use (when source_type='constant')
        - `source_type` 'document_field' | 'constant' | 'source_blob', required — Where the value comes from
        - `path` string, nullable — JSONPath to document field (when source_type='document_field'), or blob property name (when source_type='source_blob', e.g. 'image')
        - `value` string, nullable — Constant value to use (when source_type='constant')
    - `write_back_fields` WriteBackFieldMapping[], required — Which retriever result fields to write back to the document
      - `source_field` string, required — Field path in retriever result (dot notation supported)
      - `target_field` string, required — Field name to write on the document
      - `mode` 'first' | 'all_as_array' | 'concat' — How to aggregate values from multiple results. 'first': top result only. 'all_as_array': collect into list. 'concat': join strings with ', '.
    - `execution_phase` 1 | 2 | 3 | 4 — Execution phases for post-processing applications. Applications execute in phase order (lower = earlier). Within a phase, applications execute by priority (higher = earlier). Phases: TAXONOMY (1): Classification and labeling operations CLUSTER (2): Grouping and clustering operations ALERT (3): Notifications and alerts (default for alerts) RETRIEVER_ENRICHMENT (4): Retriever-based enrichment operations The default phase for each application type: - TaxonomyApplicationConfig: TAXONOMY - ClusterApplicationConfig: CLUSTER - AlertApplicationConfig: ALERT - RetrieverEnrichmentConfig: RETRIEVER_ENRICHMENT Users can override the phase via the `execution_phase` field to run applications in non-default order. For example, an alert can be configured to run in Phase 1 alongside taxonomies if early notification is needed. Example: # Default: Alert runs after taxonomies and clusters AlertApplicationConfig(alert_id="alt_123", execution_phase=PostProcessingPhase.ALERT) # Override: Run alert early, in taxonomy phase AlertApplicationConfig(alert_id="alt_urgent", execution_phase=PostProcessingPhase.TAXONOMY)
    - `priority` integer — Priority within the execution phase (higher = runs first)
    - `scroll_filters` LogicalOperatorOutput — Represents a logical operation (AND, OR, NOT) on filter conditions. Allows nesting with a defined depth limit. Also supports shorthand syntax where field names can be passed directly as key-value pairs for equality filtering (e.g., {"metadata.title": "value"}).
      - `AND` union[], nullable — Logical AND operation - all conditions must be true
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `OR` union[], nullable — Logical OR operation - at least one condition must be true
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `NOT` union[], nullable — Logical NOT operation - all conditions must be false
        - union
          - LogicalOperatorOutput — recursive
          - FilterCondition — Represents a single filter condition. Attributes: field: The field to filter on operator: The comparison operator value: The value to compare against
            - `field` string, required — Field name to filter on
            - `operator` 'eq' | 'ne' | 'gt' | 'lt' | 'gte' | 'lte' | 'in' | 'nin' | 'contains' | 'starts_with' | 'ends_with' | 'regex' | 'exists' | 'is_null' | 'text' | 'phrase' | 'geo_radius' | 'geo_bounding_box' | 'geo_polygon' — Supported filter operators across database implementations.
            - `value` union, required — Value to compare against
              - …
      - `case_sensitive` boolean, nullable — Whether to perform case-sensitive matching
    - `enabled` boolean — Whether this enrichment is active
  - `vector_count` integer, nullable — Total number of vector entries across all documents in this collection. Computed as document_count * number_of_vector_indexes. Each document stores one vector per configured vector index (e.g. a collection with 1 embedding produces 1 vector per document).
  - `taxonomy_count` integer, nullable — Number of taxonomies connected to this collection
  - `retriever_count` integer, nullable — Number of retrievers connected to this collection

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `403` — Forbidden
- `404` — Not Found
- `422` — Validation Error
- `500` — Internal Server Error

---

[API](https://skmtc.net/mixpeek/apis/mixpeek-api.md) · [All operations](https://skmtc.net/mixpeek/apis/mixpeek-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/mixpeek/mixpeek-api/versions/220a3b263fda/schema)