Sign in

v5

latestOpenAPI 3.1.02026-08-025631,1012.8 MB
Collections

Trigger Collection Processing

Process data through a collection - works for both bucket-sourced and collection-sourced collections.

**For bucket-sourced collections:**
Discovers objects from source bucket(s), creates a batch, and submits for processing.
Use `include_buckets` to limit which source buckets to process from.

**For collection-sourced collections:**
Processes existing documents from upstream collection(s).
Use `include_collections` to limit which source collections to process from.

**Filtering:**
- `source_filters`: Field-level filters using LogicalOperator format
- Example: `{"AND": [{"field": "status", "operator": "eq", "value": "pending"}]}`
- For specific objects: `{"AND": [{"field": "object_id", "operator": "in", "value": ["obj_1", "obj_2"]}]}`

**Returns:**
- batch_id: Track progress via GET /batches/{batch_id}
- task_id: Monitor via GET /tasks/{task_id}
post/v1/collections/{collection_identifier}/trigger

Path parameters

collection_identifierstring required

The ID or name of the collection to trigger

The ID or name of the collection to trigger

Request body

include_bucketsstring[] nullable

Limit processing to objects from these specific buckets (IDs or names). Only applies to bucket-sourced collections. If not provided, all configured source buckets are used.

include_collectionsstring[] nullable

Limit processing to documents from these specific collections (IDs or names). Only applies to collection-sourced collections. If not provided, all configured source collections are used.

object_idsstring[] nullable

Limit processing to these specific object IDs. Only applies to bucket-sourced collections. This is a convenience shorthand — equivalent to using source_filters with {"AND": [{"field": "object_id", "operator": "in", "value": [...]}]}.

dedup_strategy'skip' | 'replace' | 'force'

Controls how duplicate objects are handled during batch processing.

Dedup is scoped to (bucket_id, collection_id): an object is considered a duplicate if the target collection already has documents produced from the same source object in any prior batch.

Example request

{
  "source_filters": {
    "AND": [
      {
        "field": "name",
        "operator": "eq",
        "value": "John"
      },
      {
        "field": "age",
        "operator": "gte",
        "value": 30
      }
    ],
    "OR": [
      {
        "field": "status",
        "operator": "eq",
        "value": "active"
      },
      {
        "field": "role",
        "operator": "eq",
        "value": "admin"
      }
    ],
    "NOT": [
      {
        "field": "department",
        "operator": "eq",
        "value": "HR"
      },
      {
        "field": "location",
        "operator": "eq",
        "value": "remote"
      }
    ],
    "case_sensitive": true
  }
}

Response

Successful Response

batch_idstring required

ID of the created batch for tracking progress.

task_idstring required

Task ID for monitoring via GET /v1/tasks/{task_id}.

collection_idstring required

ID of the collection being processed.

source_bucket_idsstring[] nullable

Bucket IDs that objects were discovered from (bucket-sourced collections).

source_collection_idsstring[] nullable

Collection IDs that documents were read from (collection-sourced collections).

object_countinteger nullable

Total number of objects included in the batch (bucket-sourced collections).

document_countinteger nullable

Total number of documents to process (collection-sourced collections).

total_tiersinteger required

Number of processing tiers in the DAG.

messagestring required

Human-readable status message.