--- title: "Batch Create Datasets" method: POST path: "/api/dataset/batch_create_datasets" tags: ["Dataset"] --- # Batch Create Datasets `POST /api/dataset/batch_create_datasets` Datasets will be created in the org specified via the TR-Organization header. Auth'ed user must be an owner of the organization to create datasets. If a tracking_id is ignored due to it already existing on the org, the response will not contain a dataset with that tracking_id and it can be assumed that a dataset with the missing tracking_id already exists. ## Headers - `TR-Organization` string, uuid, required ## Request body - CreateDatasetBatchReqPayload - `datasets` CreateBatchDataset[], required — List of datasets to create - `dataset_name` string, required — Name of the dataset. - `server_configuration` DatasetConfigurationDTO — Lets you specify the configuration for a dataset - `AIMON_RERANKER_TASK_DEFINITION` string, nullable - `BM25_AVG_LEN` number, float, nullable — The average length of the chunks in the index for BM25 - `BM25_B` number, float, nullable — The BM25 B parameter - `BM25_ENABLED` boolean, nullable — Whether to use BM25 - `BM25_K` number, float, nullable — The BM25 K parameter - `DISABLE_ANALYTICS` boolean, nullable — Whether to disable analytics - `DISTANCE_METRIC` 'euclidean' | 'cosine' | 'manhattan' | 'dot' - `EMBEDDING_BASE_URL` string, nullable — The base URL for the embedding API - `EMBEDDING_MODEL_NAME` string, nullable — The name of the embedding model to use - `EMBEDDING_QUERY_PREFIX` string, nullable — The prefix to use for the embedding query - `EMBEDDING_SIZE` integer, nullable — The size of the embeddings - `FREQUENCY_PENALTY` number, double, nullable — The frequency penalty to use - `FULLTEXT_ENABLED` boolean, nullable — Whether to use fulltext search - `INDEXED_ONLY` boolean, nullable — Whether to only use indexed chunks - `LLM_API_VERSION` string, nullable — The API version for the LLM API - `LLM_BASE_URL` string, nullable — The base URL for the LLM API - `LLM_DEFAULT_MODEL` string, nullable — The default model to use for the LLM - `LOCKED` boolean, nullable — Whether the dataset is locked to prevent changes or deletion - `MAX_LIMIT` integer, nullable — The maximum limit for the number of chunks for counting - `MAX_TOKENS` integer, nullable — The maximum number of tokens to use in LLM Response - `MESSAGE_TO_QUERY_PROMPT` string, nullable — The prompt to use for converting a message to a query - `N_RETRIEVALS_TO_INCLUDE` integer, nullable — The number of retrievals to include with the RAG model - `PAGEFIND_ENABLED` boolean, nullable — Whether to enable pagefind indexing - `PRESENCE_PENALTY` number, double, nullable — The presence penalty to use - `PUBLIC_DATASET` PublicDatasetOptions - `enabled` boolean, required - `extra_params` PublicPageParameters - `allowSwitchingModes` boolean, nullable - `analytics` boolean, nullable - `apiKey` string, nullable - `baseUrl` string, nullable - `brandColor` string, nullable - `brandFontFamily` string, nullable - `brandLogoImgSrcUrl` string, nullable - `brandName` string, nullable - `buttonTriggers` ButtonTrigger[], nullable - `mode` string, required - `removeTriggers` boolean, nullable - `selector` string, required - `chat` boolean, nullable - `creatorLinkedInUrl` string, nullable - `creatorName` string, nullable - `currencyPosition` string, nullable - `datasetId` string, uuid, nullable - `debounceMs` integer, nullable - `defaultAiQuestions` DefaultSearchQueryType[], nullable - union - … - `defaultCurrency` string, nullable - `defaultImageQuestion` string, nullable - `defaultSearchMode` string, nullable - `defaultSearchQueries` DefaultSearchQueryType[], nullable - union - … - `defaultSearchQuery` string, nullable - `floatingButtonPosition` string, nullable - `floatingButtonVersion` string, nullable - `floatingSearchIconPosition` string, nullable - `followupQuestions` boolean, nullable - `forBrandName` string, nullable - `headingPrefix` string, nullable - `heroPattern` HeroPattern - `backgroundColor` string, nullable - `foregroundColor` string, nullable - `foregroundOpacity` number, float, nullable - `heroPatternName` string, nullable - `heroPatternSvg` string, nullable - `hideDrawnText` boolean, nullable - `imageStarterText` string, nullable - `inline` boolean, nullable - `inlineHeader` string, nullable - `isTestMode` boolean, nullable - `navLogoImgSrcUrl` string, nullable - `notFilterToolCallOptions` NotFilterToolCallOptions - `toolDescription` string, nullable - `userMessageTextPrefix` string, nullable - `numberOfSuggestions` integer, nullable - `openGraphMetadata` OpenGraphMetadata - `description` string, nullable - `image` string, nullable - `title` string, nullable - `openLinksInNewTab` boolean, nullable - `placeholder` string, nullable - `priceToolCallOptions` PriceToolCallOptions - `maxPriceDescription` string, nullable - `minPriceDescription` string, nullable - `toolDescription` string, required - `problemLink` string, nullable - `relevanceToolCallOptions` RelevanceToolCallOptions - `highDescription` string, nullable - `includeImages` boolean, nullable - `lowDescription` string, nullable - `mediumDescription` string, nullable - `toolDescription` string, required - `userMessageTextPrefix` string, nullable - `responsive` boolean, nullable - `searchBar` boolean, nullable - `searchOptions` PublicPageSearchOptions - `content_only` boolean, nullable — Set content_only to true to only returning the chunk_html of the chunks. This is useful for when you want to reduce amount of data over the wire for latency improvement (typically 10-50ms). Default is false. - `filters` ChunkFilter — ChunkFilter is a JSON object which can be used to filter chunks. This is useful for when you want to filter chunks by arbitrary metadata. Unlike with tag filtering, there is a performance hit for filtering on metadata. - … - `get_total_pages` boolean, nullable — Get total page count for the query accounting for the applied filters. Defaults to false, but can be set to true when the latency penalty is acceptable (typically 50-200ms). - `page` integer, nullable — Page of chunks to fetch. Page is 1-indexed. - `page_size` integer, nullable — Page size is the number of chunks to fetch. This can be used to fetch more than 10 chunks at a time. - `remove_stop_words` boolean, nullable — If true, stop words (specified in server/src/stop-words.txt in the git repo) will be removed. Queries that are entirely stop words will be preserved. - `score_threshold` number, float, nullable — Set score_threshold to a float to filter out chunks with a score below the threshold for cosine distance metric. For Manhattan Distance, Euclidean Distance, and Dot Product, it will filter out scores above the threshold distance. This threshold applies before weight and bias modifications. If not specified, this defaults to no threshold. A threshold of 0 will default to no threshold. - `scoring_options` ScoringOptions — Scoring options provides ways to modify the sparse or dense vector created for the query in order to change how potential matches are scored. If not specified, this defaults to no modifications. - … - `search_type` 'fulltext' | 'semantic' | 'hybrid' | 'bm25' - `slim_chunks` boolean, nullable — Set slim_chunks to true to avoid returning the content and chunk_html of the chunks. This is useful for when you want to reduce amount of data over the wire for latency improvement (typically 10-50ms). Default is false. - `sort_options` SortOptions — Sort Options lets you specify different methods to rerank the chunks in the result set. If not specified, this defaults to the score of the chunks. - … - `typo_options` TypoOptions — Typo Options lets you specify different methods to correct typos in the query. If not specified, typos will not be corrected. - … - `use_autocomplete` boolean, nullable — Enables autocomplete on the search modal. - `use_quote_negated_terms` boolean, nullable — If true, quoted and - prefixed words will be parsed from the queries and used as required and negated words respectively. Default is false. - `user_id` string, nullable — User ID is the id of the user who is making the request. This is used to track user interactions with the search results. - `searchPageProps` SearchPageProps - `display` boolean, nullable - `filterSidebarProps` SidebarFilters - … - `searchToolCallOptions` SearchToolCallOptions - `noSearchRagContext` string, nullable - `toolDescription` string, nullable - `userMessageTextPrefix` string, nullable - `showFloatingButton` boolean, nullable - `showFloatingInput` boolean, nullable - `showFloatingSearchIcon` boolean, nullable - `showResultHighlights` boolean, nullable - `singleProductOptions` SingleProductOptions - `enabled` boolean, nullable - `groupTrackingId` string, nullable - `pdpPrompt` string, nullable - `productDescriptionHtml` string, nullable - `productName` string, nullable - `productPrimaryImageUrl` string, nullable - `productQuestions` DefaultSearchQuery[], nullable - … - `productTrackingId` string, nullable - `recSearchQuery` string, nullable - `suggestedQueries` boolean, nullable - `tabMessages` PublicPageTabMessage[], nullable - `showComponentCode` boolean, required - `tabInnerHtml` string, required - `title` string, required - `tags` PublicPageTag[], nullable - `description` string, nullable - `iconClassName` string, nullable - `label` string, nullable - `selected` boolean, nullable - `tag` string, required - `theme` 'light' | 'dark' - `type` string, nullable - `useGroupSearch` boolean, nullable - `useLocal` boolean, nullable - `usePagefind` boolean, nullable - `videoLink` string, nullable - `videoPosition` string, nullable - `zIndex` integer, nullable - `QDRANT_ONLY` boolean, nullable — Whether or not to insert chunks into Postgres - `RAG_PROMPT` string, nullable — The prompt to use for the RAG model - `RERANKER_BASE_URL` string, nullable — The base URL for the reranker API - `RERANKER_MODEL_NAME` string, nullable — The model name for the Reranker API - `SEMANTIC_ENABLED` boolean, nullable — Whether to use semantic search - `STOP_TOKENS` string[], nullable — The stop tokens to use - `SYSTEM_PROMPT` string, nullable — The system prompt to use for the LLM - `TEMPERATURE` number, double, nullable — The temperature to use - `TOOL_CONFIGURATION` ToolConfiguration - `query_tool_options` QueryToolOptions - `max_price_option_description` string, nullable - `min_price_option_description` string, nullable - `price_filter_description` string, nullable - `query_parameter_description` string, nullable - `tool_description` string, nullable - `USE_MESSAGE_TO_QUERY_PROMPT` boolean, nullable — Whether to use the message to query prompt - `tracking_id` string, nullable — Optional tracking ID for the dataset. Can be used to track the dataset in external systems. Must be unique within the organization. Strongly recommended to not use a valid uuid value as that will not work with the TR-Dataset header. - `upsert` boolean, nullable — Upsert when a dataset with one of the specified tracking_ids already exists. By default this is false and specified datasets with a tracking_id that already exists in the org will not be ignored. If true, the existing dataset will be updated with the new dataset's details. ## Response `200` Page of tags requested with all tags and the number of chunks in the dataset with that tag plus the total number of unique tags for the whole datset - Dataset[] — Datasets - `created_at` string, date-time, required — Timestamp of the creation of the dataset - `deleted` integer, required — Flag to indicate if the dataset has been deleted. Deletes are handled async after the flag is set so as to avoid expensive search index compaction. - `id` string, uuid, required — Unique identifier of the dataset, auto-generated uuid created by Trieve - `name` string, required — Name of the dataset - `organization_id` string, uuid, required — Unique identifier of the organization that owns the dataset - `server_configuration` unknown, required - `tracking_id` string, nullable — Tracking ID of the dataset, can be any string, determined by the user. Tracking ID's are unique identifiers for datasets within an organization. They are designed to match the unique identifier of the dataset in the user's system. - `updated_at` string, date-time, required — Timestamp of the last update of the dataset ## Other responses - `400` — Service error relating to finding items by tag --- [API](https://skmtc.net/devflowinc/apis/trieve-api.md) · [All operations](https://skmtc.net/devflowinc/apis/trieve-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/devflowinc/trieve-api/revisions/84583e7c9fc1/schema)