--- title: "Create Dataset" method: POST path: "/api/dataset" tags: ["Dataset"] --- # Create Dataset `POST /api/dataset` Dataset will be created in the org specified via the TR-Organization header. Auth'ed user must be an owner of the organization to create a dataset. ## Headers - `TR-Organization` string, uuid, required ## Request body - CreateDatasetReqPayload - `dataset_name` string, required — Name of the dataset. - `server_configuration` DatasetConfigurationDTO — Lets you specify the configuration for a dataset - `AIMON_RERANKER_TASK_DEFINITION` string, nullable - `BM25_AVG_LEN` number, float, nullable — The average length of the chunks in the index for BM25 - `BM25_B` number, float, nullable — The BM25 B parameter - `BM25_ENABLED` boolean, nullable — Whether to use BM25 - `BM25_K` number, float, nullable — The BM25 K parameter - `DISABLE_ANALYTICS` boolean, nullable — Whether to disable analytics - `DISTANCE_METRIC` 'euclidean' | 'cosine' | 'manhattan' | 'dot' - `EMBEDDING_BASE_URL` string, nullable — The base URL for the embedding API - `EMBEDDING_MODEL_NAME` string, nullable — The name of the embedding model to use - `EMBEDDING_QUERY_PREFIX` string, nullable — The prefix to use for the embedding query - `EMBEDDING_SIZE` integer, nullable — The size of the embeddings - `FREQUENCY_PENALTY` number, double, nullable — The frequency penalty to use - `FULLTEXT_ENABLED` boolean, nullable — Whether to use fulltext search - `INDEXED_ONLY` boolean, nullable — Whether to only use indexed chunks - `LLM_API_VERSION` string, nullable — The API version for the LLM API - `LLM_BASE_URL` string, nullable — The base URL for the LLM API - `LLM_DEFAULT_MODEL` string, nullable — The default model to use for the LLM - `LOCKED` boolean, nullable — Whether the dataset is locked to prevent changes or deletion - `MAX_LIMIT` integer, nullable — The maximum limit for the number of chunks for counting - `MAX_TOKENS` integer, nullable — The maximum number of tokens to use in LLM Response - `MESSAGE_TO_QUERY_PROMPT` string, nullable — The prompt to use for converting a message to a query - `N_RETRIEVALS_TO_INCLUDE` integer, nullable — The number of retrievals to include with the RAG model - `PAGEFIND_ENABLED` boolean, nullable — Whether to enable pagefind indexing - `PRESENCE_PENALTY` number, double, nullable — The presence penalty to use - `PUBLIC_DATASET` PublicDatasetOptions - `enabled` boolean, required - `extra_params` PublicPageParameters - `allowSwitchingModes` boolean, nullable - `analytics` boolean, nullable - `apiKey` string, nullable - `baseUrl` string, nullable - `brandColor` string, nullable - `brandFontFamily` string, nullable - `brandLogoImgSrcUrl` string, nullable - `brandName` string, nullable - `buttonTriggers` ButtonTrigger[], nullable - `mode` string, required - `removeTriggers` boolean, nullable - `selector` string, required - `chat` boolean, nullable - `creatorLinkedInUrl` string, nullable - `creatorName` string, nullable - `currencyPosition` string, nullable - `datasetId` string, uuid, nullable - `debounceMs` integer, nullable - `defaultAiQuestions` DefaultSearchQueryType[], nullable - union - string - DefaultSearchQuery - … - `defaultCurrency` string, nullable - `defaultImageQuestion` string, nullable - `defaultSearchMode` string, nullable - `defaultSearchQueries` DefaultSearchQueryType[], nullable - union - string - DefaultSearchQuery - … - `defaultSearchQuery` string, nullable - `floatingButtonPosition` string, nullable - `floatingButtonVersion` string, nullable - `floatingSearchIconPosition` string, nullable - `followupQuestions` boolean, nullable - `forBrandName` string, nullable - `headingPrefix` string, nullable - `heroPattern` HeroPattern - `backgroundColor` string, nullable - `foregroundColor` string, nullable - `foregroundOpacity` number, float, nullable - `heroPatternName` string, nullable - `heroPatternSvg` string, nullable - `hideDrawnText` boolean, nullable - `imageStarterText` string, nullable - `inline` boolean, nullable - `inlineHeader` string, nullable - `isTestMode` boolean, nullable - `navLogoImgSrcUrl` string, nullable - `notFilterToolCallOptions` NotFilterToolCallOptions - `toolDescription` string, nullable - `userMessageTextPrefix` string, nullable - `numberOfSuggestions` integer, nullable - `openGraphMetadata` OpenGraphMetadata - `description` string, nullable - `image` string, nullable - `title` string, nullable - `openLinksInNewTab` boolean, nullable - `placeholder` string, nullable - `priceToolCallOptions` PriceToolCallOptions - `maxPriceDescription` string, nullable - `minPriceDescription` string, nullable - `toolDescription` string, required - `problemLink` string, nullable - `relevanceToolCallOptions` RelevanceToolCallOptions - `highDescription` string, nullable - `includeImages` boolean, nullable - `lowDescription` string, nullable - `mediumDescription` string, nullable - `toolDescription` string, required - `userMessageTextPrefix` string, nullable - `responsive` boolean, nullable - `searchBar` boolean, nullable - `searchOptions` PublicPageSearchOptions - `content_only` boolean, nullable — Set content_only to true to only returning the chunk_html of the chunks. This is useful for when you want to reduce amount of data over the wire for latency improvement (typically 10-50ms). Default is false. - `filters` ChunkFilter — ChunkFilter is a JSON object which can be used to filter chunks. This is useful for when you want to filter chunks by arbitrary metadata. Unlike with tag filtering, there is a performance hit for filtering on metadata. - `must` ConditionType[], nullable — All of these field conditions have to match for the chunk to be included in the result set. - … - `must_not` ConditionType[], nullable — None of these field conditions can match for the chunk to be included in the result set. - … - `should` ConditionType[], nullable — Only one of these field conditions has to match for the chunk to be included in the result set. - … - `get_total_pages` boolean, nullable — Get total page count for the query accounting for the applied filters. Defaults to false, but can be set to true when the latency penalty is acceptable (typically 50-200ms). - `page` integer, nullable — Page of chunks to fetch. Page is 1-indexed. - `page_size` integer, nullable — Page size is the number of chunks to fetch. This can be used to fetch more than 10 chunks at a time. - `remove_stop_words` boolean, nullable — If true, stop words (specified in server/src/stop-words.txt in the git repo) will be removed. Queries that are entirely stop words will be preserved. - `score_threshold` number, float, nullable — Set score_threshold to a float to filter out chunks with a score below the threshold for cosine distance metric. For Manhattan Distance, Euclidean Distance, and Dot Product, it will filter out scores above the threshold distance. This threshold applies before weight and bias modifications. If not specified, this defaults to no threshold. A threshold of 0 will default to no threshold. - `scoring_options` ScoringOptions — Scoring options provides ways to modify the sparse or dense vector created for the query in order to change how potential matches are scored. If not specified, this defaults to no modifications. - `fulltext_boost` FullTextBoost — Boost the presence of certain tokens for fulltext (SPLADE) and keyword (BM25) search. I.e. boosting title phrases to priortize title matches or making sure that the listing for AirBNB itself ranks higher than companies who make software for AirBNB hosts by boosting the in-document-frequency of the AirBNB token (AKA word) for its official listing. Conceptually it multiples the in-document-importance second value in the tuples of the SPLADE or BM25 sparse vector of the chunk_html innerText for all tokens present in the boost phrase by the boost factor like so: (token, in-document-importance) -> (token, in-document-importance*boost_factor). - … - `semantic_boost` SemanticBoost — Semantic boosting moves the dense vector of the chunk in the direction of the distance phrase for semantic search. I.e. you can force a cluster by moving every chunk for a PDF closer to its title or push a chunk with a chunk_html of "iphone" 25% closer to the term "flagship" by using the distance phrase "flagship" and a distance factor of 0.25. Conceptually it's drawing a line (euclidean/L2 distance) between the vector for the innerText of the chunk_html and distance_phrase then moving the vector of the chunk_html distance_factor*L2Distance closer to or away from the distance_phrase point along the line between the two points. - … - `search_type` 'fulltext' | 'semantic' | 'hybrid' | 'bm25' - `slim_chunks` boolean, nullable — Set slim_chunks to true to avoid returning the content and chunk_html of the chunks. This is useful for when you want to reduce amount of data over the wire for latency improvement (typically 10-50ms). Default is false. - `sort_options` SortOptions — Sort Options lets you specify different methods to rerank the chunks in the result set. If not specified, this defaults to the score of the chunks. - `location_bias` GeoInfoWithBias — Location bias lets you rank your results by distance from a location. If not specified, this has no effect. Bias allows you to determine how much of an effect the location of chunks will have on the search results. If not specified, this defaults to 0.0. We recommend setting this to 1.0 for a gentle reranking of the results, >3.0 for a strong reranking of the results. - … - `mmr` MmrOptions — MMR Options lets you specify different methods to rerank the chunks in the result set using Maximal Marginal Relevance. If not specified, this defaults to the score of the chunks. - … - `recency_bias` number, float, nullable — Recency Bias lets you determine how much of an effect the recency of chunks will have on the search results. If not specified, this defaults to 0.0. We recommend setting this to 1.0 for a gentle reranking of the results, >3.0 for a strong reranking of the results. - `sort_by` union — Sort by lets you specify a method to sort the results by. If not specified, this defaults to the score of the chunks. If specified, this can be any key in the chunk metadata. This key must be a numeric value within the payload. - … - `tag_weights` object, nullable — Tag weights is a JSON object which can be used to boost the ranking of chunks with certain tags. This is useful for when you want to be able to bias towards chunks with a certain tag on the fly. The keys are the tag names and the values are the weights. - `use_weights` boolean, nullable — Set use_weights to true to use the weights of the chunks in the result set in order to sort them. If not specified, this defaults to true. - `typo_options` TypoOptions — Typo Options lets you specify different methods to correct typos in the query. If not specified, typos will not be corrected. - `correct_typos` boolean, nullable — Set correct_typos to true to correct typos in the query. If not specified, this defaults to false. - `disable_on_word` string[], nullable — Words that should not be corrected. If not specified, this defaults to an empty list. - `one_typo_word_range` TypoRange — The TypoRange struct is used to specify the range of which the query will be corrected if it has a typo. - … - `prioritize_domain_specifc_words` boolean, nullable — Auto-require non-english words present in the dataset to exist in each results chunk_html text. If not specified, this defaults to true. - `two_typo_word_range` TypoRange — The TypoRange struct is used to specify the range of which the query will be corrected if it has a typo. - … - `use_autocomplete` boolean, nullable — Enables autocomplete on the search modal. - `use_quote_negated_terms` boolean, nullable — If true, quoted and - prefixed words will be parsed from the queries and used as required and negated words respectively. Default is false. - `user_id` string, nullable — User ID is the id of the user who is making the request. This is used to track user interactions with the search results. - `searchPageProps` SearchPageProps - `display` boolean, nullable - `filterSidebarProps` SidebarFilters - `sections` FilterSidebarSection[], nullable - … - `searchToolCallOptions` SearchToolCallOptions - `noSearchRagContext` string, nullable - `toolDescription` string, nullable - `userMessageTextPrefix` string, nullable - `showFloatingButton` boolean, nullable - `showFloatingInput` boolean, nullable - `showFloatingSearchIcon` boolean, nullable - `showResultHighlights` boolean, nullable - `singleProductOptions` SingleProductOptions - `enabled` boolean, nullable - `groupTrackingId` string, nullable - `pdpPrompt` string, nullable - `productDescriptionHtml` string, nullable - `productName` string, nullable - `productPrimaryImageUrl` string, nullable - `productQuestions` DefaultSearchQuery[], nullable - `imageUrl` string, nullable - `query` string, nullable - `productTrackingId` string, nullable - `recSearchQuery` string, nullable - `suggestedQueries` boolean, nullable - `tabMessages` PublicPageTabMessage[], nullable - `showComponentCode` boolean, required - `tabInnerHtml` string, required - `title` string, required - `tags` PublicPageTag[], nullable - `description` string, nullable - `iconClassName` string, nullable - `label` string, nullable - `selected` boolean, nullable - `tag` string, required - `theme` 'light' | 'dark' - `type` string, nullable - `useGroupSearch` boolean, nullable - `useLocal` boolean, nullable - `usePagefind` boolean, nullable - `videoLink` string, nullable - `videoPosition` string, nullable - `zIndex` integer, nullable - `QDRANT_ONLY` boolean, nullable — Whether or not to insert chunks into Postgres - `RAG_PROMPT` string, nullable — The prompt to use for the RAG model - `RERANKER_BASE_URL` string, nullable — The base URL for the reranker API - `RERANKER_MODEL_NAME` string, nullable — The model name for the Reranker API - `SEMANTIC_ENABLED` boolean, nullable — Whether to use semantic search - `STOP_TOKENS` string[], nullable — The stop tokens to use - `SYSTEM_PROMPT` string, nullable — The system prompt to use for the LLM - `TEMPERATURE` number, double, nullable — The temperature to use - `TOOL_CONFIGURATION` ToolConfiguration - `query_tool_options` QueryToolOptions - `max_price_option_description` string, nullable - `min_price_option_description` string, nullable - `price_filter_description` string, nullable - `query_parameter_description` string, nullable - `tool_description` string, nullable - `USE_MESSAGE_TO_QUERY_PROMPT` boolean, nullable — Whether to use the message to query prompt - `tracking_id` string, nullable — Optional tracking ID for the dataset. Can be used to track the dataset in external systems. Must be unique within the organization. Strongly recommended to not use a valid uuid value as that will not work with the TR-Dataset header. ## Response `200` Dataset created successfully - Dataset - `created_at` string, date-time, required — Timestamp of the creation of the dataset - `deleted` integer, required — Flag to indicate if the dataset has been deleted. Deletes are handled async after the flag is set so as to avoid expensive search index compaction. - `id` string, uuid, required — Unique identifier of the dataset, auto-generated uuid created by Trieve - `name` string, required — Name of the dataset - `organization_id` string, uuid, required — Unique identifier of the organization that owns the dataset - `server_configuration` unknown, required - `tracking_id` string, nullable — Tracking ID of the dataset, can be any string, determined by the user. Tracking ID's are unique identifiers for datasets within an organization. They are designed to match the unique identifier of the dataset in the user's system. - `updated_at` string, date-time, required — Timestamp of the last update of the dataset ## Other responses - `400` — Service error relating to creating the dataset --- [API](https://skmtc.net/devflowinc/apis/trieve-api.md) · [All operations](https://skmtc.net/devflowinc/apis/trieve-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/devflowinc/trieve-api/revisions/84583e7c9fc1/schema)