---
title: "Generate evaluation dataset from interactions"
method: POST
path: "/v1/retrievers/{retriever_id}/evaluations/generate-from-interactions"
tags: ["Retriever Evaluations"]
---

# Generate evaluation dataset from interactions

`POST /v1/retrievers/{retriever_id}/evaluations/generate-from-interactions`

Auto-generate a ground truth evaluation dataset by mining user interactions for this retriever. Queries with sufficient positive interactions (click, purchase, bookmark, etc.) are used to infer relevance labels, creating a dataset suitable for offline evaluation.

## Path parameters

- `retriever_id` string, required

## Request body

- GenerateFromInteractionsRequest — Request to auto-generate an evaluation dataset from user interactions.
  - `dataset_name` string, nullable — Name for the generated dataset. If omitted, defaults to 'auto_{retriever_id}_{date}'.
  - `min_interactions` integer — Minimum number of positive interactions a query must have to be included in the dataset. Queries with fewer interactions are excluded as low-confidence ground truth.
  - `lookback_days` integer — How many days of interaction history to consider.
  - `positive_interaction_types` string[], nullable — Interaction types considered positive signals. Defaults to: click, purchase, positive_feedback, bookmark, share.

## Response `201`

Successful Response

- EvaluationDataset — Complete evaluation dataset with metadata. An evaluation dataset is a collection of queries with ground truth relevance labels, used to measure retriever quality.
  - `dataset_id` string, required — Unique dataset identifier
  - `dataset_name` string, required — Human-readable dataset name
  - `description` string, nullable — Dataset description
  - `queries` GroundTruthQuery[], required — List of queries with ground truth
    - `query_id` string, required — Unique identifier for this query within the dataset
    - `query_input` object, required — Query input in the same format as retriever execution (e.g., {'text': '...'})
    - `relevant_documents` string[], required — List of feature_ids that are relevant for this query
    - `relevance_scores` object, nullable — Optional graded relevance scores (doc_id -> score, 0-5 where 5 is most relevant). Used for NDCG calculation.
  - `created_at` string, date-time, required — When dataset was created
  - `updated_at` string, date-time, required — Last update timestamp
  - `namespace_id` string, required — Namespace this dataset belongs to
  - `internal_id` string, required — Internal organization ID
  - `query_count` integer, required — Number of queries in dataset
  - `metadata` object, nullable — Additional metadata (e.g., labeling instructions, version info)

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `403` — Forbidden
- `404` — Not Found
- `422` — Validation Error
- `500` — Internal Server Error

---

[API](https://skmtc.net/mixpeek/apis/mixpeek-api.md) · [All operations](https://skmtc.net/mixpeek/apis/mixpeek-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/mixpeek/mixpeek-api/revisions/40af5e67ba94/schema)