---
title: "Validate Llm Scorer Dataset"
method: POST
path: "/scorers/llm/validate/dataset"
tags: ["data", "prompts", "rows"]
---

# Validate Llm Scorer Dataset

`POST /scorers/llm/validate/dataset`

## Request body

- ValidateLLMScorerDatasetRequest — Request to validate a new LLM scorer against a dataset.
  - `query` string, required
  - `response` string, required
  - `chain_poll_template` ChainPollTemplate, required — Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.
    - `metric_system_prompt` string, nullable — System prompt for the metric.
    - `metric_description` string, nullable — Description of what the metric should do.
    - `value_field_name` string — Field name to look for in the chainpoll response, for the rating.
    - `explanation_field_name` string — Field name to look for in the chainpoll response, for the explanation.
    - `template` string, required — Chainpoll prompt template.
    - `metric_few_shot_examples` FewShotExample[] — Few-shot examples for the metric.
      - `generation_prompt_and_response` string, required
      - `evaluating_response` string, required
    - `response_schema` object, nullable — Response schema for the output
  - `scorer_configuration` GeneratedScorerConfiguration, required
    - `model_alias` string
    - `num_judges` integer
    - `output_type` 'boolean' | 'categorical' | 'count' | 'discrete' | 'freeform' | 'percentage' | 'multilabel' | 'retrieved_chunk_list_boolean' | 'boolean_multilabel' — Enumeration of output types.
    - `scoreable_node_types` string[] — Types of nodes that can be scored by this scorer.
    - `cot_enabled` boolean — Whether chain of thought is enabled for this scorer.
    - `ground_truth` boolean — Whether ground truth is enabled for this scorer.
    - `multimodal_capabilities` MultimodalCapability[], nullable — Multimodal capabilities required by this scorer.
  - `normalized_input` union[], nullable — Optional multimodal content parts. When set, replaces the text-only query/response formatting in the validation job so that file content is passed through to the LLM.
    - union
      - TextContentPart — A text segment within a message.
        - `type` 'text'
        - `text` string, required
      - FileContentPart — Reference to a file associated with this message. The file_id can be resolved via the ``files`` dict returned on trace/span detail responses, which contains metadata such as modality, MIME type, and a presigned download URL.
        - `type` 'file'
        - `file_id` string, uuid4, required
  - `user_prompt` string, required
  - `dataset_id` string, uuid4, required
  - `dataset_version_index` integer, nullable
  - `limit` integer — Maximum number of dataset rows to process.
  - `starting_token` integer, nullable — Pagination offset into dataset rows.
  - `sort` object, nullable — Optional sort configuration for dataset rows.

## Response `200`

Successful Response

- ValidateLLMScorerDatasetResponse
  - `metrics_experiment_id` string, uuid4, required
  - `project_id` string, uuid4, required

## Other responses

- `422` — Validation Error

---

[API](https://skmtc.net/galileo/apis/galileo-api-server.md) · [All operations](https://skmtc.net/galileo/apis/galileo-api-server/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/galileo/galileo-api-server/versions/933e8c8e6366/schema)