---
title: "List"
method: GET
path: "/evaluations/"
tags: ["evaluations"]
---

# List

`GET /evaluations/`

## Creating an Evaluator

### LLM Evaluators

For LLM evaluators, the frontend should first fetch available evaluation forms from the
`/eval-forms/` endpoint to get the template configuration, then fill in the form and submit.

**Required fields:**
- `name` (str): Display name for the evaluator
- `evaluator_slug` (str): Unique identifier for the evaluator within the organization
- `type` (str): llm, human_boolean, human_categorical, human_numerical, human_text
- `configurations` (dict): Complete evaluation form configuration
- `description` (str, optional): Description of what this evaluator does
- `enabled` (bool, optional): Whether the evaluator is active (default: False)

**Example request body for LLM evaluator:**
```json
{
    "name": "Output Length Checker",
    "type": "llm",
    "description": "Checks if the output meets character count requirements",
    "enabled": true,
    "configurations": {
        "eval_class": "output_char_count",
        "type": "function",
        "note": "",
        "display_name": "Output Character Count",
        "description": "Evaluates the length of the output text",
        "special_fields": [],
        "required_fields": [
            {
                "name": "llm_output",
                "display_name": "LLM Output",
                "type": "textarea",
                "description": "The output text to evaluate",
                "required": true,
                "default_value": null,
                "placeholder": "",
                "choices": [],
                "value": null
            }
        ],
        "inference_filters": [],
        "allow_conditions": true,
        "score_mapping": {
            "primary_score": "output_char_count",
            "secondary_score": null,
            "tertiary_score": null,
            "quaternary_score": null
        },
        "category": "custom"
    }
}
```

### Human Annotation Evaluators

For human annotation evaluators, specify the type and provide choices for categorical evaluators.

**Human Boolean Evaluator:**
```json
{
    "name": "Quality Check",
    "evaluator_slug": "quality_check",
    "type": "human_boolean",
    "description": "Manual quality assessment"
}
```

**Human Categorical Evaluator:**
```json
{
    "name": "Sentiment Rating",
    "evaluator_slug": "sentiment_rating",
    "type": "human_categorical",
    "description": "Manual sentiment classification",
    "categorical_choices": [
        {"name": "Positive", "value": 1},
        {"name": "Neutral", "value": 0},
        {"name": "Negative", "value": -1}
    ]
}
```

**Human Numerical Evaluator:**
```json
{
    "name": "Quality Score",
    "evaluator_slug": "quality_score",
    "type": "human_numerical",
    "description": "Rate quality from 1-10"
}
```

**Human Text Evaluator:**
```json
{
    "name": "Feedback Comments",
    "evaluator_slug": "feedback_comments",
    "type": "human_text",
    "description": "Detailed feedback comments"
}
```

## Response

Returns the created evaluator with all fields populated, including auto-generated fields like
`id`, `created_at`, `updated_at`, and `evaluator_slug`.

## Validation

- For LLM evaluators: The `configurations` field is validated against the corresponding
  evaluation form schema from `EVAL_FORMS_MAP`
- For human categorical evaluators: `categorical_choices` must be a list of objects
  with `name` and `value` fields
- The `eval_class` in configurations must exist in the available evaluation forms

## Notes

- The `organization`, `created_by`, and `updated_by` fields are automatically set from
  the authenticated user
- Each evaluator gets a unique `evaluator_slug` within the organization
- LLM evaluators require a valid `eval_class` that maps to an available evaluation function

## Query parameters

- `page` integer
- `page_size` integer

## Headers

- `Authorization` string, required

## Response `200`

- PaginatedPublicEvaluatorListList
  - `count` integer, required
  - `next` string, uri, nullable
  - `previous` string, uri, nullable
  - `total_count` integer
  - `current_filters` FilterParamDictPydantic — Pydantic model for FilterParamDict. A dictionary that maps metric names to their filter parameters. Each key is a metric name (str), and each value can be: - A single MetricFilterParamPydantic (one condition) - A List[MetricFilterParamPydantic] (multiple conditions for same metric) - A FilterBundlePydantic (nested filter bundle with connector) Note: Uses extra="allow" for dynamic metric name fields. The __pydantic_extra__ annotation tells Pydantic what types to expect for extra fields, and generates typed additionalProperties in JSON Schema.
  - `filters_data` PaginatedPublicEvaluatorListListFiltersData
  - `results` PublicEvaluatorList[], required
    - `version_id` string
    - `configurations` object
    - `categorical_choices` object[], required
    - `score_config` object
    - `passing_conditions` object, nullable
    - `llm_config` object, nullable
    - `code_config` object, nullable
    - `project` string, nullable
    - `is_public` boolean, required
    - `created_by` Editor, required
      - `id` integer, required
      - `email` string, required
      - `name` string, required
      - `username` string
      - `first_name` string
      - `last_name` string
    - `updated_by` Editor, required
      - `id` integer, required
      - `email` string, required
      - `name` string, required
      - `username` string
      - `first_name` string
      - `last_name` string
    - `editor` Editor, required
      - `id` integer, required
      - `email` string, required
      - `name` string, required
      - `username` string
      - `first_name` string
      - `last_name` string
    - `tags` GenericTagDisplay[], required
      - `id` string, required
      - `name` string, required
      - `color` string, required
      - `created_at` string, date-time, required
      - `updated_at` string, date-time, required
    - `version_count` integer, required — Number of versions for this evaluator. Reads the ``version_count`` annotation added by the list querysets (see ``annotate_evaluator_version_count``). Falls back to 0 when the queryset was not annotated, so the serializer never issues a per-row count query.
    - `id` string
    - `version` integer
    - `is_read_only` boolean
    - `version_description` string
    - `evaluator_slug` string
    - `name` string, required
    - `description` string
    - `created_at` string, date-time, required
    - `updated_at` string, date-time, required
    - `type` 'llm' | 'code' | 'human' | 'function' | 'human_numerical' | 'human_categorical' | 'human_boolean' | 'human_text' | 'custom' — * `llm` - Llm * `code` - Code * `human` - Human * `function` - Function * `human_numerical` - Human Numerical * `human_categorical` - Human Categorical * `human_boolean` - Human Boolean * `human_text` - Human Text * `custom` - Custom
    - `score_value_type` 'numerical' | 'boolean' | 'percentage' | 'single_select' | 'multi_select' | 'text' | 'json' | 'comment' | 'categorical' — * `numerical` - Numerical * `boolean` - Boolean * `percentage` - Percentage * `single_select` - Single Select * `multi_select` - Multi Select * `text` - Text * `json` - Json * `comment` - Comment * `categorical` - Categorical
    - `eval_class` union
      - 'ragas_faithfulness' | 'ragas_noise_sensitivity' | 'ragas_response_relevancy' | 'ragas_answer_relevancy' | 'ragas_context_precision' | 'ragas_context_recall' | 'ragas_context_entity_recall' | 'ragas_factual_correctness' | 'ragas_semantic_similarity' | 'ragas_non_llm_string_similarity' | 'ragas_non_llm_string_presence' | 'ragas_non_llm_exact_match' | 'relari_llm_based_custom_metric' | 'relari_llm_based_answer_correctness' | 'keywordsai_custom_evaluator' | 'keywordsai_custom_llm' | 'output_char_count' | 'output_word_count' | 'custom_code' — * `ragas_faithfulness` - ragas_faithfulness * `ragas_noise_sensitivity` - ragas_noise_sensitivity * `ragas_response_relevancy` - ragas_response_relevancy * `ragas_answer_relevancy` - ragas_answer_relevancy * `ragas_context_precision` - ragas_context_precision * `ragas_context_recall` - ragas_context_recall * `ragas_context_entity_recall` - ragas_context_entity_recall * `ragas_factual_correctness` - ragas_factual_correctness * `ragas_semantic_similarity` - ragas_semantic_similarity * `ragas_non_llm_string_similarity` - ragas_non_llm_string_similarity * `ragas_non_llm_string_presence` - ragas_non_llm_string_presence * `ragas_non_llm_exact_match` - ragas_non_llm_exact_match * `relari_llm_based_custom_metric` - relari_llm_based_custom_metric * `relari_llm_based_answer_correctness` - relari_llm_based_answer_correctness * `keywordsai_custom_evaluator` - keywordsai_custom_evaluator * `keywordsai_custom_llm` - keywordsai_custom_llm * `output_char_count` - output_char_count * `output_word_count` - output_word_count * `custom_code` - custom_code
      - ''
    - `custom_required_fields` string[]
    - `starred` boolean

---

[API](https://skmtc.net/keywordsai/apis/api-reference.md) · [All operations](https://skmtc.net/keywordsai/apis/api-reference/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/keywordsai/api-reference/versions/c26d550029f8/schema)