---
title: "Validate Rule"
method: POST
path: "/api/v1/runs/rules/validate"
tags: ["run"]
---

# Validate Rule

`POST /api/v1/runs/rules/validate`

Validate a rule by executing it with test data without creating a saved rule.

This endpoint allows testing LLM-as-judge evaluators before saving them. It accepts
a rule configuration (same as rule creation) and test data, executes the evaluator,
and returns the evaluation results in the same format as batch_invoke_evaluator.

Only LLM-as-judge rules (evaluators) are supported. Code evaluators are not allowed.

The evaluator execution traces are written to the database (in the "evaluators"
project), which allows users to see the evaluator execution history.

## Request body

- RunRulesValidateSchema — Schema for validating rules without creating them. Extends RunRulesCreateSchema with test data fields for validation. Only LLM-as-judge rules (evaluators) are supported, not code_evaluators.
  - `display_name` string, required
  - `session_id` string, uuid, nullable
  - `is_enabled` boolean
  - `dataset_id` string, uuid, nullable
  - `sampling_rate` number, required
  - `filter` string, nullable
  - `trace_filter` string, nullable
  - `tree_filter` string, nullable
  - `backfill_from` string, date-time, nullable
  - `use_corrections_dataset` boolean
  - `num_few_shot_examples` integer, nullable
  - `extend_only` boolean
  - `transient` boolean
  - `add_to_annotation_queue_id` string, uuid, nullable
  - `add_to_dataset_id` string, uuid, nullable
  - `add_to_dataset_prefer_correction` boolean
  - `evaluators` EvaluatorTopLevel[], nullable
    - `structured` EvaluatorStructuredOutput, required — Evaluator structured output schema.
      - `hub_ref` string, nullable
      - `prompt` array[], nullable
        - unknown[]
          - unknown
      - `template_format` string, nullable
      - `schema` object, nullable
      - `variable_mapping` object, nullable
      - `model` object, nullable
  - `code_evaluators` CodeEvaluatorTopLevel[], nullable
    - `code` string, required
    - `language` 'python' | 'javascript'
  - `evaluator_id` string, uuid, nullable
  - `alerts` RunRulesPagerdutyAlertSchema[], nullable
    - `type` 'pagerduty' — Enum for alert types.
    - `routing_key` string, required
    - `summary` string, nullable
    - `severity` 'critical' | 'warning' | 'error' | 'info' — Enum for severity.
  - `webhooks` RunRulesWebhookSchema[], nullable
    - `url` string, required
    - `headers` object, nullable
  - `evaluator_version` integer, nullable
  - `create_alignment_queue` boolean
  - `include_extended_stats` boolean
  - `group_by` 'thread_id', nullable
  - `spend_limit` RunRuleSpendLimitSchemaInput
    - `limit_usd` union, required
      - number
      - string
    - `window` 'weekly', required
  - `tracer_session_issue_id` string, uuid, nullable
  - `test_inputs` object, nullable
  - `test_outputs` object, nullable
  - `test_reference_outputs` object, nullable
  - `test_attachments` object, nullable

## Response `200`

Successful Response

- object[]

## Other responses

- `422` — Validation Error

---

[API](https://skmtc.net/langchain-ai/apis/langsmith.md) · [All operations](https://skmtc.net/langchain-ai/apis/langsmith/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/langchain-ai/langsmith/versions/a0acb3a6a101/schema)