---
title: "Analyze Evals"
method: POST
path: "/v2/eval-analysis"
tags: ["Eval Analysis"]
---

# Analyze Evals

`POST /v2/eval-analysis`

Analysis for evals with advanced filtering and aggregated statistics.

This endpoint allows analyzing across both eval metadata and score run performance data,
providing comprehensive filtering capabilities and aggregated statistics for each eval.

Args:
    analysis_request (EvalAnalysisRequest): Analysis parameters and filters including:
        - Eval metadata filters (name, type, status, language, etc.)
        - Score run performance filters (pass rate, run count, etc.)
        - Sorting and pagination options

Returns:
    EvalAnalysisResponse: Paginated results with matching evals and their statistics

Raises:
    AymaraAPIError: If the request is invalid or analysis parameters are malformed

Example:
    POST /api/v2/eval_analysis
    {
        "name": "safety",
        "eval_type": "safety",
        "min_pass_rate": 0.8,
        "has_score_runs": true,
        "sort_by": "pass_rate",
        "sort_order": "desc",
        "limit": 20,
        "offset": 0
    }

## Request body

- EvalAnalysisRequest — Schema for eval analysis request parameters.
  - `name` string, nullable — Filter by eval names (case-insensitive partial match)
  - `eval_type` string, nullable — Filter by eval type (safety, accuracy, jailbreak, image_safety)
  - `status` string, nullable — Filter by eval status (created, processing, finished, failed)
  - `language` string, nullable — Filter by language code (e.g., en, es)
  - `modality` string, nullable — Filter by modality (text, image)
  - `created_by` string, nullable — Filter by creator email
  - `created_after` string, date-time, nullable — Filter evals created after this date
  - `created_before` string, date-time, nullable — Filter evals created before this date
  - `is_jailbreak` boolean, nullable — Filter by jailbreak status
  - `is_sandbox` boolean, nullable — Filter by sandbox status
  - `workspace_uuid` string, nullable — Filter by workspace UUID
  - `min_pass_rate` number, nullable — Minimum average pass rate (0.0-1.0)
  - `max_pass_rate` number, nullable — Maximum average pass rate (0.0-1.0)
  - `has_score_runs` boolean, nullable — Only include evals that have score runs
  - `score_run_status` string, nullable — Filter by any score run status
  - `run_created_after` string, date-time, nullable — Filter by score runs created after this date
  - `run_created_before` string, date-time, nullable — Filter by score runs created before this date
  - `sort_by` 'created_at' | 'updated_at' | 'name' | 'pass_rate' | 'num_score_runs' | 'last_run_date' — Field to sort by
  - `sort_order` 'asc' | 'desc' — Sort order
  - `limit` integer — Maximum number of results (1-100)
  - `offset` integer — Number of results to skip

## Response `200`

OK

- EvalAnalysisResponse — Schema for eval analysis response.
  - `results` EvalAnalysisResult[], required — List of matching evals with statistics
    - `eval` Eval, required — Schema for configuring an Eval based on a eval_type.
      - `eval_uuid` string, nullable — Unique identifier for the evaluation.
      - `name` string, nullable — Name of the evaluation.
      - `created_by` string, nullable — Name of the user who created the evaluation.
      - `ai_description` string, required — Description of the AI under evaluation.
      - `ai_instructions` union — Instructions the AI should follow. String for normal evals, AgentInstructions for single-agent evals, WorkflowInstructions for multi-agent workflows.
        - string
        - AgentInstructions
          - `agent_name` string, nullable
          - `system_prompt` string, required
          - `tools` union — Instructions for the agent, can be a string or a list/dict of tools.
            - ToolArray — Container for an array of tools
              - …
            - ToolDict — Container for a tool dictionary
              - …
            - ToolString — Container for string-based tool instructions
              - …
        - WorkflowInstructions — Workflow instructions for multi-agent workflows.
          - `instructions` AgentInstructions[], required — List of agent instructions for the workflow. Must contain at least one agent.
            - `agent_name` string, nullable
            - `system_prompt` string, required
            - `tools` union — Instructions for the agent, can be a string or a list/dict of tools.
              - …
      - `eval_type` string, required — Type of the eval (safety, accuracy, etc.)
      - `eval_instructions` string, nullable — Additional instructions for the eval, if any.
      - `language` string, nullable — Language code for the eval (default: "en").
      - `modality` 'text' | 'image' | 'video' — Content type for AI interactions.
      - `ground_truth` union — Ground truth data or reference file, if any.
        - string
        - FileReference — Reference to a file, either by file_uuid (preferred) or legacy remote_file_path. When file_uuid is provided, the system will look up the File record and use its remote_file_path. The remote_file_path field is maintained for backwards compatibility.
          - `file_uuid` string, nullable
          - `remote_file_path` string, nullable
      - `num_prompts` integer, nullable — Number of prompts/questions in the eval (default: 50).
      - `prompt_examples` PromptExample[], nullable — List of example prompts for the eval.
        - `content` string, required — Content of the example prompt.
        - `example_uuid` string, nullable — Unique identifier for the example, if any.
        - `type` 'good' | 'bad'
        - `explanation` string, nullable — Explanation for the example, if any.
      - `is_jailbreak` boolean — Indicates if the eval is a jailbreak test.
      - `is_sandbox` boolean — Indicates if the eval results are sandboxed.
      - `workspace_uuid` string, nullable — UUID of the associated workspace, if any.
      - `status` 'created' | 'processing' | 'finished' | 'failed' — Resource status.
      - `created_at` string, date-time, nullable — Timestamp when the eval was created.
      - `updated_at` string, date-time, nullable — Timestamp when the eval was last updated.
    - `stats` EvalStats, required — Aggregated statistics for an eval across all its score runs.
      - `total_score_runs` integer, required — Total number of score runs for this eval
      - `avg_pass_rate` number, nullable — Average pass rate across all score runs
      - `best_pass_rate` number, nullable — Best (highest) pass rate achieved
      - `worst_pass_rate` number, nullable — Worst (lowest) pass rate achieved
      - `last_run_date` string, date-time, nullable — Date of the most recent score run
      - `total_responses_scored` integer, required — Total number of responses scored across all runs
  - `total_count` integer, required — Total number of evals matching the analysis criteria
  - `has_more` boolean, required — Whether there are more results available

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `403` — Forbidden
- `404` — Not Found
- `409` — Conflict
- `422` — Unprocessable Entity
- `429` — Too Many Requests
- `500` — Internal Server Error
- `503` — Service Unavailable

---

[API](https://skmtc.net/aymara-ai/apis/aymara-api.md) · [All operations](https://skmtc.net/aymara-ai/apis/aymara-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/aymara-ai/aymara-api/revisions/0cb64ef11adf/schema)