--- title: "Create Eval Run Suite Report" method: POST path: "/v2/eval-reports" tags: ["Eval Reports"] --- # Create Eval Run Suite Report `POST /v2/eval-reports` Create a summary for a suite of eval runs. ## Query parameters - `workspace_uuid` string ## Request body - EvalSuiteReportRequest — Schema for creating an eval run suite summary. - `eval_run_uuids` string[], required — List of eval run UUIDs to include in the suite summary. ## Response `200` OK - EvalSuiteReport — Schema for returning eval run suite summary data. - `eval_suite_report_uuid` string, required — Unique identifier for the eval suite report. - `status` 'created' | 'processing' | 'finished' | 'failed', required — Resource status. - `eval_run_reports` EvalRunReport[], required — List of eval run reports included in the suite. - `eval_run_report_uuid` string, required — Unique identifier for the eval run report. - `eval_run_uuid` string, required — Unique identifier for the eval run. - `eval_run` EvalRunResult, required — Schema for returning eval run data. - `eval_run_uuid` string, required — Unique identifier for the eval run. - `eval_uuid` string, required — Unique identifier for the eval. - `name` string, nullable — Name of the eval run, if any (defaults to the eval name + timestamp). - `status` 'created' | 'processing' | 'finished' | 'failed', required — Resource status. - `created_at` string, date-time, required — Timestamp when the eval run was created. - `updated_at` string, date-time, required — Timestamp when the eval run was last updated. - `evaluation` Eval — Schema for configuring an Eval based on a eval_type. - `eval_uuid` string, nullable — Unique identifier for the evaluation. - `name` string, nullable — Name of the evaluation. - `created_by` string, nullable — Name of the user who created the evaluation. - `ai_description` string, required — Description of the AI under evaluation. - `ai_instructions` union — Instructions the AI should follow. String for normal evals, AgentInstructions for single-agent evals, WorkflowInstructions for multi-agent workflows. - string - AgentInstructions - `agent_name` string, nullable - `system_prompt` string, required - `tools` union — Instructions for the agent, can be a string or a list/dict of tools. - … - WorkflowInstructions — Workflow instructions for multi-agent workflows. - `instructions` AgentInstructions[], required — List of agent instructions for the workflow. Must contain at least one agent. - … - `eval_type` string, required — Type of the eval (safety, accuracy, etc.) - `eval_instructions` string, nullable — Additional instructions for the eval, if any. - `language` string, nullable — Language code for the eval (default: "en"). - `modality` 'text' | 'image' | 'video' — Content type for AI interactions. - `ground_truth` union — Ground truth data or reference file, if any. - string - FileReference — Reference to a file, either by file_uuid (preferred) or legacy remote_file_path. When file_uuid is provided, the system will look up the File record and use its remote_file_path. The remote_file_path field is maintained for backwards compatibility. - `file_uuid` string, nullable - `remote_file_path` string, nullable - `num_prompts` integer, nullable — Number of prompts/questions in the eval (default: 50). - `prompt_examples` PromptExample[], nullable — List of example prompts for the eval. - `content` string, required — Content of the example prompt. - `example_uuid` string, nullable — Unique identifier for the example, if any. - `type` 'good' | 'bad' - `explanation` string, nullable — Explanation for the example, if any. - `is_jailbreak` boolean — Indicates if the eval is a jailbreak test. - `is_sandbox` boolean — Indicates if the eval results are sandboxed. - `workspace_uuid` string, nullable — UUID of the associated workspace, if any. - `status` 'created' | 'processing' | 'finished' | 'failed' — Resource status. - `created_at` string, date-time, nullable — Timestamp when the eval was created. - `updated_at` string, date-time, nullable — Timestamp when the eval was last updated. - `ai_description` string, nullable — Description of the AI for this run, if any. - `is_sandbox` boolean — Indicates if the eval run is sandboxed (default: False). - `workspace_uuid` string, nullable — UUID of the associated workspace, if any. - `pass_rate` number, nullable — Pass rate for the eval run, if any. - `num_prompts` integer, nullable — Number of prompts in the eval run, if any. - `num_responses_scored` integer, nullable — Number of responses scored in the eval run, if any. - `eval_run_examples` EvalRunExample[], nullable — Examples that were included with the eval run, if any. - `example_uuid` string, nullable — Unique identifier for the example, if any. - `type` 'pass' | 'fail', required — Type of the example: "pass" or "fail". - `prompt` string, required — Prompt text for the example. - `response` string, required — Expected response for the example. - `explanation` string, nullable — Explanation for the example, if any. - `responses` EvalScoredResponse[], nullable — List of scored responses for the eval run, if any. - `prompt_uuid` string, required — Unique identifier for the prompt. - `thread_uuid` string, nullable — Unique identifier for the thread, if any. - `turn_number` integer — Turn number in the conversation (default: 1). - `continue_thread` boolean — Whether to continue the thread after this response. - `content` union — Content of the AI response or a file reference. - string - FileReference — Reference to a file, either by file_uuid (preferred) or legacy remote_file_path. When file_uuid is provided, the system will look up the File record and use its remote_file_path. The remote_file_path field is maintained for backwards compatibility. - `file_uuid` string, nullable - `remote_file_path` string, nullable - `content_type` 'text' | 'image' | 'video' — Content type for AI interactions. - `exclude_from_scoring` boolean — Whether to exclude this response from scoring. - `ai_refused` boolean — Whether the AI refused to answer the prompt. - `response_uuid` string, nullable — Unique identifier for the response, if any. - `explanation` string, nullable — Explanation for the response, if any. - `confidence` number, nullable — Confidence score for the response, if any. - `is_passed` boolean, nullable — Whether the response passed the evaluation, if any. - `next_prompt` EvalPrompt - `prompt_uuid` string, required — Unique identifier for the prompt. - `thread_uuid` string, nullable — Unique identifier for the thread, if any. - `turn_number` integer — Turn number in the conversation (default: 1). - `content` string, required — Content of the prompt. - `category` string, nullable — Category of the prompt, if any. - `manually_updated` boolean — Whether this response was manually updated. - `manually_updated_by` UserOutSchema - `email` string, required - `organization` OrganizationOutSchema - `org_uuid` string, required - `name` string, required - `workspace` WorkspaceOutSchema - `workspace_uuid` string, nullable - `name` string, nullable - `organization_name` string, nullable - `organization_uuid` string, nullable - `org_admin_emails` string[], nullable - `is_admin` boolean, required - `is_impersonating` boolean, required - `feature_flags` FeatureFlags, required - `enable_dashboard` boolean, required - `manually_updated_at` string, date-time, nullable — Timestamp when this response was last manually updated. - `update_history` AnswerHistoryOut[], nullable — History of manual updates, if requested. - `answer_history_uuid` string, required — Unique identifier for the history entry. - `updated_by` UserOutSchema - `email` string, required - `organization` OrganizationOutSchema - … - `workspace` WorkspaceOutSchema - … - `org_admin_emails` string[], nullable - `is_admin` boolean, required - `is_impersonating` boolean, required - `feature_flags` FeatureFlags, required - … - `previous_is_passed` boolean, nullable — Previous is_passed value. - `previous_confidence` number, nullable — Previous confidence value. - `previous_explanation` string, nullable — Previous explanation value. - `new_is_passed` boolean, nullable — New is_passed value. - `new_confidence` number, nullable — New confidence value. - `new_explanation` string, nullable — New explanation value. - `updated_at` string, date-time, required — Timestamp when the update was made. - `passing_responses_summary` string, required — Summary of passing responses. - `failing_responses_summary` string, required — Summary of failing responses. - `improvement_advice` string, required — Advice for improving future responses. - `overall_improvement_advice` string, nullable — Overall advice for improving future responses, if any. - `overall_failing_responses_summary` string, nullable — Overall summary of failing responses, if any. - `overall_passing_responses_summary` string, nullable — Overall summary of passing responses, if any. - `created_at` string, date-time, required — Timestamp when the eval suite report was created. - `updated_at` string, date-time, required — Timestamp when the eval suite report was last updated. - `remaining_reports` integer, nullable — Number of remaining reports to be generated, if any. ## Other responses - `400` — Bad Request - `401` — Unauthorized - `403` — Forbidden - `404` — Not Found - `409` — Conflict - `422` — Unprocessable Entity - `429` — Too Many Requests - `500` — Internal Server Error - `503` — Service Unavailable --- [API](https://skmtc.net/aymara-ai/apis/aymara-api.md) · [All operations](https://skmtc.net/aymara-ai/apis/aymara-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/aymara-ai/aymara-api/versions/0cb64ef11adf/schema)