v1

latestOpenAPI 3.1.0raw.githubusercontent.com2026-04-21149158448.6 KB

Diamond

evaluations

Create Evaluation

Create a new evaluation job.

Creates a Kubernetes Job to run the evaluation and tracks it in memory. Fetches agent configuration from Agent Registry using the provided agent_id. JWT token is retrieved from request context (set by JWTAuthMiddleware).

Args: request: Evaluation configuration with agent_id and team_id claims: JWT claims with user and team info diamond_domain: Diamond domain orchestrator

Returns: Evaluation ID, status, and status URL

post/v1/evaluations/

Request body

agent_idstring uuid required

UUID of the agent from Agent Registry

team_idstring uuid required

UUID of the team that owns this evaluation (required)

typestring

Type of evaluation to run. Currently only 'behavioral' is supported.

harness_namesstring[] required

List of harnesses to run (e.g., ['safety', 'ethics', 'privacy', 'security', 'toxicity'])

harness_type'standard' | 'custom'

Type of harness.

sample_sizeinteger nullable

Number of prompts to randomly sample per harness. If omitted, all prompts run (~1250 for security). Recommended: 10 for fast iteration, 50 for moderate, 100 for thorough.

evaluation_idstring uuid nullable

Optional UUID for the evaluation. If provided, this UUID will be used when creating the evaluation in Diamond. If not provided, Diamond will generate one.

Response

Successful Response

evaluation_idstring uuid required

statusstring required