v4

latestOpenAPI 3.1.02026-08-01207318738.7 KB

AI Eval

Create an eval prompt set

Create a new eval prompt set bound to a shared model. Initial prompts can be supplied; additional prompts can be added later via PATCH.

post/api/v1/ai/eval/prompt-sets

Request body

descriptionstring nullable

Optional human-readable description of the prompt set. Max 1024 characters.

model_idstring uuid required

The shared model this prompt set is bound to.

namestring required

Human-readable name for the prompt set. 255 characters or fewer.

slugstring required

URL-safe identifier for the prompt set. Must be unique per model_id and match ^[a-z][a-z0-9-]*$. Max 255 characters.

Example request

{
  "description": "Regression suite for the orders topic",
  "model_id": "880e8400-e29b-41d4-a716-446655440003",
  "name": "Orders regression",
  "prompts": [
    {
      "expectation": "The top product by revenue should be Aniseed Syrup.",
      "prompt_text": "What are the top 5 products by revenue?"
    }
  ],
  "slug": "orders-regression"
}

Response

Prompt set created successfully.

Example response

{
  "prompt_set": {
    "created_at": "2025-01-15T10:00:00.000Z",
    "description": "Regression suite for the orders topic",
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "model_id": "880e8400-e29b-41d4-a716-446655440003",
    "name": "Orders regression",
    "prompts": [
      {
        "created_at": "2025-01-15T10:00:00.000Z",
        "expectation": "The top product by revenue should be Aniseed Syrup.",
        "id": "770e8400-e29b-41d4-a716-446655440002",
        "prompt_text": "What are the top 5 products by revenue?",
        "updated_at": "2025-01-15T10:00:00.000Z"
      }
    ],
    "slug": "orders-regression",
    "updated_at": "2025-01-15T10:00:00.000Z"
  }
}