Sign in

v26

latestOpenAPI 3.1.0raw.githubusercontent.com2026-02-081039106.2 KB
Prompt Optimization

Get prompt optimization results

Retrieve the complete results of a prompt optimization run, including optimized prompts for all target models.

This endpoint returns the optimized prompts and evaluation metrics for each target model in your optimization request. Call this endpoint after the optimization status is 'completed' to get your optimized prompts.

Response Structure:

  • origin_model: Baseline performance of your original prompt on the origin model
    • Includes: system_prompt, user_message_template, score, evaluation metrics, cost
  • target_models: Array of results for each target model
    • Includes: optimized system_prompt, user_message_template, template_fields
    • pre_optimization_score: Performance before optimization
    • post_optimization_score: Performance after optimization
    • Evaluation metrics and cost information

Using Optimized Prompts:

  1. Extract the system_prompt and user_message_template from each target model result
  2. Use user_message_template_fields to know which fields to substitute
  3. Apply the optimized prompts when calling the respective target models
  4. Compare pre/post optimization scores to see improvement

Status Handling:

  • If optimization is still processing, target model results will have result_status: "processing"
  • Only completed target models will have system_prompt and template values
  • Failed target models will have result_status: "failed" with null values

Cost Information:

  • Each model result includes cost in USD for the optimization process
  • Costs vary based on model pricing and number of evaluation examples
  • Typical range: $0.10 - $2.00 per target model

Best Practices:

  1. Wait for status 'completed' before calling this endpoint
  2. Check result_status for each target model
  3. Validate that post_optimization_score > pre_optimization_score
  4. Save optimized prompts for production use
  5. A/B test optimized prompts against originals in production
get/v2/prompt/optimizeResults/{optimization_run_id}

Path parameters

optimization_run_idstring required

Response

Successfully retrieved optimization results

evaluation_metricstring nullable
evaluation_configstring nullable
idstring required

Unique ID for this adaptation run

created_atstring date-time required

Timestamp when this adaptation run was created

updated_atstring date-time nullable required

Timestamp of last update to this adaptation run

job_status'created' | 'queued' | 'processing' | 'completed' | 'failed' | 'cancelled' required

Status enum for asynchronous jobs (prompt adaptation, custom router training, etc.).

Represents the current state of a long-running operation:

  • created: Job has been initialized but not yet queued
  • queued: Job is waiting in the queue to be processed
  • processing: Job is currently being executed
  • completed: Job finished successfully and results are available
  • failed: Job encountered an error and did not complete
  • cancelled: Job was cancelled due to a restart operation
llm_request_metricsobject[] nullable

Metrics for the LLM requests made during the adaptation run. List of {model: str, num_requests: int}.

prototype_modeboolean

Whether this adaptation run was created with prototype mode (3-24 training examples allowed). Prototype mode may have degraded performance compared to standard mode (25+ examples)

Example response

{
  "created_at": "2024-01-15T10:00:00.123456",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "job_status": "completed",
  "llm_request_metrics": {
    "avg_latency_ms": 1250.5,
    "total_requests": 45
  },
  "origin_model": {
    "cost": 0.012,
    "evaluation_type": "llm_as_judge",
    "model": "openai/gpt-4",
    "score": 0.85,
    "system_prompt": "You are a helpful assistant.",
    "user_message_template": "Question: {question}"
  },
  "target_models": [
    {
      "cost": 0.018,
      "evaluation_type": "llm_as_judge",
      "model": "anthropic/claude-3-sonnet-20240229",
      "post_optimization_score": 0.89,
      "pre_optimization_score": 0.72,
      "score_improvement": 0.17,
      "system_prompt": "You are an expert assistant focused on providing clear, accurate answers.",
      "user_message_template": "Question: {question}\n\nProvide a detailed response.",
      "user_message_template_fields": [
        "question"
      ]
    }
  ],
  "updated_at": "2024-01-15T10:15:30.789012"
}