Adapt prompts across LLMs

Optimize your prompt from one LLM to work optimally across different target LLMs.

This endpoint automatically optimizes your prompt (system prompt + user message template) to improve accuracy on your use case across various models. Each model has unique characteristics, and what works well for GPT-5 might not work as well for Claude or Gemini.

How Prompt Optimization Works:

You provide your current prompt and optionally your current origin model
You specify the target models you want to optimize your prompt to
You provide evaluation examples (golden records) with expected answers
The system runs optimization to find the best prompt for each target model
You receive optimized prompts that perform well on your target models

Evaluation Metrics: Choose either a standard metric or provide custom evaluation:

Standard metrics: LLMaaJ:Sem_Sim_1 (semantic similarity), JSON_Match
Custom evaluation: Provide evaluation_config with your own LLM judge, prompt, and cutoff

Dataset Requirements:

Minimum 25 examples in train_goldens (more examples = better optimization)
Prototype mode: Set prototype_mode: true to use as few as 3 examples for prototyping
- Recommended when you don't have enough data yet to build a proof-of-concept
- Note: Performance may be degraded compared to standard mode (25+ examples)
- Trade-off: Faster iteration with less data vs. potentially less generalizability
Each example must have fields matching your template placeholders
Supervised evaluation requires 'answer' field in each golden record
Unsupervised evaluation can work without answers

Training Time:

Processing is asynchronous and typically takes 10-30 minutes
Time depends on: number of target models, dataset size, model availability
Use the returned optimization_run_id to check status and retrieve results

Example Workflow:

1. POST /v2/prompt/optimize - Submit optimization request
2. GET /v2/prompt/optimizeStatus/{id} - Poll status until completed
3. GET /v2/prompt/optimizeResults/{id} - Retrieve optimized prompts
4. Use optimized prompts in production with target models

Request body

evaluation_metricstring nullable

evaluation_configstring nullable

system_promptstring required

System prompt to use with the origin model. This sets the context and role for the LLM

templatestring required

User message template with placeholders for fields. Use curly braces for field substitution

fieldsstring[] required

List of field names that will be substituted into the template. Must match keys in golden records

origin_model_evaluation_scorenumber nullable

Optional baseline score for the origin model. If provided, can skip origin model evaluation

prototype_modeboolean

Enable prototype mode to use as few as 3 training examples (instead of 25). Note: Performance may be degraded with fewer examples. Recommended for prototyping AI applications when you don't have enough data yet

Example request

{
  "system_prompt": "You are a helpful assistant that answers questions about science.",
  "template": "Question: {question}\nContext: {context}\nAnswer:",
  "fields": [
    "question",
    "context"
  ],
  "goldens": [
    {
      "fields": {
        "context": "Basic arithmetic",
        "question": "What is 2+2?"
      },
      "answer": "4"
    }
  ],
  "train_goldens": [
    {
      "fields": {
        "context": "Basic arithmetic",
        "question": "What is 2+2?"
      },
      "answer": "4"
    }
  ],
  "test_goldens": [
    {
      "fields": {
        "context": "Basic arithmetic",
        "question": "What is 2+2?"
      },
      "answer": "4"
    }
  ],
  "origin_model": {
    "provider": "openai",
    "model": "gpt-4o"
  },
  "target_models": [
    {
      "provider": "openai",
      "model": "gpt-4o"
    }
  ]
}

Response

Successfully started prompt optimization

optimization_run_idstring required

Unique identifier for this optimization run. Use this to poll status and retrieve optimized prompts when complete

Example response

{
  "optimization_run_id": "550e8400-e29b-41d4-a716-446655440000"
}