Explain Retriever Execution Plan

Get a detailed execution plan for a retriever without actually executing it. Similar to MongoDB's explain plan or SQL's EXPLAIN command, this endpoint helps you understand performance characteristics, identify bottlenecks, estimate costs, and troubleshoot retrieval issues before running expensive queries.

What This Returns:

• Stage-by-stage execution plan (AFTER automatic optimizations)
• Estimated costs (credits + time per stage)
• Document flow projections (input/output counts per stage)
• Efficiency metrics (selectivity ratios, cache likelihood)
• Bottleneck identification (slowest/most expensive stages)
• Optimization details (transformations applied by the optimizer)
• Performance warnings and improvement suggestions

Key Features:

• Cost Estimation: See how many credits and milliseconds each stage will consume
• Bottleneck Detection: Identify which stages dominate execution time
• Optimization Transparency: Understand how your pipeline was optimized
• Cache Analysis: See which stages are likely to hit cache
• Accuracy Troubleshooting: Analyze stage efficiency and document flow
• Latency Analysis: Break down estimated duration by stage

Important: The execution_plan shows OPTIMIZED stages (after automatic transformations like filter push-down, stage fusion, and grouping optimization). Check optimization_details to understand what changed from your original configuration.

Use Cases:

• Debug slow retrievers by identifying bottleneck stages
• Estimate costs before running expensive queries
• Understand how the optimizer transformed your pipeline
• Troubleshoot accuracy issues by analyzing stage selectivity
• Compare different retriever configurations
• Plan budget allocation for production workloads

Example Response:

{
  "retriever_id": "ret_abc123",
  "retriever_name": "product_search",
  "execution_plan": [
    {
      "stage_index": 0,
      "stage_name": "attribute_filter",
      "stage_type": "filter",
      "estimated_input": 10000,
      "estimated_output": 5000,
      "estimated_efficiency": 0.5,
      "estimated_cost_credits": 0.01,
      "estimated_duration_ms": 20,
      "cache_likely": true,
      "optimization_notes": ["Pushed down from stage 2"],
      "warnings": []
    },
    {
      "stage_index": 1,
      "stage_name": "semantic_search",
      "stage_type": "filter",
      "estimated_input": 5000,
      "estimated_output": 100,
      "estimated_efficiency": 0.02,
      "estimated_cost_credits": 0.5,
      "estimated_duration_ms": 200,
      "cache_likely": false,
      "optimization_notes": [],
      "warnings": ["High cost stage - consider reducing limit"]
    }
  ],
  "estimated_cost": {
    "total_credits": 0.51,
    "total_duration_ms": 220
  },
  "bottleneck_stages": ["semantic_search"],
  "optimization_applied": true,
  "optimization_details": {
    "original_stage_count": 3,
    "optimized_stage_count": 2,
    "optimization_time_ms": 8.2,
    "stage_reduction_pct": 33.3,
    "decisions": [
      {
        "rule_type": "push_down_filters",
        "applied": true,
        "reason": "Moved attribute_filter before semantic_search to reduce search scope"
      }
    ]
  },
  "optimization_suggestions": [
    {
      "type": "reduce_limit",
      "stage": "semantic_search",
      "message": "Consider reducing limit to improve latency"
    }
  ]
}