---
title: "Compare Experiment Configurations"
method: GET
path: "/internal/experiment-reports/compare"
tags: ["experiment-reports"]
---

# Compare Experiment Configurations

`GET /internal/experiment-reports/compare`

Paired statistics for two chosen configurations over their shared scenarios.

## Query parameters

- `baseline_config_key` string, required — Configuration to treat as the reference arm.
- `treated_config_key` string, required — Configuration to treat as the treatment arm.
- `days` integer, nullable — Window: runs from the last N days.

## Response `200`

Successful Response

- ComparisonResponse — Paired statistics for two reader-chosen configurations. Serves an A/B (configs differing in code or runner) and an A/A noise floor (two arms of one config set) identically — the difference is only which two configurations the reader picked.
  - `baseline_config_key` string, required
  - `treated_config_key` string, required
  - `baseline_label` string, required
  - `treated_label` string, required
  - `shared_scenario_count` integer, required
  - `baseline_run_count` integer, required
  - `treated_run_count` integer, required
  - `model` string, nullable
  - `metrics` MetricRow[], required
    - `key` 'strict_success' | 'completion' | 'wall_time_seconds' | 'input_tokens' | 'output_tokens' | 'total_tokens' | 'cost_usd', required
    - `label` string, required
    - `unit` 'fraction' | 'seconds' | 'count' | 'usd', required
    - `lower_is_better` boolean, required
    - `baseline` number, required
    - `treated` number, required
    - `difference` number, required
    - `ci_low` number, required
    - `ci_high` number, required
    - `mde` number, required
    - `effect_over_mde` number, required
    - `signal` 'significant' | 'directional' | 'under_mde' | 'inconclusive', required
    - `mass_improvement` number, required
  - `generated_at` string, required
  - `warnings` string[], required

## Other responses

- `422` — Validation Error

---

[API](https://skmtc.net/tryscott/apis/one-shot-api.md) · [All operations](https://skmtc.net/tryscott/apis/one-shot-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/tryscott/one-shot-api/versions/28bdface01df/schema)