v2

latestOpenAPI 3.0.02026-08-073221853.1 MB

Experiments

Retrieve Pulse Results (Beta)

get/console/v1/experiments/{id}/pulse_results

Path parameters

idstring required

Query parameters

controlstring required

Control Group ID

teststring required

Test Group ID

cupedstring

Whether to apply CUPED. Allowed values are "true" or "false".

confidencestring

Confidence interval (0-100)

applyBonferroniPerVariantstring

Whether to apply Bonferroni Per Variant. Allowed values are "true" or "false".

applyBonferroniPerMetricstring

Whether to apply Bonferroni Per Metric. Allowed values are "true" or "false".

bonferroniPrimaryMetricWeightstring

α allocated to primary metrics

applyBenjaminiHochbergPerMetricstring

Whether to apply Benjamini-Hochberg Correction Per Metric. Allowed values are "true" or "false".

applyBenjaminiHochbergPerVariantstring

Whether to apply Benjamini-Hochberg Correction Per Variant. Allowed values are "true" or "false".

benjaminiPrimaryMetricsOnlystring

Whether Benjamini-Hochberg correction should apply only to primary metrics. Allowed values are "true" or "false".

datestring

Date for pulse results. format must be YYYY-MM-DD

benjaminiAcrossAllTestGroupsstring

Whether to compute the Benjamini-Hochberg / Bonferroni-per-group correction across all of the experiment's test groups (matching the Console Results tab default for multi-group experiments) rather than only the requested test-vs-control pair. The returned results are always scoped to the requested pair; this controls only the correction family. Allowed values are "true" or "false". Defaults to "true".

Response

Get Pulse Results Success

messagestring required

A simple string explaining the result of the operation.

Example response

{
  "message": "Successfully loaded Pulse Results",
  "data": {
    "primaryMetrics": [
      {
        "metricID": "Purchase::user_warehouse",
        "value": -0.5656598720428505,
        "confidence_interval_delta": 1.3331521511820725,
        "confidence_interval": {
          "lower": -1.8988120232249,
          "upper": 0.7674922791392
        },
        "percent_change": -0.9104559464139017,
        "percent_confidence_interval_delta": 2.145770565507353,
        "percent_confidence_interval": {
          "lower": -3.0562265119213,
          "upper": 1.2353146190935
        },
        "percent_sequential_testing_confidence_interval_delta": -1.6095466399726037,
        "percent_sequential_testing_confidence_interval": {
          "lower": 0.6990906935587,
          "upper": -2.5200025863865
        },
        "test_mean": 61.56363636363636,
        "control_mean": 62.12929623567921,
        "test_std": 0.7461734327711735,
        "control_std": 0.7248273669644147,
        "test_units": 1669,
        "control_units": 1589,
        "p_value": 0.7066986120589813,
        "sequentialTestingPValue": 0.7066986120589813,
        "adjustedAlpha": 0.05
      }
    ],
    "secondaryMetrics": [
      {
        "metricID": "Support Success Rate::user_warehouse",
        "value": 0.0032568848232868985,
        "confidence_interval_delta": 0.009120368109978388,
        "confidence_interval": {
          "lower": -0.00586348328669149,
          "upper": 0.012377252933265286
        },
        "percent_change": 0.4569873400680713,
        "percent_confidence_interval_delta": 1.2797175795779019,
        "percent_confidence_interval": {
          "lower": -0.8227302395098306,
          "upper": 1.736704919646
        },
        "percent_sequential_testing_confidence_interval_delta": -140.31424654645156,
        "percent_sequential_testing_confidence_interval": {
          "lower": 140.77123388651964,
          "upper": -139.8572592063835
        },
        "test_mean": 0.7159428911362284,
        "control_mean": 0.7126860063129415,
        "test_std": 0.0038882253564431193,
        "control_std": 0.003953024266669566,
        "test_units": 4696,
        "control_units": 4690,
        "p_value": 0.5569502167465609,
        "sequentialTestingPValue": 0.5569502167465609,
        "adjustedAlpha": 0.05
      }
    ]
  }
}