Sign in

v36

latestOpenAPI 3.1.0raw.githubusercontent.com2026-08-0187270483.0 KB
Function Accuracy

Score Function Against (input, expected) Pairs

Score a function against a list of (input, expected) pairs.

Submits a batch of (input, expected) pairs, runs the named function over each input, and returns per-pair + aggregate accuracy metrics comparing the function's actual output to the provided expected JSON.

Scoring runs asynchronously. The response carries a scoreRunID; poll GET /v3/eval/score/{scoreRunID} until status is one of completed, error, or cancelled.

matchConfig controls comparator behavior:

  • numericTolerance: relative tolerance for numeric fields (0 = exact)
  • stringMatch: exact (default) or fuzzy (Levenshtein ratio)
  • arrayMatch: by-index (default; only mode in P0)
  • ignorePaths: JSON Pointer paths to skip, supports * wildcards
post/v3/eval/score

Request body

functionNamestring required

Name of the function to score. Must be of type extract, transform, or analyze.

functionVersionNuminteger

Optional version number to score against. P0: only the function's current version is accepted; passing a different version returns 422.

Response

The request has been accepted for processing, but processing has not yet completed.

scoreRunIDstring required

Run identifier. Use with GET /v3/eval/score/{scoreRunID}.

status'pending' | 'initializing' | 'running' | 'completed' | 'error' | 'cancelled' required

Status values for an eval-score run.