v47

latestOpenAPI 3.0.3Apache 2.0raw.githubusercontent.com2024-10-01163166522.3 KB

Evals

Launch an eval

Launch an evaluation. This is the API-equivalent of the Eval function that is built into the Braintrust SDK. In the Eval API, you provide pointers to a dataset, task function, and scoring functions. The API will then run the evaluation, create an experiment, and return the results along with a link to the experiment. To learn more about evals, see the Evals guide.

post/v1/eval

Request body

project_idstring required

Unique identifier for the project to run the eval in

experiment_namestring

An optional name for the experiment created by this eval. If it conflicts with an existing experiment, it will be suffixed with a unique identifier.

metadataobject

Optional experiment-level metadata to store about the evaluation. You can later use this to slice & dice across experiments.

streamboolean

Whether to stream the results of the eval. If true, the request will return two events: one to indicate the experiment has started, and another upon completion. If false, the request will return the evaluation's summary upon completion.

Response

Eval launch response

project_namestring required

Name of the project that the experiment belongs to

experiment_namestring required

Name of the experiment

project_urlstring uri required

URL to the project's page in the Braintrust app

experiment_urlstring uri required

URL to the experiment's page in the Braintrust app

comparison_experiment_namestring nullable

The experiment which scores are baselined against

scoresobject nullable

Summary of the experiment's scores

metricsobject nullable

Summary of the experiment's metrics