Sign in

v3

latestSwagger 2.02026-07-132961114.8 KB
Evals

Create evaluation

Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the Evals guide.

post/openai/evals

Query parameters

api-versionstring required

The API version to use for this operation.

Request body

namestring

The name of the evaluation.

metadataOpenAIMetadata

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

{"stackTrail":"components:schemas:CreateEvalRequest:properties:data_source_config","oasType":"schema","type":"unknown","description":"The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation."}
propertiesobject

Set of immutable 16 key-value pairs that can be attached to an object for storing additional information. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Response

The request has succeeded.

object'eval' required

The object type.

idstring required

Unique identifier for the evaluation.

namestring required

The name of the evaluation.

{"stackTrail":"components:schemas:Eval:properties:data_source_config","oasType":"schema","type":"unknown","description":"Configuration of data sources used in runs of the evaluation."}
created_atinteger required

The Unix timestamp (in seconds) for when the eval was created.

metadataOpenAIMetadata required

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

modified_atinteger
created_bystring

the name of the person who created the run.

propertiesobject

Set of immutable 16 key-value pairs that can be attached to an object for storing additional information. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.