--- title: "Get eval runs" method: GET path: "/evals/{eval_id}/runs" tags: ["Evals"] --- # Get eval runs `GET /evals/{eval_id}/runs` Get a list of runs for an evaluation. ## Path parameters - `eval_id` string, required ## Query parameters - `after` string - `limit` integer - `order` 'asc' | 'desc' - `status` 'queued' | 'in_progress' | 'completed' | 'canceled' | 'failed' ## Response `200` A list of runs for the evaluation - EvalRunList — An object representing a list of runs for an evaluation. - `object` 'list', required — The type of this object. It is always set to "list". - `data` EvalRun[], required — An array of eval run objects. - `object` 'eval.run', required — The type of the object. Always "eval.run". - `id` string, required — Unique identifier for the evaluation run. - `eval_id` string, required — The identifier of the associated evaluation. - `status` string, required — The status of the evaluation run. - `model` string, required — The model that is evaluated, if applicable. - `name` string, required — The name of the evaluation run. - `created_at` integer, required — Unix timestamp (in seconds) when the evaluation run was created. - `report_url` string, uri, required — The URL to the rendered evaluation run report on the UI dashboard. - `result_counts` object, required — Counters summarizing the outcomes of the evaluation run. - `total` integer, required — Total number of executed output items. - `errored` integer, required — Number of output items that resulted in an error. - `failed` integer, required — Number of output items that failed to pass the evaluation. - `passed` integer, required — Number of output items that passed the evaluation. - `per_model_usage` object[], required — Usage statistics for each model during the evaluation run. - `model_name` string, required — The name of the model. - `invocation_count` integer, required — The number of invocations. - `prompt_tokens` integer, required — The number of prompt tokens used. - `completion_tokens` integer, required — The number of completion tokens generated. - `total_tokens` integer, required — The total number of tokens used. - `cached_tokens` integer, required — The number of tokens retrieved from cache. - `per_testing_criteria_results` object[], required — Results per testing criteria applied during the evaluation run. - `testing_criteria` string, required — A description of the testing criteria. - `passed` integer, required — Number of tests passed for this criteria. - `failed` integer, required — Number of tests failed for this criteria. - `data_source` union, required — Information about the run's data source. - object — A JsonlRunDataSource object with that specifies a JSONL file that matches the eval - `type` 'jsonl', required — The type of data source. Always `jsonl`. - `source` union, required — Determines what populates the `item` namespace in the data source. - EvalJsonlFileContentSource - `type` 'file_content', required — The type of jsonl source. Always `file_content`. - `content` object[], required — The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required — The type of jsonl source. Always `file_id`. - `id` string, required — The identifier of the file. - object — A CompletionsRunDataSource object describing a model sampling configuration. - `type` 'completions', required — The type of run data source. Always `completions`. - `input_messages` union — Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required — The type of input messages. Always `template`. - `template` union[], required — A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - … - object - `type` 'item_reference', required — The type of input messages. Always `item_reference`. - `item_reference` string, required — A reference to a variable in the `item` namespace. Ie, "item.input_trajectory" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable — Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://platform.openai.com/docs/guides/reasoning) for model-specific support. - `temperature` number — A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer — The maximum number of tokens in the generated output. - `top_p` number — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer — A seed value to initialize the randomness, during sampling. - `response_format` union — An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - ResponseFormatText — Default response format. Used to generate text responses. - … - ResponseFormatJsonSchema — JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs). - … - ResponseFormatJsonObject — JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so. - … - `tools` ChatCompletionTool[] — A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported. - `type` 'function', required — The type of the tool. Currently, only `function` is supported. - `function` FunctionObject, required - … - `model` string — The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required — Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required — The type of jsonl source. Always `file_content`. - `content` object[], required — The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required — The type of jsonl source. Always `file_id`. - `id` string, required — The identifier of the file. - EvalStoredCompletionsSource — A StoredCompletionsRunDataSource configuration describing a set of filters - `type` 'stored_completions', required — The type of source. Always `stored_completions`. - `metadata` Metadata, nullable — Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `model` string, nullable — An optional model to filter by (e.g., 'gpt-4o'). - `created_after` integer, nullable — An optional Unix timestamp to filter items created after this time. - `created_before` integer, nullable — An optional Unix timestamp to filter items created before this time. - `limit` integer, nullable — An optional maximum number of items to return. - object — A ResponsesRunDataSource object describing a model sampling configuration. - `type` 'responses', required — The type of run data source. Always `responses`. - `input_messages` union — Used when sampling from a model. Dictates the structure of the messages passed into the model. Can either be a reference to a prebuilt trajectory (ie, `item.input_trajectory`), or a template with variable references to the `item` namespace. - object - `type` 'template', required — The type of input messages. Always `template`. - `template` union[], required — A list of chat messages forming the prompt or context. May include variable references to the `item` namespace, ie {{item.name}}. - … - object - `type` 'item_reference', required — The type of input messages. Always `item_reference`. - `item_reference` string, required — A reference to a variable in the `item` namespace. Ie, "item.name" - `sampling_params` object - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable — Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://platform.openai.com/docs/guides/reasoning) for model-specific support. - `temperature` number — A higher temperature increases randomness in the outputs. - `max_completion_tokens` integer — The maximum number of tokens in the generated output. - `top_p` number — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `seed` integer — A seed value to initialize the randomness, during sampling. - `tools` Tool[] — An array of tools the model may call while generating a response. You can specify which tool to use by setting the `tool_choice` parameter. The two categories of tools you can provide the model are: - **Built-in tools**: Tools that are provided by OpenAI that extend the model's capabilities, like [web search](https://platform.openai.com/docs/guides/tools-web-search) or [file search](https://platform.openai.com/docs/guides/tools-file-search). Learn more about [built-in tools](https://platform.openai.com/docs/guides/tools). - **Function calls (custom tools)**: Functions that are defined by you, enabling the model to call your own code. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling). - union — A tool that can be used to generate a response. - … - `text` object — Configuration options for a text response from the model. Can be plain text or structured JSON data. Learn more: - [Text inputs and outputs](https://platform.openai.com/docs/guides/text) - [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - `format` union — An object specifying the format that the model must output. Configuring `{ "type": "json_schema" }` enables Structured Outputs, which ensures the model will match your supplied JSON schema. Learn more in the [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs). The default format is `{ "type": "text" }` with no additional options. **Not recommended for gpt-4o and newer models:** Setting to `{ "type": "json_object" }` enables the older JSON mode, which ensures the message the model generates is valid JSON. Using `json_schema` is preferred for models that support it. - … - `model` string — The name of the model to use for generating completions (e.g. "o3-mini"). - `source` union, required — Determines what populates the `item` namespace in this run's data source. - EvalJsonlFileContentSource - `type` 'file_content', required — The type of jsonl source. Always `file_content`. - `content` object[], required — The content of the jsonl file. - … - EvalJsonlFileIdSource - `type` 'file_id', required — The type of jsonl source. Always `file_id`. - `id` string, required — The identifier of the file. - EvalResponsesSource — A EvalResponsesSource object describing a run data source configuration. - `type` 'responses', required — The type of run data source. Always `responses`. - `metadata` object, nullable — Metadata filter for the responses. This is a query parameter used to select responses. - `model` string, nullable — The name of the model to find responses for. This is a query parameter used to select responses. - `instructions_search` string, nullable — Optional string to search the 'instructions' field. This is a query parameter used to select responses. - `created_after` integer, nullable — Only include items created after this timestamp (inclusive). This is a query parameter used to select responses. - `created_before` integer, nullable — Only include items created before this timestamp (inclusive). This is a query parameter used to select responses. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max', nullable — Constrains effort on reasoning for reasoning models. Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. Not all reasoning models support every value. See the [reasoning guide](https://platform.openai.com/docs/guides/reasoning) for model-specific support. - `temperature` number, nullable — Sampling temperature. This is a query parameter used to select responses. - `top_p` number, nullable — Nucleus sampling parameter. This is a query parameter used to select responses. - `users` string[], nullable — List of user identifiers. This is a query parameter used to select responses. - `tools` string[], nullable — List of tool names. This is a query parameter used to select responses. - `metadata` Metadata, nullable, required — Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. - `error` EvalApiError, required — An object representing an error response from the Eval API. - `code` string, required — The error code. - `message` string, required — The error message. - `first_id` string, required — The identifier of the first eval run in the data array. - `last_id` string, required — The identifier of the last eval run in the data array. - `has_more` boolean, required — Indicates whether there are more evals available. --- [API](https://skmtc.net/openai/apis/openai-api.md) · [All operations](https://skmtc.net/openai/apis/openai-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/openai/openai-api/revisions/dcd35773a322/schema)