Sign in

v1

latestOpenAPI 3.1.02026-07-22152755.8 KB
test_framework

Retrieve multiple runs in a single request.

success: true indicates the run passed its quality evaluation. A run with success=false either failed to connect (check error_message) or failed an evaluation check. For detailed outcomes check: error_message, metadata.ended_reason, and evaluation.metrics[].result.

get/test_framework/v2/runs/bulk/

Query parameters

run_idsstring required

Comma-separated list of run IDs. Example: 1,2,3

Must be a plain comma-separated string, not a JSON array — passing a JSON array returns a 400 error.

agent_idinteger

Tenant scope. Supply one of agent_id, result_id, project_id when authenticating with a user session or OAuth bearer token; omit when using an API key already scoped to the tenant.

result_idinteger

Tenant scope. Supply one of agent_id, result_id, project_id when authenticating with a user session or OAuth bearer token; omit when using an API key already scoped to the tenant.

project_idinteger

Tenant scope. Supply one of agent_id, result_id, project_id when authenticating with a user session or OAuth bearer token; omit when using an API key already scoped to the tenant.

Response

idinteger
provider_call_idstring

Call ID from the Assistant provider Example: call_abc123

trace_idstring

OpenTelemetry trace ID (32-char hex). Set when OTel tracing is enabled.

scenariointeger nullable
scenario_namestring
scenario_typestring

Type of the run's scenario (e.g. instruction, conditional_actions), null if the scenario was deleted

personalityinteger nullable

Personality this run belongs to

personality_namestring

Name of the personality this run belongs to

agentinteger

ID of the agent associated with this run

agent_namestring

Name of the agent associated with this run

{"stackTrail":"components:schemas:ExpectedOutcome","oasType":"schema","type":"unknown","description":"Referenced by this API but not published in the vendor's docs pages."}
result_idinteger
result_namestring

Name of the result this run belongs to

expected_outcome_promptstring
expected_outcomestring
{"stackTrail":"components:schemas:Evaluation","oasType":"schema","type":"unknown","description":"Referenced by this API but not published in the vendor's docs pages."}
{"stackTrail":"components:schemas:Run:properties:rubric:oneOf","oasType":"schema","type":"unknown","description":"\nDetailed rubric evaluation data\nExample:\n","nullable":true}