Sign in

v3

OpenAPI 3.1.02026-07-31350397683.8 KB
Custom Metrics

Create Custom Metric

Create a new custom metric.

post/v1/create-custom-metric

Headers

X-API-Keystring required

API key required to authenticate requests.

Request body

bluejay_as_code_idstring uuid nullable

Stable code-addressable identifier

agent_idinteger nullable

ID of the agent this metric belongs to (deprecated, use agent_ids instead)

agent_idsinteger[] nullable

List of agent IDs to associate this metric with

namestring required

Name of the custom metric

promptstring nullable

LLM-judge system prompt (required for llm_judge; null for other types)

summarystring nullable

Short plain-English description of the metric (LLM-generated)

metric_typestring

Specific metric kind (llm_judge, tool_call, audio_*, vocal_fry); legacy alias: type

settingsobject

Type-specific config (e.g. tool_call: {tool_names: [...]})

eval_segment'agent' | 'user' | 'all'

Which side of the conversation is evaluated

scope'observability' | 'simulations' | 'all'

Which pipeline runs this metric

response_type'pass_fail' | 'yes_no' | 'qualitative' | 'quantitative' | 'json' | 'enum' | 'int' | 'float' | 'boolean' | 'tool_call' required

Enum representing the possible response types for custom metrics.

min_valuenumber nullable

Minimum value for quantitative metrics

max_valuenumber nullable

Maximum value for quantitative metrics

categorystring nullable

Category for organizing metrics

tagsstring[] nullable

Tags for categorizing the metric

scoring_guidancestring nullable

Guidance on how to score this metric

eval_modality'AUDIO' | 'TEXT' | 'AUTO'

Enum representing the evaluation route of a call.

modelstring nullable

Model to use for text evaluation of the metric

audio_modelstring nullable

Model to use for audio evaluation of the metric

temperaturenumber nullable

Temperature for evaluating the metric

enum_optionsstring[] nullable

Options for enum metrics

tool_namesstring[] nullable

Expected tool names for tool_call metrics

json_schemaobject nullable

JSON Schema describing the object the judge must return (required for json metrics)

allow_not_applicableboolean

Whether this metric allows 'not applicable' responses.

template_idstring uuid nullable

Library template this metric was created from

Response

Successful Response

idstring required

Unique identifier for the custom metric

bluejay_as_code_idstring nullable

Stable code-addressable identifier (same as id for custom metrics)

namestring required

Name of the custom metric

promptstring nullable

LLM-judge system prompt for this metric (null for non-judge types)

descriptionstring nullable

Deprecated: same value as prompt

summarystring nullable

Short plain-English description of the metric (LLM-generated)

metric_typestring

Specific metric kind

eval_methodstring

Evaluation category (llm_judge|statistical|ml_model|deterministic)

settingsobject

Type-specific config

eval_segmentstring

Which side of the conversation is evaluated (agent|user|all)

scopestring

Which pipeline runs this metric (observability|simulations|all)

response_type'pass_fail' | 'yes_no' | 'qualitative' | 'quantitative' | 'json' | 'enum' | 'int' | 'float' | 'boolean' | 'tool_call' required

Enum representing the possible response types for custom metrics.

agent_idsinteger[]

List of agent IDs this metric belongs to

min_valuenumber nullable

Minimum value for quantitative metrics

max_valuenumber nullable

Maximum value for quantitative metrics

categorystring nullable

Category for organizing metrics

tagsstring[] nullable

Tags for categorizing the metric

scoring_guidancestring nullable

Guidance on how to score this metric

created_atstring date-time required

When this metric was created

updated_atstring date-time nullable

When this metric was last updated

updated_bystring nullable

User who last updated this metric

created_bystring nullable

User who created this metric

eval_modality'AUDIO' | 'TEXT' | 'AUTO'

Enum representing the evaluation route of a call.

modelstring nullable

Model to use for text evaluation of the metric

audio_modelstring nullable

Model to use for audio evaluation of the metric

temperaturenumber nullable

Temperature for evaluating the metric

enum_optionsstring[] nullable

Options for enum metrics

tool_namesstring[] nullable

Deprecated: mirror of settings.tool_names for tool_call metrics

json_schemaobject nullable

JSON Schema describing the object the judge must return (json metrics only)

allow_not_applicableboolean

Whether this metric allows 'not applicable' responses.

eval_modality_auto_selectedboolean

Deprecated: always false (column removed).

template_idstring nullable

Library template this metric was created from