Define a custom metric

Define a metric of your own, emitted as runs land. Once it exists it behaves like any platform metric: readable through GET /orgs/{org_id}/metrics, listed in the catalog with origin: custom, and alarmable with no extra setup.

The definition is a span predicate over each completed run's trace. Span paths are {workforce}.{step} and matching is on the leaf: use llm or get_address, not the full path. Predicate fields AND together and all are optional — omitting every one matches every non-root span.

metric_name is forced under the reserved trace. prefix so a definition cannot shadow a platform series. Set label_source to span_name to break the metric out per step, then point an alarm at a single step with dims.label.

Definitions are not retroactive: matching happens as runs arrive, so a new definition only sees traces written after it. Dry-run it against stored traces first with POST /orgs/{org_id}/metrics/definitions/preview. Note also that only sum/count/avg/min/max are available on the result — percentiles are not derivable from metric bins.

Set project_id to scope the definition to one project, which also makes it editable by holders of projects.alarms.manage there. Leaving it out defines an org-wide metric that runs against every project's traces, and needs alarms.manage.

org_idstring required

Request body

has_errorboolean nullable

true = only errored spans, false = only clean ones, omitted = both.

label_sourcestring

none (default) or span_name to break the metric out per step.

metric_namestring required

Destination metric. trace. is prepended when missing.

min_duration_msinteger nullable

Only spans at least this slow. Spans still running never match.

namestring required

Human label for the definition itself, unique within the org.

project_idstring nullable

Scope to one project. Omit to apply across the org.

source'trace_span'

Where a definition draws its observations from.

One variant today. It is in the payload from the start so that adding a second source is an additive change to this enum instead of a new resource alongside /metrics/definitions — the reason the endpoint is named for metrics rather than for traces.

span_namestring nullable

Exact span name — the leaf of the span path (llm, get_address), not the full {workforce}.{step}.

span_name_prefixstring nullable

Span-name prefix, for matching a family of tools (get_).

status_codestring nullable

value_sourcestring

count (default), duration_ms, or usage:<key>.

Response

Definition created

created_atstring date-time required

enabledboolean required

has_errorboolean nullable

idinteger required

label_sourcestring required

none or span_name.

metric_namestring required

min_duration_msinteger nullable

namestring required

project_idinteger nullable

Project scope; omitted when the filter applies to the whole org.

source'trace_span' required