---
title: "Gets metrics for a Loops trainer deployment"
method: POST
path: "/v1/loops/deployments/{deployment_id}/metrics"
---

# Gets metrics for a Loops trainer deployment

`POST /v1/loops/deployments/{deployment_id}/metrics`

Returns per-node GPU/CPU/memory utilization and Knative queue-proxy request rate / concurrency / latency for the trainer pods. The sampler half of a Loops deployment is an OracleVersion and uses the existing model-metrics endpoint.

## Request body

- GetLoopsDeploymentMetricsRequestV1 — Time-range request for trainer deployment metrics.
  - `end_epoch_millis` integer, nullable — Epoch millis to end fetching metrics.
  - `start_epoch_millis` integer, nullable — Epoch millis to start fetching metrics.
  - `step_seconds` integer, nullable — Resolution of the returned series, in seconds. When omitted, a step is derived from the time range so large windows return fewer points.
  - `time_divisor_seconds` integer, nullable — Unit of time for request-volume metrics, in seconds (e.g. 60 for requests/minute). Defaults to per-second.

## Response `200`

- GetLoopsDeploymentMetricsResponseV1 — Response for ``POST /v1/loops/deployments/<id>/metrics``.
  - `deployment_id` string, required — The trainer deployment ID.
  - `metrics` LoopsDeploymentMetricsV1, required — Metrics for a trainer (Loops) deployment. Service-level fields summarize HTTP traffic into the trainer pods (the Knative queue-proxy is the source). Compute fields are the leader-pod aggregate; ``per_node_metrics`` carries the full multinode breakdown.
    - `inference_volume` TrainingJobMetricV1[], required — Number of inference requests per unit time (requests per second).
      - `value` number, required — The value of the metric.
      - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
    - `concurrent_requests` TrainingJobMetricV1[], required — Number of in-progress concurrent inference requests. Source: the queue-proxy ``revision_queue_depth`` gauge on ``http-usermetric``.
      - `value` number, required — The value of the metric.
      - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
    - `response_time_stats` ResponseTimeDatapointV1[], required — Percentiles of the response time distribution.
      - `timestamp` string, date-time, required — ISO 8601 timestamp.
      - `p50` number, nullable — 50th percentile request latency (milliseconds).
      - `p95` number, nullable — 95th percentile request latency (milliseconds).
      - `p99` number, nullable — 99th percentile request latency (milliseconds).
    - `inference_volume_by_status` InferenceVolumeByStatusDatapointV1[], required — Request rate split by response code class.
      - `timestamp` string, date-time, required — ISO 8601 timestamp.
      - `status_2xx` number, required — 2xx requests per second.
      - `status_4xx` number, required — 4xx requests per second.
      - `status_5xx` number, required — 5xx requests per second.
    - `gpu_memory_usage_bytes` object, required — Leader-pod GPU memory bytes per GPU rank.
    - `gpu_utilization` object, required — Leader-pod fractional GPU utilization per GPU rank.
    - `cpu_usage` TrainingJobMetricV1[], required — Leader-pod CPU usage in cores.
      - `value` number, required — The value of the metric.
      - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
    - `cpu_memory_usage_bytes` TrainingJobMetricV1[], required — Leader-pod CPU memory usage bytes.
      - `value` number, required — The value of the metric.
      - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
    - `ephemeral_storage` StorageMetricsV1, required — A metric for a training job.
      - `usage_bytes` TrainingJobMetricV1[], required — The number of bytes used on the storage entity.
        - `value` number, required — The value of the metric.
        - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
      - `utilization` TrainingJobMetricV1[], required — The utilization of the storage entity as a decimal percentage.
        - `value` number, required — The value of the metric.
        - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
    - `per_node_metrics` LoopsDeploymentNodeMetricsV1[], required — Per-node compute breakdown for multinode trainer deployments.
      - `node_id` string, required — Identifier for the node.
      - `gpu_memory_usage_bytes` object, required — GPU memory usage bytes per GPU rank.
      - `gpu_utilization` object, required — Fractional GPU utilization per GPU rank.
      - `cpu_usage` TrainingJobMetricV1[], required — CPU usage in cores.
        - `value` number, required — The value of the metric.
        - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
      - `cpu_memory_usage_bytes` TrainingJobMetricV1[], required — CPU memory usage bytes.
        - `value` number, required — The value of the metric.
        - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
      - `ephemeral_storage` StorageMetricsV1, required — A metric for a training job.
        - `usage_bytes` TrainingJobMetricV1[], required — The number of bytes used on the storage entity.
          - `value` number, required — The value of the metric.
          - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.
        - `utilization` TrainingJobMetricV1[], required — The utilization of the storage entity as a decimal percentage.
          - `value` number, required — The value of the metric.
          - `timestamp` string, date-time, required — The timestamp of the metric in ISO 8601 format.

---

[API](https://skmtc.net/baseten/apis/baseten-inference-api.md) · [All operations](https://skmtc.net/baseten/apis/baseten-inference-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/baseten/baseten-inference-api/versions/a55a2f64c3c5/schema)