---
title: "Create a batch inference job"
method: POST
path: "/v1/batches"
tags: ["Batches"]
---

# Create a batch inference job

`POST /v1/batches`

Submit a batch from an uploaded input file (input_file_id) or inline request objects (requests); provide exactly one of the two. The batch completes within the 24h window at a discounted rate; poll it with get, then download results via the output and error file ids.

## Request body

- object
  - `completion_window` '24h', required — Time budget for the batch; only "24h" is supported.
  - `endpoint` '/v1/chat/completions' | '/v1/completions' | '/v1/embeddings', required — The API family every record in the batch calls.
  - `input_file_id` string — Id of an uploaded JSONL input file (from the file upload).
  - `metadata` object — Up to 16 key-value pairs echoed back on the batch object.
  - `model` string — Optional display hint shown on the batch immediately; validation settles the authoritative value from the input file.
  - `requests` BatchRequestRecord[] — Inline request objects (max 50,000), each with a custom_id and the request body for the chosen endpoint. Alternative to input_file_id for small batches.
    - `body` object — The request body you would send to that endpoint online, for example a {model, messages} object for /v1/chat/completions. Streaming is not supported, so "stream": true fails that record. A chat record may also carry OpenRelay request extensions under the reserved "openrelay" key. A bare top-level tool_config is the retired spelling of that extension and fails the record with code tool_config_moved.
      - `openrelay` BatchOpenRelayExtensions — The reserved "openrelay" object on a batch record body: the one top-level key OpenRelay request extensions live under, mirroring the "openrelay" response namespace that carries tool_rounds. Additive, so new keys may appear without a breaking change. The worker strips the keys it owns from the body before every model invocation.
        - `tool_config` BatchToolConfig — The OpenRelay tool-calling extension, carried at body.openrelay.tool_config on a /v1/chat/completions record: where to execute the model's tool calls between turns. When a turn ends with finish_reason "tool_calls", the batch worker POSTs the calls to this endpoint, appends the assistant and tool messages, and re-invokes the model, up to max_rounds times. The field is stripped from the body before every model invocation and never appears in result files. A malformed value fails that one record with code invalid_tool_config.
          - `authorization` string — Sent verbatim as the Authorization header on every executor call.
          - `context` object — Opaque JSON forwarded to the executor on every call.
          - `max_rounds` integer — Maximum tool round-trips for this record. 0 or omitted takes the default of 8; values above 16 are clamped to 16. A record can therefore cost up to max_rounds + 1 model invocations.
          - `timeout_ms` integer — Per-executor-call timeout in milliseconds. 0 or omitted takes the default of 30000; values above 120000 are clamped to 120000.
          - `type` 'http' | 'mcp' — Executor protocol. "http" is the plain POST contract ({custom_id, context, tool_calls} in, {results:[{tool_call_id, content}]} out); "mcp" is a Model Context Protocol server speaking Streamable HTTP. Defaults to "http".
          - `url` string, uri, required — Your tool-executor endpoint. HTTPS only, because the request carries your authorization value. Endpoints resolving to internal or private addresses are rejected.
    - `custom_id` string — Your identifier for this request, echoed on the matching result line so you can join results back to inputs. Unique within the batch.
    - `method` 'POST' — HTTP method for the record; only POST is supported.
    - `url` string — The endpoint this record targets; must match the batch's endpoint.

## Response `200`

OK

- BatchObject — A batch inference job (OpenAI-compatible).
  - `cancelled_at` integer
  - `completed_at` integer
  - `completion_window` '24h', required
  - `created_at` integer, required — Unix timestamp (seconds).
  - `endpoint` string, required — The API family every record in the batch calls.
  - `error_file_id` string — Present once results are written; fetch its content for failed records.
  - `expired_at` integer
  - `expires_at` integer — When the completion window closes (created_at + 24h).
  - `failed_at` integer
  - `id` string, required — Batch id (batch_…).
  - `input_file_id` string, required
  - `metadata` object — Your key-value pairs, echoed back unchanged.
  - `model` string — Model id, settled from the input file during validation.
  - `object` 'batch', required
  - `output_file_id` string — Present once results are written; fetch its content for successful records.
  - `request_counts` BatchRequestCounts, required — Progress counters, settled as the batch validates and shards complete.
    - `completed` integer, required
    - `failed` integer, required
    - `total` integer, required
  - `status` 'validating' | 'in_progress' | 'finalizing' | 'completed' | 'failed' | 'expired' | 'cancelling' | 'cancelled', required — Lifecycle state. validating → in_progress → finalizing → completed | failed | expired; cancelling → cancelled.
  - `usage` BatchUsage — Rolled-up token and cost totals, present once any progress is recorded. cost_nano_usd is the price you pay (batch discount applied), in nano-USD so sub-cent batches stay exact.
    - `cost_nano_usd` integer, required
    - `input_tokens` integer, required
    - `output_tokens` integer, required

## Other responses

- `400` — The request is invalid
- `401` — Missing or invalid API key
- `404` — Not found (also returned when the organization does not have Batch API access)
- `413` — The upload exceeds the 200 MB limit
- `429` — Too many concurrent uploads; retry shortly
- `503` — The action could not be dispatched; retry

---

[API](https://skmtc.net/openrelay/apis/openrelay-api.md) · [All operations](https://skmtc.net/openrelay/apis/openrelay-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/openrelay/openrelay-api/versions/32905b8f44f8/schema)