---
title: "POST /v1/chat/completions"
method: POST
path: "/v1/chat/completions"
tags: ["inference"]
---

# POST /v1/chat/completions

`POST /v1/chat/completions`

OpenAI-compatible chat completions. Strict allow-list parser: unknown top-level fields reject with 400 `unsupported_field`. Streaming (`stream: true`) is supported and emits SSE `chat.completion.chunk` events; `n > 1` streaming fans candidates out as per-`choice_index` delta chunks with per-choice closure chunks (each carrying `finish_reason`) before the single global `[DONE]`. Content is either a string or an array of content parts; image parts (`image_url` / `input_image`) with a base64 `data:` URI are accepted for generation models declaring `inputs.image` (non-vision/encode-only models reject 400 `unsupported_field`; remote URLs reject 400 `invalid_request`). `lora_adapter` is forwarded to the worker as a top-level `lora_path` generation kwarg.

## Headers

- `X-SIE-MACHINE-PROFILE` string, nullable
- `X-SIE-Pool` string, nullable
- `X-SIE-SDK-Version` string, nullable

## Request body

- ChatCompletionRequest — OpenAI-compatible ``POST /v1/chat/completions`` request. **Strict allow-list:** unknown top-level fields reject with 400 ``unsupported_field``. Type-invalid values for accepted fields reject with 400 ``invalid_request``. **Known-rejected fields**: - ``functions`` / ``function_call`` — deprecated by OpenAI; use ``tools`` instead. - ``modalities``, ``audio``, ``metadata``, ``store``, ``service_tier``, ``prediction``, ``reasoning_effort``, ``verbosity`` — out of scope. **Streaming:** ``stream: true`` is supported (SSE ``chat.completion.chunk``). ``n > 1`` streaming fans candidates out as per-``choice_index``-tagged delta chunks with a per-choice closure carrying ``finish_reason`` (and a per-choice ``delta.role:"assistant"`` once per choice) before the single global ``[DONE]``.
  - `best_of` integer, nullable — Generate this many candidates and return the top `n` by cumulative logprob. Integer in [1, 128]. Cross-field rule: `best_of >= n` (otherwise 400 invalid_request). Rejected with 400 unsupported_field when `stream: true` (mirrors OpenAI).
  - `chat_template_kwargs` ChatTemplateKwargs — Bounded tokenizer-template options accepted from a chat request. Unknown keys are rejected at the gateway instead of being forwarded to a tokenizer implementation as arbitrary keyword arguments.
    - `enable_thinking` boolean, nullable — Qwen-family switch controlling whether the template opens a thinking block.
    - `guardian_config` GuardianChatTemplateConfig — Bounded Granite Guardian options accepted by ``chat_template_kwargs``.
      - `risk_name` string, required — Non-empty Granite Guardian risk identifier, at most 128 characters.
  - `frequency_penalty` number, float, nullable — OpenAI ``frequency_penalty`` in ``[-2.0, 2.0]``; out-of-range or non-numeric values yield 400 ``invalid_request``.
  - `logit_bias` object, nullable — OpenAI `logit_bias` — `{token_id_string: bias_float}`. Keys must parse as integer token ids; values must be finite numbers in [-100.0, 100.0]. Map size capped at 1024 keys (request rejects with 400 invalid_request beyond the cap).
  - `logprobs` boolean, nullable — Return per-token logprobs. Boolean. When true, the chosen token's logprob (and optionally a top-N list via `top_logprobs`) rides on each `choices[].logprobs` entry.
  - `lora_adapter` string, nullable — SIE extension: non-empty served-name of a LoRA adapter declared in the model profile's `lora_paths`. Absent → the base model. Unknown name → 400 with `param:"lora_adapter"`. The gateway forwards it to the worker as a top-level `lora_path` generation kwarg (SGLang then selects the adapter by served name; the sampling-params placement is an SGLang implementation detail, not part of the SIE wire contract).
  - `max_completion_tokens` integer, nullable — Preferred output-token cap. Falls back to ``max_tokens`` when absent. When BOTH are omitted the gateway applies a default (1024, override via ``SIE_GATEWAY_DEFAULT_MAX_TOKENS``) rather than rejecting — matching OpenAI, where this field is optional.
  - `max_tokens` integer, nullable — Legacy compatibility — ``max_completion_tokens`` wins when both are present. Optional; see ``max_completion_tokens`` for the behaviour when neither is supplied.
  - `messages` ChatCompletionMessage[], required
    - `content` union — Either a string or an array of content parts (`{type:"text"|"input_text", text:"..."}`). Image parts (`image_url` / `input_image`) carrying a base64 `data:` URI are accepted for generation models that declare `inputs.image`; non-vision models reject with 400 unsupported_field and remote (non-`data:`) URLs reject with 400 invalid_request. May be `null` on a `role:"assistant"` message that carries `tool_calls`.
      - string
      - object[]
        - `image_url` union — Image payload for `image_url` / `input_image` parts: a base64 `data:` URI, either as a bare string or as `{ "url": "data:..." }`. Remote (non-`data:`) URLs reject with 400 invalid_request.
          - string
          - object
            - `url` string
        - `text` string
        - `type` 'text' | 'input_text' | 'image_url' | 'input_image', required
    - `role` string, required — One of ``"system" | "user" | "assistant" | "tool" | "developer"``. ``tool`` carries multi-turn tool-call replay; ``developer`` is OpenAI's newer alias for ``system`` and is normalized to ``system``. Any other role rejects with 400 ``invalid_request``.
    - `tool_call_id` string, nullable — Required on `role:"tool"` messages (matches the assistant turn's `tool_calls[].id`). Rejected on every other role with 400 invalid_request.
    - `tool_calls` object[], nullable — OpenAI tool-call replay on `role:"assistant"` messages. Each entry MUST match `{id, type:"function", function:{name, arguments}}`; `arguments` is a JSON-encoded string. Rejected on other roles with 400.
      - `function` object, required
        - `arguments` string, required — JSON-encoded argument string (OpenAI convention).
        - `name` string, required
      - `id` string, required
      - `type` 'function', required
  - `min_tokens` integer, nullable — SGLang ``sampling_params.min_new_tokens``: integer ``>= 0`` capping how many tokens the model must emit before any stop condition can fire. Use to work around models that occasionally emit the stop token as the very first decoded token (e.g. Qwen3.6 thinking-off under greedy decode). Absent → sampler default (no minimum).
  - `model` string, required
  - `n` integer, nullable — Number of candidate completions in [1, 128]. n>1 returns a multi-entry choices array; streaming with n>1 is supported (per-choice_index delta chunks + per-choice closure chunks ride before the single global [DONE]).
  - `parallel_tool_calls` boolean, nullable — OpenAI `parallel_tool_calls` — boolean controlling whether the model may emit multiple tool calls per turn.
  - `presence_penalty` number, float, nullable — OpenAI ``presence_penalty`` in ``[-2.0, 2.0]``; same validation as ``frequency_penalty``.
  - `prompt_cache_key` string, nullable — Prompt-cache hint; plumbed onto the work envelope and ignored by the worker on the chat-completions surface.
  - `repetition_penalty` number, float, nullable — Non-OpenAI `repetition_penalty`: float in (0.0, 2.0] (1.0 = no penalty). Absent → sampler default.
  - `response_format` object, nullable — OpenAI `response_format` — translated into a grammar spec on the worker. Accepted shapes: `{type:"text"}`, `{type:"json_object"}`, `{type:"json_schema", json_schema:{...}}`. Cannot be combined with a forcing `tool_choice` (`"required"` or a named function) — two competing grammars on one request reject with 400 invalid_request.
    - `json_schema` object
    - `type` 'text' | 'json_object' | 'json_schema'
  - `routing_key` string, nullable — Routing affinity hint; same plumbing as ``prompt_cache_key``.
  - `safety_identifier` string, nullable — Accepted and silently ignored (never logged, never forwarded).
  - `seed` integer, nullable — Optional signed 64-bit per-request sampling seed. Reproducibility is best effort, not guaranteed, and depends on the active generation backend and deployment configuration. Non-integer or out-of-range values reject with 400 invalid_request.
  - `stop` union — Either a string or an array of strings, mirroring OpenAI.
    - string
    - string[]
  - `stream` boolean, nullable — SSE streaming. When true, the response is a stream of `chat.completion.chunk` events terminated by `data: [DONE]`. For n > 1: per-`choice_index` delta chunks include a per-choice `delta.role:"assistant"` once per choice; per-choice closure chunks carry the `finish_reason` for that choice before the global `[DONE]`. Non-boolean values reject with 400 invalid_request.
  - `stream_options` object, nullable — OpenAI `stream_options`. Accepted sub-key: `include_usage` (boolean — when true, the gateway emits a terminal `usage` frame before `[DONE]`). Any other sub-key rejects with 400 unsupported_field. Legal with `stream:false` (options ignored).
    - `include_usage` boolean, nullable — Emit a terminal `usage` frame before `[DONE]`.
  - `temperature` number, float, nullable — Sampling temperature. Finite number ``>= 0``; non-finite values reject.
  - `tool_choice` union — OpenAI `tool_choice`. One of: `"auto"`, `"none"`, `"required"`, or `{type:"function", function:{name}}` (named function). Requires `tools` to be set (otherwise 400 invalid_request). `"required"` and named-function choices cannot be combined with `response_format` (two competing grammars; 400 invalid_request).
    - 'auto' | 'none' | 'required'
    - object
      - `function` object, required
        - `name` string, required
      - `type` 'function', required
  - `tools` object[], nullable — OpenAI tool-calling. Array of tool specs; each tool must match `{type:"function", function:{name, parameters, description?}}`. With n > 1, per-candidate `tool_calls` surface on `choices[i].message.tool_calls` (non-streaming) or ride on each `choices[].delta` (streaming).
    - `function` object, required
      - `description` string
      - `name` string, required
      - `parameters` object
    - `type` 'function', required
  - `top_k` integer, nullable — Non-OpenAI `top_k` (Together / Fireworks / vLLM extension): integer >= 1. Absent → top-k disabled (model default).
  - `top_logprobs` integer, nullable — Number of alternative top tokens to return alongside each chosen token's logprob. Integer in [0, 20]. Requires `logprobs: true` when > 0 (OpenAI rule; 400 invalid_request otherwise).
  - `top_p` number, float, nullable — Nucleus sampling. Finite number in ``(0, 1]``.
  - `user` string, nullable — OpenAI `user` — Sensitive PII. Accepted-and-dropped: debug-logged only, never persisted, never forwarded to the worker.

## Response `200`

Chat completion response

- ChatCompletionResponse
  - `choices` ChatCompletionChoice[], required
    - `finish_reason` string, required — One of ``"stop" | "length" | "tool_calls"``. The chat surface collapses unknown SIE-native finish reasons to ``stop`` so strict OpenAI clients still parse the response. ``_close_choice`` coerces a length-truncated candidate that also produced a tool call to ``"tool_calls"`` per the OpenAI convention.
    - `index` integer, required
    - `message` ChatCompletionChoiceMessage, required
      - `content` string, required
      - `role` string, required
      - `tool_calls` unknown
  - `created` integer, required — Epoch seconds.
  - `id` string, required — Always ``"chatcmpl-<request_id>"``.
  - `model` string, required
  - `object` string, required — Always ``"chat.completion"`` on the non-streaming endpoint.
  - `usage` ChatCompletionUsage, required
    - `completion_tokens` integer, required
    - `prompt_tokens` integer, required
    - `total_tokens` integer, required

## Other responses

- `400` — Invalid or unsupported request
- `401` — Missing or invalid bearer token (inference token)
- `404` — Model not found
- `413` — Request body is too large
- `500` — Worker emitted malformed response; gateway auth enabled but no tokens configured
- `503` — Provisioning in progress, queue unavailable, or model loading

---

[API](https://skmtc.net/superlinked/apis/sie-gateway.md) · [All operations](https://skmtc.net/superlinked/apis/sie-gateway/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/superlinked/sie-gateway/revisions/840fee92bd27/schema)