v36

latestOpenAPI 3.1.0raw.githubusercontent.com2026-08-03124782.6 KB

generate

Generate

Generate text from a prompt using the named model.

The model path segment uses the SIE-safe id (double-underscore separator, e.g. Qwen__Qwen3-4B-Instruct). HuggingFace-style slashes are rejected with 400 to keep parity with the gateway contract.

post/v1/generate/{model}

Path parameters

modelstring required

Headers

X-SIE-MACHINE-PROFILEstring nullable

Request body

promptstring required

Prompt text to generate from

max_new_tokensinteger required

Maximum number of tokens to generate

temperaturenumber nullable

Sampling temperature override

top_pnumber nullable

Nucleus-sampling probability override

optionsobject nullable

Governed generation runtime options; explicit top-level sampler fields override them

stopstring[] nullable

Stop sequences

streamboolean nullable

Return SIE-native Server-Sent Events when true

frequency_penaltynumber nullable

Frequency penalty

presence_penaltynumber nullable

Presence penalty

seedinteger nullable

Optional signed 64-bit per-request sampling seed. Reproducibility is best effort, not guaranteed, and depends on the active generation backend and deployment configuration. Non-integer or out-of-range values reject with 400 invalid_request.

logit_biasobject nullable

Token-id string to finite sampler bias in [-100, 100]

logprobsboolean nullable

Return per-token log probabilities; supported only with stream true

top_logprobsinteger nullable

Alternative tokens per position; requires logprobs and stream true

routing_keystring nullable

Optional routing hint

prompt_cache_keystring nullable

Optional prompt-cache routing hint

safety_identifierstring nullable

Sensitive client identifier; validated and dropped without logging or forwarding

Example request

{
  "max_new_tokens": 64,
  "prompt": "Write one sentence about vector search.",
  "stream": false,
  "temperature": 0.7,
  "top_p": 0.9
}

Response

Generated text, or a Server-Sent Event stream when stream is true

modelstring required

Model used for generation

textstring required

Generated text

finish_reasonstring required

Reason generation stopped