Sign in

v34

latestOpenAPI 3.1.0Apache-2.0raw.githubusercontent.com2026-08-0332109227.7 KB
inference

SIE-native generation with optional bounded inline images. Omit stream or set it to false for a blocking JSON response; set stream: true for SIE-native Server-Sent Events terminated by data: [DONE]. The model path parameter must use the SIE-safe ID (for example Qwen__Qwen3-4B-Instruct); HF-style slashes reject with 400.

post/v1/generate/{model}

Path parameters

modelstring required

Model id. Runtime route is Axum catch-all /v1/generate/{*model}; clients using this OpenAPI path template should percent-encode slashes in model ids, for example BAAI%2Fbge-m3.

Headers

X-SIE-MACHINE-PROFILEstring nullable

Preferred GPU or machine profile

X-SIE-Poolstring nullable

Explicit pool routing override

X-SIE-SDK-Versionstring nullable

Client SDK version for skew warnings

Request body

frequency_penaltynumber double nullable
{"stackTrail":"components:schemas:GenerateRequest:properties:grammar","oasType":"schema","type":"unknown","description":"Optional grammar object accepted by the gateway grammar validator."}
logit_biasobject nullable

Per-token additive sampler bias. Keys are integer token ids encoded as strings; values are finite numbers in [-100, 100]. At most 1024 entries are accepted.

logprobsboolean nullable

Return per-token log probabilities on native SSE chunks. Native blocking responses reject this field because they have no logprob response member.

lora_adapterstring nullable

Served LoRA adapter name. The gateway validates it against the resolved model before queue dispatch.

max_new_tokensinteger required
{"stackTrail":"components:schemas:GenerateRequest:properties:options","oasType":"schema","type":"unknown","description":"Governed generation runtime options. Explicit top-level sampler fields\noverride matching values in ``default_sampling``."}
presence_penaltynumber double nullable
promptstring required
prompt_cache_keystring nullable
routing_keystring nullable
safety_identifierstring nullable

Sensitive PII - parsed and dropped, never logged or forwarded.

seedinteger nullable

Optional signed 64-bit per-request sampling seed. Reproducibility is best effort, not guaranteed, and depends on the active generation backend and deployment configuration. Non-integer or out-of-range values reject with 400 invalid_request.

stopstring[] nullable

Stop sequences for the native generate surface.

streamboolean nullable

SSE streaming. When true, the 200 response uses text/event-stream; each data: event contains a SIE-native GenerateChunk JSON object and the stream terminates with data: [DONE]. Defaults to false. Non-boolean values reject with 400 invalid_request.

temperaturenumber float nullable
top_logprobsinteger nullable

Number of alternative tokens per position. Requires logprobs: true and stream: true.

top_pnumber float nullable

Response

Generated text as blocking JSON, or SIE-native SSE GenerateChunk events when stream: true; SSE terminates with data: [DONE].

attempt_idstring required
finish_reasonstring required
modelstring required
textstring required
tpot_msnumber double nullable
ttft_msnumber double nullable