v1
latestOpenAPI 3.0.02026-07-1741634.0 KBCreate chat completion
Generates a completion for the supplied prompt. Streaming mode is supported. All extra options described in the documentation are optional and follow the OpenAI‑compatible naming.
Request body
The unique identifier of the model to use.
If true, the server streams tokens as they are generated.
Maximum number of tokens to generate.
Stop sequences – generation stops before emitting any of these strings.
Adjusts randomness of the output.
Dynamic‑temperature range – final temperature is sampled from [temperature‑range, temperature+range].
Exponent for dynamic‑temperature scaling.
Restricts sampling to the K most probable tokens.
Nucleus sampling cutoff (cumulative probability).
Minimum probability for a token to be considered relative to the most likely token.
Enable locally typical sampling with parameter p.
Maximum number of tokens to predict; -1 = unlimited, 0 = evaluate prompt only.
Minimum line indentation for generated code.
Tokens from the prompt to retain when context is exceeded. 0 = none, -1 = all.
Presence penalty (0.0 = disabled).
Frequency penalty (0.0 = disabled).
Repetition penalty for token sequences.
How many last tokens to consider for repeat penalty (0 = disabled, -1 = context size).
DRY (Don’t Repeat Yourself) multiplier (0.0 = disabled).
DRY base value.
Length after which DRY penalty grows exponentially.
How many tokens to scan for DRY repetitions (0 = disabled, -1 = context size).
Sequence breakers for DRY sampling.
Probability for token removal via XTC sampler (0.0 = disabled).
Minimum probability threshold for XTC.
Enable Mirostat sampling (0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0).
Target entropy for Mirostat.
Learning rate for Mirostat.
Grammar for grammar‑based sampling.
JSON schema for grammar‑based sampling.
RNG seed; -1 = random.
Continue generating after EOS token.
Modify the likelihood of a token appearing. Accepts a JSON object mapping token IDs to a bias value from -100 to 100, or false to ban a token.
If > 0, top-N token probabilities are returned.
Force samplers to return at least N tokens.
Maximum generation time in milliseconds (0 = disabled).
Assign the request to a specific inference slot (-1 = auto).
Reuse KV cache from previous requests when possible.
Include raw token IDs in the response.
Ordered list of samplers to apply.
Include timing information per token.
Include prompt‑processing progress when streaming.
Return top‑N probabilities after sampling.
Select which fields to include in the response.
Base64‑encoded multimodal data (images, audio, …). Must match the number of <__media__> markers in the prompt.
Response
Completion result
Unique identifier for the completion.
Unix timestamp of creation.
Model used for the completion.
Fingerprint of the system configuration used.
The generation settings used for the completion.
The processed prompt.
True if the context size was exceeded.
Number of tokens from the prompt which were reused from a cache.
Number of tokens evaluated in total from the prompt.