Sign in

v1

latestOpenAPI 3.1.0Apache-2.02026-07-13136499.6 KB
Completions

Create completions

This function processes completion requests by using the chat completions endpoint.

Returns

Returns a Response containing either:

  • A streaming SSE connection for real-time completions
  • A single JSON response for non-streaming completions

Errors

Returns an error status code if:

  • The request processing fails
  • The streaming/non-streaming handlers encounter errors
  • The underlying inference service returns an error
post/v1/completions

Request body

best_ofinteger nullable
echoboolean nullable
frequency_penaltynumber float nullable

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far

logit_biasobject nullable

Modify the likelihood of specified tokens appearing in the completion.

Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.

logprobsinteger nullable

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability.

max_tokensinteger nullable

The maximum number of tokens to generate in the chat completion

modelstring required

ID of the model to use

ninteger nullable

How many chat completion choices to generate for each input message

presence_penaltynumber float nullable

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far

seedinteger nullable

If specified, our system will make a best effort to sample deterministically

stopstring[] nullable

Up to 4 sequences where the API will stop generating further tokens

streamboolean nullable

Whether to stream back partial progress

suffixstring nullable

The suffix that comes after a completion of inserted text.

temperaturenumber float nullable

What sampling temperature to use, between 0 and 2

top_pnumber float nullable

An alternative to sampling with temperature

userstring nullable

A unique identifier representing your end-user

Example request

{
  "best_of": 1,
  "logit_bias": {
    "1234567890": 0.5,
    "1234567891": -0.5
  },
  "logprobs": 1,
  "max_tokens": 4096,
  "model": "meta-llama/Llama-3.3-70B-Instruct",
  "n": 1,
  "seed": 123,
  "suffix": "json(\"\\n\")",
  "temperature": 0.7,
  "top_p": 1,
  "user": "user-1234"
}

Response

Chat completions

createdinteger required

The creation time of the request

idstring required

The ID of the request

modelstring required

The model used for the request

objectstring required

The object type

system_fingerprintstring required

The system fingerprint

Example response

{
  "choices": [
    {
      "text": "This is a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "id": "cmpl-1234567890",
  "model": "meta-llama/Llama-3.3-70B-Instruct",
  "object": "text_completion",
  "system_fingerprint": "system-fingerprint",
  "usage": {
    "completion_tokens": 10,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 10,
      "reasoning_tokens": 10
    },
    "prompt_tokens": 10,
    "prompt_tokens_details": {
      "cached_tokens": 10
    },
    "total_tokens": 20
  }
}
All 13 operations