v1

latestOpenAPI 3.1.02026-07-2675394318.9 KB

inference

Create completion

Creates a model completion for the given input prompt.

post/v1/completions

Query parameters

ai_project_idstring nullable

current project ID

Request body

modelstring required

ID of the model to use.

streamboolean nullable

Enable response streaming.

stream_optionsobject nullable

If set to {"include_usage": True}, usage stats will be sent with the last chunk of data

max_tokensinteger nullable

Max completion token count

temperaturenumber nullable

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

top_pnumber nullable

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

ninteger nullable

How many completions to generate for each prompt.

logprobsinteger nullable

Include the log probabilities on the logprobs most likely tokens, as well the chosen tokens. So for example, if logprobs is 5, the API will return a list of the 5 most likely tokens. The API will always return the logprob of the sampled token, so there may be up to logprobs+1 elements in the response.

echoboolean nullable

Echo back the prompt in addition to the completion.

presence_penaltynumber nullable

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

frequency_penaltynumber nullable

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

logit_biasobject nullable

Modify the likelihood of specified tokens appearing in the completion. Accepts a json object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.

userstring nullable

A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse.

extra_bodyobject nullable

To provide extra parameters.

service_tier'auto' | 'default' | 'over-limit' | 'flex' | 'no-limit'

Represents the service tier for requests.

Attributes: Auto: Automatically choose the best available tier for the request (Default or OverLimit). Analyze response to determine which tier was used. Default: Return 429 errors on hitting the rate limit, do not exceed to the OverLimit tier. OverLimit: Indicate that the request was over the user limit. This tier cannot be set by user in the request, but us used in a response for tier=Auto. Flex: Do not consume rate-limit credits, but run with lower priority. May still result in 429 errors in case of if there is no resources to process.

Example request

{
  "model": "meta-llama/Meta-Llama-3.1-70B-Instruct",
  "max_tokens": 100
}

Response

idstring required

A unique identifier for the chat completion.

objectstring required

The object type, which is always text_completion.

createdinteger required

The Unix timestamp of when the completion was created.

modelstring required

The model used for the chat completion.

service_tier'auto' | 'default' | 'over-limit' | 'flex' | 'no-limit' required

Represents the service tier for requests.