Sign in

v4

latestOpenAPI 3.0.0raw.githubusercontent.com2024-09-3042325.2 KB

Get LLM completions as a non-streaming response

post/.api/completions/code

Query parameters

api-version1 | 2 | 3
client-namestring

The name of the application sending this request. For example, "cody-web" or "jetbrains".

client-versionstring

The version of the application sending this request. For example, "2.0.0".

Request body

modelstring required

The language model to use for the completion. Call /.api/modelconfig/supported-models.json to get the list of available models.

maxTokensToSampleinteger

The maximum number of tokens to generate before stopping. Note that our models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate.

temperaturenumber double

Amount of randomness injected into the response.

Defaults to 1.0. Ranges from 0.0 to 1.0. Use temperature closer to 0.0 for analytical / multiple choice, and closer to 1.0 for creative and generative tasks.

Note that even with temperature of 0.0, the results will not be fully deterministic.

stopSequencesstring[]

Sequences that will cause the model to stop generating.

Our models stop on "\n\nHuman:", and may include additional built-in stop sequences in the future. By providing the stop_sequences parameter, you may include additional strings that will cause the model to stop generating.

topKinteger

Only sample from the top K options for each subsequent token.

Used to remove "long tail" low probability responses. Learn more technical details here: https://towardsdatascience.com/how-to-sample-from-language-models-682bceb97277

Recommended for advanced use cases only. You usually only need to use temperature.

topPnumber double

Use nucleus sampling.

In nucleus sampling, we compute the cumulative distribution over all the options for each subsequent token in decreasing probability order and cut it off once it reaches a particular probability specified by top_p. You should either alter temperature or top_p, but not both.

Recommended for advanced use cases only. You usually only need to use temperature.

streamboolean nullable

Whether to stream the response.

  • When true, the response body is an SSE stream of event: completion\ndata: JSON where each JSON blob is a CompletionResponse.
  • When false, the response body is a single JSON-encoded CompletionResponse.
logprobsinteger

Include the log probabilities on the logprobs most likely tokens, as well the chosen tokens. For example, if logprobs is 5, the API will return a list of the 5 most likely tokens. The API will always return the logprob of the sampled token, so there may be up to logprobs+1 elements in the response. The maximum value for logprobs is 5. For more details, see Fireworks API docs https://docs.fireworks.ai/api-reference/post-completions

Response

The request has succeeded.

OR