Sign in

v1

latestOpenAPI 3.1.02026-07-1320186461.3 KB

Completion

Get a completion response using either one of your organization’s deployed prompts, or by providing completion details including prompt and inputs in the request. This endpoint acts as a LLM gateway/proxy endpoint to generate completions from different LLMs.

post/api/parea/v1/completion

Request body

llm_inputsobject nullable

Key-value pairs as inputs to prompt template. Only needs to be provided if deployment_id is provided or llm_configuration.messages are templated.

project_namestring nullable

Project name which is used to associate the log with a project.

project_uuidstring nullable

Project UUID which is used to associate the log with a project. Does not need to be provided if the project_name is provided.

experiment_uuidstring nullable

Experiment UUID which is used to associate the log with an experiment.

parent_trace_idstring nullable

UUID of the parent log. If given, will be used to associate the generation in a chain & create hierarchical nested logs.

root_trace_idstring nullable

UUID of the root log. If given, will be used to associate the generation in a chain & create hierarchical nested logs.

end_user_identifierstring nullable

Special field to track the end user which is interacting with your LLM app.

deployment_idstring nullable

This is the ID for a specific deployed prompt. You can find your deployed prompts on the Deployments tab. If a deployment_id is provided, Parea will fetch all of the associated configuration including model name, model parameters, and any associated functions. Any information provided on the llm_configuration field will be used instead of the associated deployed prompts fields.

eval_metric_idsinteger[] nullable

List of evaluation metric IDs deployed on Parea which should be used to evaluate the completion output.

metadataobject nullable

Key-value pairs to be associated with the log.

tagsstring[] nullable

List of tags to be associated with the log.

targetstring nullable

Optional ground truth output for the inputs. Will be used for evaluation and can be used when creating a test case from the log.

trace_idstring nullable

UUID of the generation log. If not given, will be auto-generated.

trace_namestring nullable

Name of the generation log. If not given, will be auto-generated in the format llm-{provider}.

provider_api_keystring nullable

Provider API key to generate response. If not given, API keys saved on the platform will be used

cacheboolean

If true, the completion will be cached to avoid latency & cost for any subsequent completion using the same inputs.

log_omit_inputsboolean

If true, the inputs, llm_configuration.messages, llm_configuration.functions, llm_configuration.model_params will not be logged.

log_omit_outputsboolean

If true, the generated response will not be logged.

log_omitboolean

Equivalent to setting both log_omit_inputs and log_omit_outputs to true.

log_sample_ratenumber nullable

If specified, this log and its entire associated trace will logged with this probability. Must be between 0 and 1 (incl.). Defaults to 1.0 (i.e., keeping all logs)

inference_idstring nullable

Deprecated field which is the same trace_id

namestring nullable

Deprecated field

retryboolean

Deprecated field

fallback_strategystring[] nullable

Deprecated field

streamboolean

Deprecated field. Use /completion/stream instead.

Example request

{
  "end_user_identifier": "2596@gmail.com",
  "inputs": {
    "city": "New York"
  },
  "llm_configuration": {
    "function_call": "auto",
    "functions": [
      {
        "description": "Get the current weather in a given location.",
        "name": "get_current_weather",
        "parameters": {
          "properties": {
            "location": {
              "description": "The city and state, e.g. San Francisco, CA",
              "type": "string"
            },
            "unit": {
              "enum": [
                "celsius",
                "fahrenheit"
              ],
              "type": "string"
            }
          },
          "required": [
            "location"
          ],
          "type": "object"
        }
      }
    ],
    "messages": [
      {
        "content": "What is the weather in New York",
        "role": "user"
      }
    ],
    "model": "gpt-4o-mini",
    "model_params": {
      "max_length": 500,
      "model": "gpt-4o-mini",
      "temp": 0
    },
    "provider": "openai"
  },
  "metadata": {
    "purpose": "testing",
    "user_id": "user_2Q9afRAKe6SZrc3U7"
  },
  "tags": [
    "weather",
    "forecast"
  ]
}

Response

Successful Response

contentstring required

Generated completion content.

latencynumber required

Latency of the completion in seconds.

input_tokensinteger required

Number of tokens in the input.

output_tokensinteger required

Number of tokens in the output.

total_tokensinteger required

Total number of tokens in the input and output.

costnumber required

Cost of the completion in USD.

modelstring required

Model name.

providerstring required

Provider name.

cache_hitboolean required

If true, the completion was fetched from the cache.

statusstring required

Status of the completion. Either 'success' or 'error'.

errorstring nullable

Error message if the completion failed.

trace_idstring nullable

UUID of the log of the completion. Will be the same as the trace_id in the request if provided.

start_timestampstring required

Start timestamp of the completion.

end_timestampstring required

End timestamp of the completion.

inference_idstring required

UUID of the log of the completion. The same as the trace_id in the request if provided.

Example response

{
  "cache_hit": true,
  "content": "Sure! Here's a simple Hello World program...",
  "cost": 0.0008,
  "end_timestamp": "2024-07-24 18:05:17",
  "input_tokens": 22,
  "latency": 0,
  "model": "gpt-4o-mini-2024-07-18",
  "output_tokens": 189,
  "start_timestamp": "2024-07-24 18:05:17",
  "status": "success",
  "total_tokens": 211,
  "trace_id": "ush221-d23rfcasc-21cdsc"
}