4e064cf81dae

latestOpenAPI 3.1.02026-08-131831,331960.0 KB

gateway

Create chat completion

Send a chat completion request through the Respan gateway. Supports 250+ models across OpenAI, Anthropic, Google, Azure, and more with automatic logging, fallbacks, caching, and prompt management.

Accepts all OpenAI chat completion parameters. Respan-specific parameters can be passed three ways:

Top-level body fields - add directly to the request body
Nested under respan_params - explicit namespacing to avoid conflicts
Header X-Data-Respan-Params - base64-encoded JSON header

Merge order: top-level body fields > respan_params > header.

Legacy compatibility:

keywordsai_params is still accepted and merged into respan_params
X-Data-Keywordsai-Params is still accepted and auto-renamed internally

When using the OpenAI SDK, pass Respan parameters via extra_body.

post/api/chat/completions

Headers

Authorizationstring required

Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.

X-Data-Respan-Paramsstring

Base64-encoded JSON object of Respan parameters. Legacy X-Data-Keywordsai-Params is still accepted.

X-Respan-Route-Providerstring

Pin the request to a specific provider without changing the model slug. Example: vertex_ai routes a claude-sonnet-4-5-20250929 request to Vertex AI Claude.

X-Respan-Betastring

Comma-separated beta feature flags. Available: token-breakdown-2026-03-26, env-scoped-integrations-2026-03-28

Request body

modelstring required

Model to use. See Models for available options.

streamboolean

Stream back partial progress token by token as server-sent events.

toolsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaToolsItems[]

Tools the model may call. Currently only functions are supported.

tool_choiceApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaToolChoice

Controls tool selection. "none" = no tools, "auto" = model decides, or specify a tool object.

frequency_penaltynumber double

Penalizes tokens based on frequency in text so far (-2 to 2).

max_tokensnumber double

Maximum tokens to generate.

temperaturenumber double

Sampling temperature (0-2). Higher = more random.

nnumber double

Number of completions to generate. Note: costs multiply with n.

logprobsboolean

Return log probabilities of output tokens.

echoboolean

Echo back the prompt in addition to the completion

stopstring[]

Stop sequences where generation halts.

presence_penaltynumber double

Penalizes tokens already present in text (-2 to 2).

logit_biasApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLogitBias

Used to modify the probability of tokens appearing in the response

response_formatApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaResponseFormat

Output format. Set {"type": "json_schema", "json_schema": {...}} for structured output, or {"type": "json_object"} for JSON mode.

parallel_tool_callsboolean

Enable parallel function calling during tool use.

load_balance_groupApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLoadBalanceGroup

Load balance group selection. Use {"group_id": "..."} to route through a configured group.

fallback_modelsstring[]

Backup models (ranked by priority) if the primary model fails.

customer_credentialsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCustomerCredentials

Per-customer LLM provider credentials. Keys are provider names, values are API keys.

credential_overrideApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCredentialOverride

One-off credential overrides per provider. Overrides uploaded provider keys for this request only.

cache_enabledboolean

Enable response caching. See Caching.

cache_ttlnumber double

Cache time-to-live in seconds. Default: 30 days.

promptApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaPrompt

Prompt template config. Properties: prompt_id (required), variables (template variables), version (number, or "latest" for draft), echo (return rendered prompt), override (use override_params), override_params (OpenAI params to override), schema_version (1 = legacy, 2 = prompt config wins). See Prompt management.

retry_paramsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaRetryParams

Retry config. Properties: retry_enabled (boolean, required), num_retries (number), retry_after (seconds to wait).

disable_logboolean

When true, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.

model_name_mapApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaModelNameMap

Azure deployment name mapping. Maps your custom Azure deployment names to standard model names.

modelsstring[]

Model list for LLM router selection.

exclude_providersstring[]

Providers to exclude from routing. All models under excluded providers are skipped.

exclude_modelsstring[]

Specific models to exclude from routing.

metadataApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaMetadata

Custom key-value metadata attached to the span.

custom_identifierstring

Indexed custom tag for fast querying.

customer_identifierstring

End user identifier for analytics and budgets.

customer_paramsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCustomerParams

Extended customer info. Properties: customer_identifier (required), group_identifier, name, email, period_budget, budget_duration (daily/weekly/monthly), total_budget, markup_percentage.

request_breakdownboolean

Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.

positive_feedbackboolean

User feedback. true = liked, false = disliked.

load_balance_modelsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLoadBalanceModelsItems[]

Inline load balancing options. Each item can include model, weight, and optional credentials.

thread_identifierstring

Conversation thread ID. Spans with the same thread_identifier are grouped together.

propertiesApiChatCompletionsPostRequestBodyContentApplicationJsonSchema

Typed metadata preserving native types (numbers, booleans, nested objects). Unlike metadata which coerces to strings.

retriesinteger

Number of retries on failure.

weightnumber double

Load balancing weight.

span_namestring

Custom span name for tracing.

respan_paramsApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaRespanParams

Namespaced container for all Respan parameters. Alternative to passing them at top level.

Response

Successful response for Create chat completion

idstring required

Chat completion ID.

objectstring required

createdinteger required

Unix timestamp for when the completion was created.

modelstring required

Model used for the completion.