---
title: "Create chat completion"
method: POST
path: "/api/chat/completions"
tags: ["gateway"]
---

# Create chat completion

`POST /api/chat/completions`

Send a chat completion request through the Respan gateway. Supports 250+ models across OpenAI, Anthropic, Google, Azure, and more with automatic logging, fallbacks, caching, and prompt management.

Accepts all [OpenAI chat completion parameters](https://platform.openai.com/docs/apis/chat). Respan-specific parameters can be passed three ways:
1. **Top-level body fields** - add directly to the request body
2. **Nested under `respan_params`** - explicit namespacing to avoid conflicts
3. **Header `X-Data-Respan-Params`** - base64-encoded JSON header

Merge order: top-level body fields > `respan_params` > header.

Legacy compatibility:
- `keywordsai_params` is still accepted and merged into `respan_params`
- `X-Data-Keywordsai-Params` is still accepted and auto-renamed internally

When using the OpenAI SDK, pass Respan parameters via `extra_body`.

## Headers

- `Authorization` string, required
- `X-Data-Respan-Params` string
- `X-Respan-Route-Provider` string
- `X-Respan-Beta` string

## Request body

- object
  - `messages` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaMessagesItems[], required — Array of messages in the conversation. Each message has `role` (`system`, `user`, `assistant`, `tool`) and `content`.
    - `role` 'system' | 'user' | 'assistant' | 'tool', required — Message role.
    - `content` union, required — Message content. Use a string for text-only requests, or an array of content parts for multimodal requests.
      - string
      - ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaMessagesItemsContentOneOf1Items[]
    - `name` string — Optional participant name.
    - `tool_call_id` string — Required for tool response messages.
  - `model` string, required — Model to use. See [Models](https://platform.respan.ai/platform/models) for available options.
  - `stream` boolean — Stream back partial progress token by token as server-sent events.
  - `tools` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaToolsItems[] — Tools the model may call. Currently only functions are supported.
  - `tool_choice` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaToolChoice — Controls tool selection. `"none"` = no tools, `"auto"` = model decides, or specify a tool object.
  - `frequency_penalty` number, double — Penalizes tokens based on frequency in text so far (-2 to 2).
  - `max_tokens` number, double — Maximum tokens to generate.
  - `temperature` number, double — Sampling temperature (0-2). Higher = more random.
  - `n` number, double — Number of completions to generate. Note: costs multiply with `n`.
  - `logprobs` boolean — Return log probabilities of output tokens.
  - `echo` boolean — Echo back the prompt in addition to the completion
  - `stop` string[] — Stop sequences where generation halts.
  - `presence_penalty` number, double — Penalizes tokens already present in text (-2 to 2).
  - `logit_bias` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLogitBias — Used to modify the probability of tokens appearing in the response
  - `response_format` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaResponseFormat — Output format. Set `{"type": "json_schema", "json_schema": {...}}` for structured output, or `{"type": "json_object"}` for JSON mode.
  - `parallel_tool_calls` boolean — Enable parallel function calling during tool use.
  - `load_balance_group` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLoadBalanceGroup — Load balance group selection. Use `{"group_id": "..."}` to route through a configured group.
  - `fallback_models` string[] — Backup models (ranked by priority) if the primary model fails.
  - `customer_credentials` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCustomerCredentials — Per-customer LLM provider credentials. Keys are provider names, values are API keys.
  - `credential_override` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCredentialOverride — One-off credential overrides per provider. Overrides uploaded provider keys for this request only.
  - `cache_enabled` boolean — Enable response caching. See [Caching](/docs/documentation/features/gateway/advanced).
  - `cache_ttl` number, double — Cache time-to-live in seconds. Default: 30 days.
  - `cache_options` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCacheOptions — Cache behavior options. Properties: `cache_by_customer`, `is_cached_by_model`, `omit_log`.
    - `cache_by_customer` boolean — Partition cache entries by customer identifier.
    - `is_cached_by_model` boolean — Partition cache entries by model name.
    - `omit_log` boolean — Suppress log creation for cache hits.
  - `prompt` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaPrompt — Prompt template config. Properties: `prompt_id` (required), `variables` (template variables), `version` (number, or `"latest"` for draft), `echo` (return rendered prompt), `override` (use override_params), `override_params` (OpenAI params to override), `schema_version` (`1` = legacy, `2` = prompt config wins). See [Prompt management](/docs/documentation/features/prompt-management/advanced).
  - `retry_params` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaRetryParams — Retry config. Properties: `retry_enabled` (boolean, required), `num_retries` (number), `retry_after` (seconds to wait).
  - `disable_log` boolean — When `true`, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.
  - `model_name_map` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaModelNameMap — Azure deployment name mapping. Maps your custom Azure deployment names to standard model names.
  - `models` string[] — Model list for LLM router selection.
  - `exclude_providers` string[] — Providers to exclude from routing. All models under excluded providers are skipped.
  - `exclude_models` string[] — Specific models to exclude from routing.
  - `metadata` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaMetadata — Custom key-value metadata attached to the span.
  - `custom_identifier` string — Indexed custom tag for fast querying.
  - `customer_identifier` string — End user identifier for analytics and budgets.
  - `customer_params` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaCustomerParams — Extended customer info. Properties: `customer_identifier` (required), `group_identifier`, `name`, `email`, `period_budget`, `budget_duration` (`daily`/`weekly`/`monthly`), `total_budget`, `markup_percentage`.
  - `request_breakdown` boolean — Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.
  - `positive_feedback` boolean — User feedback. `true` = liked, `false` = disliked.
  - `load_balance_models` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaLoadBalanceModelsItems[] — Inline load balancing options. Each item can include `model`, `weight`, and optional `credentials`.
  - `thread_identifier` string — Conversation thread ID. Spans with the same `thread_identifier` are grouped together.
  - `properties` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchema — Typed metadata preserving native types (numbers, booleans, nested objects). Unlike `metadata` which coerces to strings.
  - `retries` integer — Number of retries on failure.
  - `weight` number, double — Load balancing weight.
  - `span_name` string — Custom span name for tracing.
  - `respan_params` ApiChatCompletionsPostRequestBodyContentApplicationJsonSchemaRespanParams — Namespaced container for all Respan parameters. Alternative to passing them at top level.

## Response `200`

Successful response for Create chat completion

- GatewayCreateChatCompletionResponse200
  - `id` string, required — Chat completion ID.
  - `object` string, required
  - `created` integer, required — Unix timestamp for when the completion was created.
  - `model` string, required — Model used for the completion.
  - `choices` ApiChatCompletionsPostResponsesContentApplicationJsonSchemaChoicesItems[], required
    - `index` integer
    - `message` ApiChatCompletionsPostResponsesContentApplicationJsonSchemaChoicesItemsMessage
      - `role` string
      - `content` string
    - `finish_reason` string
  - `usage` ApiChatCompletionsPostResponsesContentApplicationJsonSchemaUsage
    - `prompt_tokens` integer
    - `completion_tokens` integer
    - `total_tokens` integer

## Other responses

- `400` — Invalid request or preprocessing failure.
- `401` — Unauthorized - Missing/invalid authentication
- `424` — The upstream model provider failed.

---

[API](https://skmtc.net/keywordsai/apis/api-reference.md) · [All operations](https://skmtc.net/keywordsai/apis/api-reference/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/keywordsai/api-reference/revisions/4e064cf81dae/schema)