---
title: "Create chat completion"
method: POST
path: "/v2/router/chat/completions"
tags: ["Chat"]
---

# Create chat completion

`POST /v2/router/chat/completions`

Creates a model response for the given chat conversation with support for retries, fallbacks, prompts, and variables.

## Request body

- object
  - `messages` union[], required — A list of messages comprising the conversation so far.
    - union
      - object — Developer-provided instructions that the model should follow, regardless of messages sent by the user.
        - `role` 'system', required — The role of the messages author, in this case `system`.
        - `content` union, required — The contents of the system message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'developer', required — The role of the messages author, in this case `developer`.
        - `content` union, required — The contents of the developer message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'user', required — The role of the messages author, in this case `user`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `content` union, required — The contents of the user message.
          - string — The text contents of the message.
          - union[] — An array of content parts with a defined type. Supported options differ based on the model being used to generate the response. Can contain text, image, or audio inputs.
            - union
              - …
      - object
        - `content` union — The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified.
          - string — The contents of the assistant message.
          - union[] — An array of content parts with a defined type. Can be one or more of type `text`, or exactly one of type `refusal`.
            - union
              - …
        - `refusal` string, nullable — The refusal message by the assistant.
        - `role` 'assistant', required — The role of the messages author, in this case `assistant`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `audio` object, nullable — Data about a previous audio response from the model.
          - `id` string, required — Unique identifier for a previous audio response from the model.
        - `tool_calls` object[] — The tool calls generated by the model, such as function calls.
          - `id` string, required — The ID of the tool call.
          - `type` 'function', required — The type of the tool. Currently, only `function` is supported.
          - `function` object, required
            - `name` string — The name of the function to call.
            - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.
          - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call.
      - object
        - `role` 'tool', required — The role of the messages author, in this case tool.
        - `content` union, required — The contents of the tool message.
          - string
          - TextContentPartSchema[]
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `tool_call_id` string, nullable, required — Tool call that this message is responding to.
        - `cache_control` object
          - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral".
          - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models.
  - `model` string, required — Model ID used to generate the response, like `openai/gpt-4o` or `anthropic/claude-haiku-4-5-20251001`. The AI Gateway offers a wide range of models with different capabilities, performance characteristics, and price points. Refer to the (Supported models)[/docs/proxy/supported-models] to browse available models.
  - `metadata` object — Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format. Keys can have a maximum length of 64 characters and values can have a maximum length of 512 characters.
  - `name` string — The name to display on the trace. If not specified, the default system name will be used.
  - `audio` object, nullable — Parameters for audio output. Required when audio output is requested with modalities: ["audio"]. Learn more.
    - `voice` 'alloy' | 'echo' | 'fable' | 'onyx' | 'nova' | 'shimmer', required — The voice the model uses to respond. Supported voices are alloy, echo, fable, onyx, nova, and shimmer.
    - `format` 'wav' | 'mp3' | 'flac' | 'opus' | 'pcm16', required — Specifies the output audio format. Must be one of wav, mp3, flac, opus, or pcm16.
  - `frequency_penalty` number, nullable — Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
  - `max_tokens` integer, nullable — `[Deprecated]`. The maximum number of tokens that can be generated in the chat completion. This value can be used to control costs for text generated via API. This value is now `deprecated` in favor of `max_completion_tokens`, and is not compatible with o1 series models.
  - `max_completion_tokens` integer, nullable — An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens
  - `logprobs` boolean, nullable — Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.
  - `top_logprobs` integer, nullable — An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.
  - `n` integer, nullable — How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.
  - `presence_penalty` number, nullable — Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
  - `response_format` union — An object specifying the format that the model must output
    - object — Default response format. Used to generate text responses
      - `type` 'text', required
    - object — JSON object response format. An older method of generating JSON responses. Using `json_schema` is recommended for models that support it. Note that the model will not generate JSON without a system or user message instructing it to do so.
      - `type` 'json_object', required
    - object — JSON Schema response format. Used to generate structured JSON responses
      - `type` 'json_schema', required
      - `json_schema` object, required
        - `description` string — A description of what the response format is for, used by the model to determine how to respond in the format.
        - `name` string, required — The name of the response format. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.
        - `schema` unknown
        - `strict` boolean — Whether to enable strict schema adherence when generating the output. If set to true, the model will always follow the exact schema defined in the schema field. Only a subset of JSON Schema is supported when strict is true.
  - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' — Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is currently only supported for `gpt-5.1-codex-max`. Any of "none", "minimal", "low", "medium", "high", "xhigh".
  - `verbosity` string — Adjusts response verbosity. Lower levels yield shorter answers.
  - `seed` number, nullable — If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.
  - `stop` union — Up to 4 sequences where the API will stop generating further tokens.
    - string
    - string[]
  - `stream_options` object, nullable — Options for streaming response. Only set this when you set stream: true.
    - `include_usage` boolean — If set, an additional chunk will be streamed before the data: [DONE] message. The usage field on this chunk shows the token usage statistics for the entire request, and the choices field will always be an empty array. All other chunks will also include a usage field, but with a null value.
  - `thinking` union
    - ThinkingConfigDisabledSchema — Disables the thinking mode capability
      - `type` 'disabled', required — Disables the thinking mode capability
    - ThinkingConfigEnabledSchema — Enables the thinking mode capability
      - `type` 'enabled', required — Enables or disables the thinking mode capability
      - `budget_tokens` number, required — Determines how many tokens the model can use for its internal reasoning process. Larger budgets can enable more thorough analysis for complex problems, improving response quality. Must be ≥1024 and less than `max_tokens`.
      - `thinking_level` 'minimal' | 'low' | 'medium' | 'high' — The level of reasoning the model should use. This setting is supported only by `gemini-3` models. If budget_tokens is specified and `thinking_level` is available, `budget_tokens` will be ignored.
    - ThinkingConfigAdaptiveSchema — Enables adaptive thinking mode where the model dynamically determines thinking depth
      - `type` 'adaptive', required — Lets the model dynamically determine when and how much to use extended thinking based on the complexity of each request. Supported on Claude Opus 4.6 and Sonnet 4.6.
  - `temperature` number, nullable — What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
  - `top_p` number, nullable — An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass.
  - `top_k` number, nullable — Limits the model to consider only the top k most likely tokens at each step.
  - `tools` object[] — A list of tools the model may call.
    - `type` 'function' — The type of the tool. Currently, only function is supported.
    - `function` object, required
      - `name` string, required — The name of the function to call.
      - `description` string — A description of what the function does, used by the model to choose when and how to call the function.
      - `parameters` object — The parameters the functions accepts, described as a JSON Schema object
        - `type` 'object', required
        - `properties` object, required
        - `required` string[]
        - `additionalProperties` boolean
      - `strict` boolean — Whether to enable strict schema adherence when generating the function call.
  - `tool_choice` union — Controls which (if any) tool is called by the model.
    - 'none' | 'auto' | 'required'
    - object
      - `type` 'function' — The type of the tool. Currently, only function is supported.
      - `function` object, required
        - `name` string, required — The name of the function to call.
  - `parallel_tool_calls` boolean — Whether to enable parallel function calling during tool use.
  - `modalities` string[], nullable — Output types that you would like the model to generate. Most models are capable of generating text, which is the default: ["text"]. The gpt-4o-audio-preview model can also be used to generate audio. To request that this model generate both text and audio responses, you can use: ["text", "audio"].
  - `guardrails` object[] — A list of guardrails to apply to the request.
    - `id` union, required
      - 'orq_pii_detection' | 'orq_secret_detection' | 'orq_sexual_moderation' | 'orq_harmful_moderation' — The key of the guardrail.
      - string — Unique key or identifier of the evaluator
    - `execute_on` 'input' | 'output', required — Determines whether the guardrail runs on the input (user message) or output (model response).
  - `plugins` union[] — Request-scoped transforms applied to the text exchanged with the model. Supports `pii_redaction`, which replaces PII with placeholders before the provider sees it and restores the original values in the response, and `response_healing`, which repairs malformed JSON in non-streaming output.
    - union
      - PIIRedactionPluginAuto
        - `id` 'pii_redaction', required — Plugin discriminator. Must be `pii_redaction`.
        - `on_failure` 'block' | 'passthrough' — Behavior when redaction is unavailable. `block` (default) fails the request; `passthrough` sends the original text.
        - `threshold` number — Detector confidence cutoff in [0,1].
        - `entities` string[] — Entity types to redact when no detector language is configured. Omit to redact every detected type.
      - PIIRedactionPluginEn
        - `id` 'pii_redaction', required — Plugin discriminator. Must be `pii_redaction`.
        - `on_failure` 'block' | 'passthrough' — Behavior when redaction is unavailable. `block` (default) fails the request; `passthrough` sends the original text.
        - `threshold` number — Detector confidence cutoff in [0,1].
        - `language` 'en', required — Detector language.
        - `entities` string[] — English entity types to redact. Omit to redact every type detected for the language.
      - PIIRedactionPluginNl
        - `id` 'pii_redaction', required — Plugin discriminator. Must be `pii_redaction`.
        - `on_failure` 'block' | 'passthrough' — Behavior when redaction is unavailable. `block` (default) fails the request; `passthrough` sends the original text.
        - `threshold` number — Detector confidence cutoff in [0,1].
        - `language` 'nl', required — Detector language.
        - `entities` string[] — Dutch entity types to redact. Omit to redact every type detected for the language.
      - ResponseHealingPlugin
        - `id` 'response_healing', required — Plugin discriminator. Must be `response_healing`.
      - TraceScrubbingPlugin
        - `id` 'trace_scrubbing', required — Plugin discriminator. Must be `trace_scrubbing`.
        - `mask` string[], required — Trace surfaces to scrub. `all` includes system, input, output, metadata, and variables.
  - `fallbacks` object[] — Array of fallback models to use if primary model fails
    - `model` string, required — Fallback model identifier
  - `retry` object — Retry configuration for the request
    - `count` number — Number of retry attempts (1-5)
    - `on_codes` number[] — HTTP status codes that trigger retry logic
  - `cache` object — Cache configuration for the request.
    - `ttl` number — Time to live for cached responses in seconds. Maximum 259200 seconds (3 days).
    - `type` 'exact_match', required
  - `load_balancer` object — Load balancer configuration for the request.
    - `type` 'weight_based', required
    - `models` object[], required
      - `model` string, required — Model identifier for load balancing
      - `weight` number — Weight assigned to this model for load balancing
  - `timeout` object — Timeout configuration to apply to the request. If the request exceeds the timeout, it will be retried or fallback to the next model if configured.
    - `call_timeout` number, required — Timeout value in milliseconds
  - `variables` object — Variables to substitute in message templates. Uses f-string syntax ({{variableName}}) by default. For advanced templating with Jinja or Mustache syntax, use in conjunction with `template_engine`.
  - `cache_control` object — Provider-level prompt caching configuration applied to the request. Creates a cache control breakpoint covering the request content. Only supported by `Anthropic` Claude models.
    - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral".
    - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models.
  - `prompt_cache_key` string — Used by OpenAI to cache responses for similar requests to optimize your cache hit rates. Replaces the legacy `user` field for prompt caching.
  - `orq` object — Leverage Orq's intelligent routing capabilities to enhance your AI application with enterprise-grade reliability and observability. Orq provides automatic request management including retries on failures, model fallbacks for high availability, identity-level analytics tracking, conversation threading, and dynamic prompt templating with variable substitution.
    - `name` string — The name to display on the trace. If not specified, the default system name will be used.
    - `retry` object — Retry configuration for the request
      - `count` number — Number of retry attempts (1-5)
      - `on_codes` number[] — HTTP status codes that trigger retry logic
    - `fallbacks` object[] — Array of fallback models to use if primary model fails
      - `model` string, required — Fallback model identifier
    - `prompt` object — Prompt configuration for the request
      - `id` string, required — Unique identifier of the prompt to use
      - `version` 'latest', required — Version of the prompt to use (currently only "latest" supported)
    - `identity` PublicIdentity — Information about the identity making the request. If the identity does not exist, it will be created automatically.
      - `id` string, required — Unique identifier for the contact
      - `display_name` string — Display name of the contact
      - `email` string, email — Email address of the contact
      - `metadata` object[] — A hash of key/value pairs containing any other data about the contact
      - `logo_url` string — URL to the contact's avatar or logo
      - `tags` string[] — A list of tags associated with the contact
    - `contact` PublicContact — @deprecated Use identity instead. Information about the contact making the request.
      - `id` string, required — Unique identifier for the contact
      - `display_name` string — Display name of the contact
      - `email` string, email — Email address of the contact
      - `metadata` object[] — A hash of key/value pairs containing any other data about the contact
      - `logo_url` string — URL to the contact's avatar or logo
      - `tags` string[] — A list of tags associated with the contact
    - `thread` object — Thread information to group related requests
      - `id` string, required — Unique thread identifier to group related invocations.
      - `tags` string[] — Optional tags to differentiate or categorize threads
    - `inputs` union — @deprecated Use top-level `variables` field instead. Values to replace in the prompt messages using {{variableName}} syntax.
      - object
      - object[]
        - `key` string, required
        - `value` unknown
        - `is_pii` boolean
    - `cache` object — Cache configuration for the request.
      - `ttl` number — Time to live for cached responses in seconds. Maximum 259200 seconds (3 days).
      - `type` 'exact_match', required
    - `knowledge_bases` object[]
      - `top_k` integer, nullable — The number of results to return. Send `null` or omit to use the knowledge base configured `top_k`.
      - `threshold` number, nullable — The threshold to apply to the search. Send `null` or omit to use the knowledge base configured `threshold`
      - `search_type` 'vector_search' | 'keyword_search' | 'hybrid_search' | 'null', nullable — The type of search to perform. Send `null` or omit to use the knowledge base configured `retrieval_type`
      - `filter_by` union — The metadata filter to apply to the search. Check the [Searching a Knowledge Base](https://docs.orq.ai/docs/knowledge/api#knowledge-base-search) for more information.
        - object
        - object — And
          - `and` object[], required
        - object — Or
          - `or` object[], required
      - `search_options` object — Additional search options
        - `include_vectors` boolean — Whether to include the vector in the chunk
        - `include_metadata` boolean — Whether to include the metadata in the chunk
        - `include_scores` boolean — Whether to include the scores in the chunk
      - `rerank_config` object — Override the rerank configuration for this search. If not provided, will use the knowledge base configured rerank settings.
        - `model` string, required — The name of the rerank model to use. Refer to the [model list](https://docs.orq.ai/docs/proxy#/rerank-models).
        - `threshold` number — The threshold value used to filter the rerank results, only documents with a relevance score greater than the threshold will be returned
        - `top_k` integer — The number of top results to return after reranking. Defaults to `10`.
      - `agentic_rag_config` object — Override the agentic RAG configuration for this search. If not provided, will use the knowledge base configured agentic RAG settings.
        - `model` string, required — The name of the model for the Agent to use. Refer to the [model list](https://docs.orq.ai/docs/proxy#/chat-models).
      - `knowledge_id` string, required — Unique identifier of the knowledge base to search
      - `query` string — The query to use to search the knowledge base. If not provided we will use the last user message from the messages of the requests
    - `load_balancer` object — Array of models with weights for load balancing requests
      - `type` 'weight_based', required
      - `models` object[], required
        - `model` string, required — Model identifier for load balancing
        - `weight` number — Weight assigned to this model for load balancing
    - `timeout` object — Timeout configuration to apply to the request. If the request exceeds the timeout, it will be retried or fallback to the next model if configured.
      - `call_timeout` number, required — Timeout value in milliseconds
  - `stream` boolean

## Response `200`

Returns a chat completion object, or a streamed sequence of chat completion chunk objects if the request is streamed.

- object — Represents a chat completion response returned by model, based on the provided input.
  - `id` string, required — A unique identifier for the chat completion.
  - `choices` object[], required — A list of chat completion choices. Can be more than one if n is greater than 1.
    - `finish_reason` 'stop' | 'length' | 'tool_calls' | 'content_filter' | 'function_call' | 'null', nullable, required — The reason the model stopped generating tokens.
    - `index` number — The index of the choice in the list of choices.
    - `message` object, required — A chat completion message generated by the model.
      - `content` string, nullable
      - `refusal` string, nullable
      - `tool_calls` object[]
        - `index` number
        - `id` string
        - `type` 'function'
        - `function` object
          - `name` string — The name of the function to be called. Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64.
          - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.
        - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call.
      - `role` 'assistant'
      - `reasoning` string, nullable — Internal thought process of the model
      - `reasoning_signature` string, nullable — The signature holds a cryptographic token which verifies that the thinking block was generated by the model, and is verified when thinking is part of a multiturn conversation. This value should not be modified and should always be sent to the API when the reasoning is redacted. Currently only supported by `Anthropic`.
      - `redacted_reasoning` string — Occasionally the model's internal reasoning will be flagged by the safety systems of the provider. When this occurs, the provider will encrypt the reasoning. These redacted reasoning is decrypted when passed back to the API, allowing the model to continue its response without losing context.
      - `audio` object, nullable — If the audio output modality is requested, this object contains data about the audio response from the model.
        - `id` string, required
        - `expires_at` integer, required
        - `data` string, required
        - `transcript` string, required
    - `logprobs` object, nullable — Log probability information for the choice.
      - `content` object[], nullable, required — A list of message content tokens with log probability information.
        - `token` string, required — The token.
        - `logprob` number, required — The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.
        - `bytes` number[], nullable, required — A list of integers representing the UTF-8 bytes representation of the token.
        - `top_logprobs` object[], required — List of the most likely tokens and their log probability, at this token position.
          - `token` string, required — The token.
          - `logprob` number, required — The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.
          - `bytes` number[], nullable, required — A list of integers representing the UTF-8 bytes representation of the token.
      - `refusal` object[], nullable, required — A list of message refusal tokens with log probability information.
        - `token` string, required — The token.
        - `logprob` number, required — The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.
        - `bytes` number[], nullable, required — A list of integers representing the UTF-8 bytes representation of the token.
        - `top_logprobs` object[], required — List of the most likely tokens and their log probability, at this token position.
          - `token` string, required — The token.
          - `logprob` number, required — The log probability of this token, if it is within the top 20 most likely tokens. Otherwise, the value -9999.0 is used to signify that the token is very unlikely.
          - `bytes` number[], nullable, required — A list of integers representing the UTF-8 bytes representation of the token.
  - `created` number, required — The Unix timestamp (in seconds) of when the chat completion was created.
  - `model` string, required — The model used for the chat completion.
  - `system_fingerprint` string, nullable — This fingerprint represents the backend configuration that the model runs with.
  - `usage` object, nullable — Usage statistics for the completion request.
    - `completion_tokens` number — Number of tokens in the generated completion.
    - `prompt_tokens` number — Number of tokens in the prompt.
    - `total_tokens` number — Total number of tokens used in the request (prompt + completion).
    - `prompt_tokens_details` object, nullable
      - `cached_tokens` integer, nullable
      - `cache_creation_tokens` integer, nullable
      - `audio_tokens` integer, nullable — The number of audio input tokens consumed by the request.
    - `completion_tokens_details` object, nullable
      - `reasoning_tokens` number, nullable
      - `accepted_prediction_tokens` number, nullable
      - `rejected_prediction_tokens` number, nullable
      - `audio_tokens` integer, nullable — The number of audio output tokens produced by the response.
  - `object` 'chat.completion', required

---

[API](https://skmtc.net/orq-ai/apis/orq-ai-api.md) · [All operations](https://skmtc.net/orq-ai/apis/orq-ai-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/orq-ai/orq-ai-api/revisions/f7a624eccddb/schema)