--- title: "Create a chat completion" method: POST path: "/v1/chat/completions" tags: ["Chat Completions"] --- # Create a chat completion `POST /v1/chat/completions` Creates a chat completion for the provided conversation. This endpoint is fully compatible with the OpenAI Chat Completions API, allowing you to use standard OpenAI SDKs by changing only the base URL and API key. ## Request body - ChatCompletionRequest — Request body for creating a chat completion. - `messages` ChatCompletionMessage[], required — A list of messages representing the conversation history. Supports roles: `system`, `user`, `assistant`, and `tool`. - `role` 'system' | 'user' | 'assistant' | 'tool', required — The role of the message author: `system` (instructions), `user` (input), `assistant` (model response), or `tool` (tool result). - `content` union — The message content. Can be a string or an array of content parts (text, image, audio) for multimodal inputs. - string - union[] - union - ChatCompletionContentPartTextParam — Text content part. - `type` 'text', required — The content type, always `text`. - `text` string, required — The text content. - ChatCompletionContentPartImageParam — Image content part for vision models. - `type` 'image_url', required — The content type, always `image_url`. - `image_url` ImageURL, required — An image URL with optional detail settings. - … - ChatCompletionContentPartInputAudioParam — Audio content part for audio-capable models. - `type` 'input_audio', required — The content type, always `input_audio`. - `input_audio` InputAudio, required — Audio input data. - … - `name` string — An optional name for the participant. Useful for distinguishing between multiple users or assistants. - `tool_calls` ChatCompletionMessageToolCallParam[] — Tool calls generated by the model (for assistant messages). - `id` string, required — The ID of the tool call. - `index` integer — The index of the tool call. - `function` Function, required — The arguments to call the function with, as generated by the model in JSON format. The model may not always generate valid JSON and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. - `arguments` union, required — The function arguments as a JSON string or object. - string - object - `name` string, required — The name of the function. - `type` 'function', required — The type, always `function`. - `tool_call_id` string — The ID of the tool call this message responds to (required for tool messages). - `model` string, required — The model slug to use for completion, such as `deepseek-ai/DeepSeek-V4-Pro`. Find available models at [Model APIs](https://app.baseten.co/model-apis/create). - `frequency_penalty` number — Penalizes tokens based on how frequently they appear in the text so far. Positive values decrease repetition. Support varies by model. - `logit_bias` object — A map of token IDs to bias values (-100 to 100). Use this to increase or decrease the likelihood of specific tokens appearing in the output. - `logprobs` boolean — If `true`, returns log probabilities of the output tokens. Log probability support varies by model. - `top_logprobs` integer — Number of most likely tokens to return at each position (0-20). Requires `logprobs: true`. Log probability support varies by model. - `max_tokens` integer — Maximum number of tokens to generate. If your request input plus `max_tokens` exceeds the model's context length, `max_tokens` is truncated. If your request exceeds the context length by more than 16k tokens or if `max_tokens` signals no preference, context reservation is throttled to 49512 tokens. Higher `max_tokens` values slightly deprioritize request scheduling. - `n` integer — Number of completions to generate. Only `1` is supported. - `presence_penalty` number — Penalizes tokens based on whether they have appeared in the text so far. Positive values encourage the model to discuss new topics. Support varies by model. - `response_format` union — Specifies the output format. Use `{"type": "json_object"}` for JSON mode, or `{"type": "json_schema", "json_schema": {...}}` for structured outputs with a specific schema. - ResponseFormatText — Plain text response format. - `type` 'text', required — The response format type, always `text`. - ResponseFormatJson — JSON schema response format for structured outputs. - `type` 'json_schema', required — The response format type, always `json_schema`. - `json_schema` JsonSchema, required — A JSON schema for structured output. - `name` string, required — The name of the schema. - `description` string — A description of the schema. - `schema` object, required — The JSON Schema definition. - `strict` true — If `true`, enables strict schema adherence. - ResponseFormatJsonObject — JSON object response format. - `type` 'json_object', required — The response format type, always `json_object`. - ResponseFormatGrammar — Grammar-based response format. - `type` 'grammar', required — The response format type, always `grammar`. - `grammar` string, required — The grammar definition string. - ResponseFormatStructuralTag — Structural tag response format. - `type` 'structural_tag', required — The response format type, always `structural_tag`. - `structural_tag` string, required — The structural tag definition. - `seed` integer — Random seed for deterministic generation. Determinism is not guaranteed across different hardware or model versions. - `stop` union — Up to 32 sequences where the API stops generating further tokens. Can be a string or array of strings. - string - string[] - `stream` boolean — If `true`, responses are streamed back as server-sent events (SSE) as they are generated. - `stream_options` StreamOptions — Options for streaming responses. - `include_usage` boolean — If `true`, includes token usage statistics in the final streaming chunk. - `continuous_usage_stats` boolean — If `true`, includes running token usage statistics in each streaming chunk. - `temperature` number — Controls randomness in the output. Lower values like 0.2 produce more focused and deterministic responses. Higher values like 1.5 produce more creative and varied output. - `top_p` number — Nucleus sampling: only consider tokens with cumulative probability up to this value. Lower values like 0.1 produce more focused output. - `tools` ChatCompletionToolsParam[] — A list of tools (functions) the model may call. Each tool should have a `type: "function"` and a `function` object with `name`, `description`, and `parameters`. - `type` 'function' — The type of tool, always `function`. - `function` FunctionDefinition, required — A function definition that the model can call. - `name` string, required — The name of the function. - `description` string — A description of what the function does. - `parameters` object — The parameters the function accepts, as a JSON Schema object. - `strict` boolean — If `true`, enables strict schema adherence. - `tool_choice` union — Controls which tool (if any) the model calls. - `none`: Never call a tool. - `auto`: Model decides whether to call a tool. - `required`: Model must call at least one tool. - `{"type": "function", "function": {"name": "..."}}`: Call a specific function. - 'none' | 'required' | 'auto' - ChatCompletionNamedToolChoiceParam — Forces the model to call a specific function. - `function` ChatCompletionNamedFunction, required — Specifies a function to call by name. - `name` string, required — The name of the function to call. - `type` 'function' — The type, always `function`. - `parallel_tool_calls` boolean — If `true`, the model can call multiple tools in a single response. - `user` string — A unique identifier for the end-user, useful for tracking and abuse detection. - `best_of` integer — Number of candidate sequences to generate and return the best from. Only a value of 1 is supported. - `top_k` integer — Limits token selection to the top K most probable tokens at each step. Lower values like 10 produce more focused output. Set to -1 to disable. - `top_p_min` number — Minimum value for dynamic `top_p`. When set, `top_p` dynamically adjusts but does not go below this value. - `min_p` number — Minimum probability threshold for token selection. Filters out tokens with probability below `min_p * max_probability`. - `repetition_penalty` number — Multiplicative penalty for repeated tokens. Values greater than 1.0 discourage repetition, values less than 1.0 encourage it. - `length_penalty` number — Exponential penalty applied to sequence length during beam search. Values greater than 1.0 favor longer sequences. - `early_stopping` boolean — If `true`, stops generation when at least `n` complete candidates are found. - `bad` union — Words or phrases to avoid in the output. Support varies by model. - string - string[] - `bad_token_ids` integer[] — Token IDs to avoid in the output. Support varies by model. - `stop_token_ids` integer[] — List of token IDs that cause generation to stop when encountered. - `include_stop_str_in_output` boolean — If `true`, includes the matched stop string in the output. - `ignore_eos` boolean — If `true`, continues generating past the end-of-sequence token. - `min_tokens` integer — Minimum number of tokens to generate before stopping. Useful for ensuring responses are not too short. - `skip_special_tokens` boolean — If `true`, removes special tokens from the generated output. - `spaces_between_special_tokens` boolean — If `true`, adds spaces between special tokens in the output. - `truncate_prompt_tokens` integer — If set, truncates the prompt to this many tokens. Useful for handling inputs that may exceed context limits. - `echo` boolean — If `true` and the last message role matches the generation role, prepends that message to the output. - `add_generation_prompt` boolean — If `true`, adds the generation prompt from the chat template, such as `<|assistant|>`. Set to `false` for completion-style generation. - `add_special_tokens` boolean — If `true`, adds special tokens like BOS to the prompt beyond what the chat template adds. For most models, the chat template handles special tokens, so this should be `false`. - `documents` object[] — A list of documents for RAG (retrieval-augmented generation). Each document is a dict with string keys and values that the model can reference. - `chat_template` string — A custom Jinja template for formatting the conversation. If not provided, uses the model's default template. - `chat_template_args` object — Additional arguments to pass to the chat template renderer. - `disaggregated_params` DisaggregatedParams — Advanced parameters for disaggregated serving. Used internally. - `request_type` string, required — The type of disaggregated request. - `first_gen_tokens` integer[] — First generation tokens for continuation. - `ctx_request_id` integer — Context request identifier. - `opaque_state` string — Opaque state for continuation. - `draft_tokens` integer[] — Draft tokens for speculative decoding. - `multimodal_embedding_handles` object[] — Handles for multimodal embeddings. - `multimodal_hashes` array[] — Hashes for multimodal content. - integer[] ## Response `200` Successful response - ChatCompletionResponse — A chat completion response returned by the model. - `id` string — A unique identifier for the chat completion. - `object` 'chat.completion.chunk' — The object type, always `chat.completion` or `chat.completion.chunk` for streaming. - `created` integer — The Unix timestamp (in seconds) of when the completion was created. - `model` string, required — The model used for the completion. - `choices` ChatCompletionResponseStreamChoice[], required — A list of chat completion choices. - `index` integer, required — The index of this choice in the list of choices. - `delta` DeltaMessage, required — A delta message chunk in a streaming response. - `role` string — The role of the message author (typically `assistant`). - `content` string — The content chunk generated by the model. - `tool_calls` ToolCall[] — Tool calls generated by the model. - `index` integer, required — The index of this tool call in the list of tool calls. - `id` string — A unique identifier for this tool call. - `type` 'function' — The type of tool call (always `function`). - `function` FunctionCall, required — The name and arguments of a function that should be called, as generated by the model. - `name` string — The name of the function to call. - `arguments` string, required — The arguments to call the function with, as a JSON string. - `logprobs` ChatCompletionLogProbs — Log probability information for the completion. - `content` ChatCompletionLogProbsContent[] — A list of log probability information for each token in the content. - `token` string, required — The token string. - `logprob` number — The log probability of the token. - `bytes` integer[] — The UTF-8 byte representation of the token. - `top_logprobs` ChatCompletionLogProb[] — List of the most likely tokens and their log probabilities at this position. - `token` string, required — The token string. - `logprob` number — The log probability of the token. - `bytes` integer[] — The UTF-8 byte representation of the token. - `finish_reason` string — The reason the model stopped generating: `stop` (natural stop or stop sequence), `length` (max tokens reached), or `tool_calls` (model called a tool). - `stop_reason` union — The specific stop sequence or token ID that caused generation to stop. - integer - string - `usage` UsageInfo — Token usage statistics for the request. - `completion_tokens` integer — Number of tokens in the generated completion. - `prompt_tokens` integer — Number of tokens in the prompt. - `total_tokens` integer — Total number of tokens used (prompt + completion). - `completion_tokens_details` CompletionTokensDetails — Breakdown of tokens used in the completion. - `accepted_prediction_tokens` integer — Number of tokens in accepted predictions (for speculative decoding). - `audio_tokens` integer — Number of audio tokens generated. - `reasoning_tokens` integer — Number of tokens used for reasoning (for models that support extended thinking). - `rejected_prediction_tokens` integer — Number of tokens in rejected predictions (for speculative decoding). - `prompt_tokens_details` PromptTokensDetails — Breakdown of tokens used in the prompt. - `audio_tokens` integer — Number of audio tokens in the prompt. - `cached_tokens` integer — Number of tokens retrieved from cache. ## Other responses - `400` — Bad request: invalid parameters. - `401` — Unauthorized: invalid or missing API key. - `429` — Rate limit exceeded. - `500` — Internal server error. --- [API](https://skmtc.net/baseten/apis/baseten-inference-api.md) · [All operations](https://skmtc.net/baseten/apis/baseten-inference-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/baseten/baseten-inference-api/revisions/a55a2f64c3c5/schema)