--- title: "Chat API (v2)" method: POST path: "/v2/chat" --- # Chat API (v2) `POST /v2/chat` Generates a text response to a user message and streams it down, token by token. To learn how to use the Chat API with streaming follow our [Text Generation guides](https://docs.cohere.com/v2/docs/chat-api). Follow the [Migration Guide](https://docs.cohere.com/v2/docs/migrating-v1-to-v2) for instructions on moving from API v1 to API v2. ## Headers - `X-Client-Name` string ## Request body - object - `stream` boolean — Defaults to `false`. When `true`, the response will be a SSE stream of events. Streaming is beneficial for user interfaces that render the contents of the response piece by piece, as it gets generated. - `model` string, required — The name of a compatible [Cohere model](https://docs.cohere.com/v2/docs/models). - `messages` ChatMessageV2[], required — A list of chat messages in chronological order, representing a conversation between the user and the model. Messages can be from `User`, `Assistant`, `Tool` and `System` roles. Learn more about messages and roles in [the Chat API guide](https://docs.cohere.com/v2/docs/chat-api). - union — Represents a single message in the chat history from a given role. - object — A message from the user. - `role` 'user', required - `content` union, required — The content of the message. This can be a string or a list of content blocks. If a string is provided, it will be treated as a text content block. - string - Content[] - union — A Content block which contains information about the content type and the content itself. - … - object — A message from the assistant role can contain text and tool call information. - `role` 'assistant', required - `tool_calls` ToolCallV2[] - `id` string, required - `type` 'function', required - `function` object - `name` string - `arguments` string - `tool_plan` string — A chain-of-thought style reflection and plan that the model generates when working with Tools. - `content` union - string - union[] - union - … - `citations` Citation[] - `start` integer — Start index of the cited snippet in the original source text. - `end` integer — End index of the cited snippet in the original source text. - `text` string — Text snippet that is being cited. - `sources` Source[] - union — A source object containing information about the source of the data cited. - … - `content_index` integer — Index of the content block in which this citation appears. - `type` 'TEXT_CONTENT' | 'THINKING_CONTENT' | 'PLAN' — The type of citation which indicates what part of the response the citation is for. - object — A message from the system. - `role` 'system', required - `content` union, required - string - ChatTextContent[] - `type` 'text', required - `text` string, required - object — A message with Tool outputs. - `role` 'tool', required - `tool_call_id` string, required — The id of the associated tool call that has provided the given content - `content` union, required — Outputs from a tool. The content should formatted as a JSON object string, or a list of tool content blocks - string - ToolContent[] - union — A content block which contains information about the content of a tool result - … - `tools` ToolV2[] — A list of tools (functions) available to the model. The model response may contain 'tool_calls' to the specified tools. Learn more in the [Tool Use guide](https://docs.cohere.com/docs/tools). - `type` 'function', required - `function` object — The function to be executed. - `name` string, required — The name of the function. - `description` string — The description of the function. - `parameters` object, required — The parameters of the function as a JSON schema. - `strict_tools` boolean — When set to `true`, tool calls in the Assistant message will be forced to follow the tool definition strictly. Learn more in the [Structured Outputs (Tools) guide](https://docs.cohere.com/docs/structured-outputs-json#structured-outputs-tools). **Note**: The first few requests with a new set of tools will take longer to process. - `documents` union[] — A list of relevant documents that the model can cite to generate a more accurate reply. Each document is either a string or document object with content and metadata. - union - string - Document — Relevant information that could be used by the model to generate a more accurate reply. The content of each document are generally short (should be under 300 words). Metadata should be used to provide additional information, both the key name and the value will be passed to the model. - `data` object, required — A relevant document that the model can cite to generate a more accurate reply. Each document is a string-any dictionary. - `id` string — Unique identifier for this document which will be referenced in citations. If not provided an ID will be automatically generated. - `citation_options` CitationOptions — Options for controlling citation generation. - `mode` 'ENABLED' | 'DISABLED' | 'FAST' | 'ACCURATE' | 'OFF' — Defaults to `"enabled"`. Citations are enabled by default for models that support it, but can be turned off by setting `"type": "disabled"`. - `response_format` union — Configuration for forcing the model output to adhere to the specified format. Supported on [Command R](https://docs.cohere.com/v2/docs/command-r), [Command R+](https://docs.cohere.com/v2/docs/command-r-plus) and newer models. The model can be forced into outputting JSON objects by setting `{ "type": "json_object" }`. A [JSON Schema](https://json-schema.org/) can optionally be provided, to ensure a specific structure. **Note**: When using `{ "type": "json_object" }` your `message` should always explicitly instruct the model to generate a JSON (eg: _"Generate a JSON ..."_) . Otherwise the model may end up getting stuck generating an infinite stream of characters and eventually run out of context length. **Note**: When `json_schema` is not specified, the generated object can have up to 5 layers of nesting. **Limitation**: The parameter is not supported when used in combinations with the `documents` or `tools` parameters. - object - `type` 'text' | 'json_object', required — Defaults to `"text"`. When set to `"json_object"`, the model's output will be a valid JSON Object. - object - `type` 'text' | 'json_object', required — Defaults to `"text"`. When set to `"json_object"`, the model's output will be a valid JSON Object. - `json_schema` object — A [JSON schema](https://json-schema.org/overview/what-is-jsonschema) object that the output will adhere to. There are some restrictions we have on the schema, refer to [our guide](https://docs.cohere.com/docs/structured-outputs-json#schema-constraints) for more information. Example (required name and age object): ```json { "type": "object", "properties": { "name": {"type": "string"}, "age": {"type": "integer"} }, "required": ["name", "age"] } ``` **Note**: This field must not be specified when the `type` is set to `"text"`. - `safety_mode` 'CONTEXTUAL' | 'STRICT' | 'OFF' — Used to select the [safety instruction](https://docs.cohere.com/v2/docs/safety-modes) inserted into the prompt. Defaults to `CONTEXTUAL`. When `OFF` is specified, the safety instruction will be omitted. Safety modes are not yet configurable in combination with `tools` and `documents` parameters. **Note**: This parameter is only compatible newer Cohere models, starting with [Command R 08-2024](https://docs.cohere.com/docs/command-r#august-2024-release) and [Command R+ 08-2024](https://docs.cohere.com/docs/command-r-plus#august-2024-release). **Note**: `command-r7b-12-2024` and newer models only support `"CONTEXTUAL"` and `"STRICT"` modes. - `max_tokens` integer — The maximum number of output tokens the model will generate in the response. If not set, `max_tokens` defaults to the model's maximum output token limit. You can find the maximum output token limits for each model in the [model documentation](https://docs.cohere.com/docs/models). **Note**: Setting a low value may result in incomplete generations. In such cases, the `finish_reason` field in the response will be set to `"MAX_TOKENS"`. **Note**: If `max_tokens` is set higher than the model's maximum output token limit, the generation will be capped at that model-specific maximum limit. - `stop_sequences` string[] — A list of up to 5 strings that the model will use to stop generation. If the model generates a string that matches any of the strings in the list, it will stop generating tokens and return the generated text up to that point not including the stop sequence. - `temperature` number, float — Defaults to `0.3`. A non-negative float that tunes the degree of randomness in generation. Lower temperatures mean less random generations, and higher temperatures mean more random generations. Randomness can be further maximized by increasing the value of the `p` parameter. - `seed` integer — If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed. - `frequency_penalty` number, float — Defaults to `0.0`, min value of `0.0`, max value of `1.0`. Used to reduce repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation. - `presence_penalty` number, float — Defaults to `0.0`, min value of `0.0`, max value of `1.0`. Used to reduce repetitiveness of generated tokens. Similar to `frequency_penalty`, except that this penalty is applied equally to all tokens that have already appeared, regardless of their exact frequencies. - `k` integer — Ensures that only the top `k` most likely tokens are considered for generation at each step. When `k` is set to `0`, k-sampling is disabled. Defaults to `0`, min value of `0`, max value of `500`. - `p` number, float — Ensures that only the most likely tokens, with total probability mass of `p`, are considered for generation at each step. If both `k` and `p` are enabled, `p` acts after `k`. Defaults to `0.75`. min value of `0.01`, max value of `0.99`. - `logprobs` boolean — Defaults to `false`. When set to `true`, the log probabilities of the generated tokens will be included in the response. - `tool_choice` 'REQUIRED' | 'NONE' — Used to control whether or not the model will be forced to use a tool when answering. When `REQUIRED` is specified, the model will be forced to use at least one of the user-defined tools, and the `tools` parameter must be passed in the request. When `NONE` is specified, the model will be forced **not** to use one of the specified tools, and give a direct response. If tool_choice isn't specified, then the model is free to choose whether to use the specified tools or not. **Note**: This parameter is only compatible with models [Command-r7b](https://docs.cohere.com/v2/docs/command-r7b) and newer. - `thinking` Thinking — Configuration for [reasoning features](https://docs.cohere.com/docs/reasoning). - `type` 'enabled' | 'disabled', required — Reasoning is enabled by default for models that support it, but can be turned off by setting `"type": "disabled"`. - `token_budget` integer — The maximum number of tokens the model can use for thinking, which must be set to a positive integer. The model will stop thinking if it reaches the thinking token budget and will proceed with the response. - `priority` integer — Controls how early the request is handled. Lower numbers indicate higher priority (default: 0, the highest). When the system is under load, higher-priority requests are processed first and are the least likely to be dropped. ## Response `200` OK - union - ChatResponseV2 - `id` string, required — Unique identifier for the generated reply. Useful for submitting feedback. - `finish_reason` 'COMPLETE' | 'STOP_SEQUENCE' | 'MAX_TOKENS' | 'TOOL_CALL' | 'ERROR' | 'TIMEOUT', required — The reason a chat request has finished. - **complete**: The model finished sending a complete message. - **max_tokens**: The number of generated tokens exceeded the model's context length or the value specified via the `max_tokens` parameter. - **stop_sequence**: One of the provided `stop_sequence` entries was reached in the model's generation. - **tool_call**: The model generated a Tool Call and is expecting a Tool Message in return - **error**: The generation failed due to an internal error - **timeout**: The generation was stopped because it exceeded the allowed time limit. - `message` AssistantMessageResponse, required — A message from the assistant role can contain text and tool call information. - `role` 'assistant', required - `tool_calls` ToolCallV2[] - `id` string, required - `type` 'function', required - `function` object - `name` string - `arguments` string - `tool_plan` string — A chain-of-thought style reflection and plan that the model generates when working with Tools. - `content` union[] - union - ChatTextContent — Text content of the message. - `type` 'text', required - `text` string, required - ChatThinkingContent — Thinking content of the message. This will be present when `thinking` is enabled, and will contain the models internal reasoning. - `type` 'thinking', required - `thinking` string, required - `citations` Citation[] - `start` integer — Start index of the cited snippet in the original source text. - `end` integer — End index of the cited snippet in the original source text. - `text` string — Text snippet that is being cited. - `sources` Source[] - union — A source object containing information about the source of the data cited. - object - … - object — A document source object containing the unique identifier of the document and the document itself. - … - `content_index` integer — Index of the content block in which this citation appears. - `type` 'TEXT_CONTENT' | 'THINKING_CONTENT' | 'PLAN' — The type of citation which indicates what part of the response the citation is for. - `usage` Usage - `billed_units` object - `input_tokens` number — The number of billed input tokens. - `output_tokens` number — The number of billed output tokens. - `search_units` number — The number of billed search units. - `classifications` number — The number of billed classifications units. - `tokens` object - `input_tokens` number — The number of tokens used as input to the model. - `output_tokens` number — The number of tokens produced by the model. - `cached_tokens` number — The number of prompt tokens that hit the inference cache. - `logprobs` LogprobItem[] - `text` string — The text chunk for which the log probabilities was calculated. - `token_ids` integer[], required — The token ids of each token used to construct the text chunk. - `logprobs` number[] — The log probability of each token used to construct the text chunk. - union — StreamedChatResponse is returned in streaming mode (specified with `stream=True` in the request). - ChatMessageStartEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `id` string — Unique identifier for the generated reply. - `delta` object - `message` object - `role` 'assistant' — The role of the message. - ChatContentStartEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - `delta` object - `message` object - `content` object - `thinking` string - `text` string - `type` 'text' | 'thinking' - ChatContentDeltaEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - `delta` object - `message` object - `content` object - `thinking` string - `text` string - `logprobs` LogprobItem - `text` string — The text chunk for which the log probabilities was calculated. - `token_ids` integer[], required — The token ids of each token used to construct the text chunk. - `logprobs` number[] — The log probability of each token used to construct the text chunk. - ChatContentEndEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - ChatToolPlanDeltaEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `delta` object - `message` object - `tool_plan` string - ChatToolCallStartEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - `delta` object - `message` object - `tool_calls` ToolCallV2 — An array of tool calls to be made. - `id` string, required - `type` 'function', required - `function` object - … - ChatToolCallDeltaEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - `delta` object - `message` object - `tool_calls` object - `function` object - … - ChatToolCallEndEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `index` integer - ChatMessageEndEvent — The streamed event types - `type` 'message-start' | 'content-start' | 'content-delta' | 'content-end' | 'tool-call-start' | 'tool-call-delta' | 'tool-call-end' | 'tool-plan-delta' | 'citation-start' | 'citation-end' | 'message-end', required - `id` string - `delta` object - `error` string — An error message if an error occurred during the generation. - `finish_reason` 'COMPLETE' | 'STOP_SEQUENCE' | 'MAX_TOKENS' | 'TOOL_CALL' | 'ERROR' | 'TIMEOUT' — The reason a chat request has finished. - **complete**: The model finished sending a complete message. - **max_tokens**: The number of generated tokens exceeded the model's context length or the value specified via the `max_tokens` parameter. - **stop_sequence**: One of the provided `stop_sequence` entries was reached in the model's generation. - **tool_call**: The model generated a Tool Call and is expecting a Tool Message in return - **error**: The generation failed due to an internal error - **timeout**: The generation was stopped because it exceeded the allowed time limit. - `usage` Usage - `billed_units` object - `input_tokens` number — The number of billed input tokens. - `output_tokens` number — The number of billed output tokens. - `search_units` number — The number of billed search units. - `classifications` number — The number of billed classifications units. - `tokens` object - `input_tokens` number — The number of tokens used as input to the model. - `output_tokens` number — The number of tokens produced by the model. - `cached_tokens` number — The number of prompt tokens that hit the inference cache. - ChatDebugEvent - `event_type` 'stream-start' | 'search-queries-generation' | 'search-results' | 'text-generation' | 'citation-generation' | 'stream-end' | 'debug', required - `prompt` string ## Other responses - `400` — This error is returned when the request is not well formed. This could be because: - JSON is invalid - The request is missing required fields - The request contains an invalid combination of fields - `401` — This error indicates that the operation attempted to be performed is not allowed. This could be because: - The api token is invalid - The user does not have the necessary permissions - `403` — This error indicates that the operation attempted to be performed is not allowed. This could be because: - The api token is invalid - The user does not have the necessary permissions - `404` — This error is returned when a resource is not found. This could be because: - The endpoint does not exist - The resource does not exist eg model id, dataset id - `422` — This error is returned when the request is not well formed. This could be because: - JSON is invalid - The request is missing required fields - The request contains an invalid combination of fields - `429` — Too many requests - `498` — This error is returned when a request or response contains a deny-listed token. - `499` — This error is returned when a request is cancelled by the user. - `500` — This error is returned when an uncategorised internal server error occurs. - `501` — This error is returned when the requested feature is not implemented. - `503` — This error is returned when the service is unavailable. This could be due to: - Too many users trying to access the service at the same time - `504` — This error is returned when a request to the server times out. This could be due to: - An internal services taking too long to respond --- [API](https://skmtc.net/cohere-ai/apis/api-reference.md) · [All operations](https://skmtc.net/cohere-ai/apis/api-reference/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/cohere-ai/api-reference/revisions/dbed69eda210/schema)