Chat API (v2)

Defaults to false.

When true, the response will be a SSE stream of events.

Streaming is beneficial for user interfaces that render the contents of the response piece by piece, as it gets generated.

The name of a compatible Cohere model.

When set to true, tool calls in the Assistant message will be forced to follow the tool definition strictly. Learn more in the Structured Outputs (Tools) guide.

Note: The first few requests with a new set of tools will take longer to process.

Used to select the safety instruction inserted into the prompt. Defaults to CONTEXTUAL. When OFF is specified, the safety instruction will be omitted.

Safety modes are not yet configurable in combination with tools and documents parameters.

Note: This parameter is only compatible newer Cohere models, starting with Command R 08-2024 and Command R+ 08-2024.

Note: command-r7b-12-2024 and newer models only support "CONTEXTUAL" and "STRICT" modes.

The maximum number of output tokens the model will generate in the response. If not set, max_tokens defaults to the model's maximum output token limit. You can find the maximum output token limits for each model in the model documentation.

Note: Setting a low value may result in incomplete generations. In such cases, the finish_reason field in the response will be set to "MAX_TOKENS".

Note: If max_tokens is set higher than the model's maximum output token limit, the generation will be capped at that model-specific maximum limit.

Defaults to 0.3.

A non-negative float that tunes the degree of randomness in generation. Lower temperatures mean less random generations, and higher temperatures mean more random generations.

Randomness can be further maximized by increasing the value of the p parameter.

If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed.

Defaults to 0.0, min value of 0.0, max value of 1.0. Used to reduce repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to previously present tokens, proportional to how many times they have already appeared in the prompt or prior generation.

Defaults to 0.0, min value of 0.0, max value of 1.0. Used to reduce repetitiveness of generated tokens. Similar to frequency_penalty, except that this penalty is applied equally to all tokens that have already appeared, regardless of their exact frequencies.

Ensures that only the top k most likely tokens are considered for generation at each step. When k is set to 0, k-sampling is disabled. Defaults to 0, min value of 0, max value of 500.

Ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step. If both k and p are enabled, p acts after k. Defaults to 0.75. min value of 0.01, max value of 0.99.

Defaults to false. When set to true, the log probabilities of the generated tokens will be included in the response.

Used to control whether or not the model will be forced to use a tool when answering. When REQUIRED is specified, the model will be forced to use at least one of the user-defined tools, and the tools parameter must be passed in the request. When NONE is specified, the model will be forced not to use one of the specified tools, and give a direct response. If tool_choice isn't specified, then the model is free to choose whether to use the specified tools or not.

Note: This parameter is only compatible with models Command-r7b and newer.

Controls how early the request is handled. Lower numbers indicate higher priority (default: 0, the highest). When the system is under load, higher-priority requests are processed first and are the least likely to be dropped.