---
title: "Stream"
method: POST
path: "/v2/deployments/stream"
tags: ["Deployments"]
---

# Stream

`POST /v2/deployments/stream`

Stream deployment generation. Only supported for completions and chat completions.

## Request body

- object
  - `key` string, required — The deployment key to invoke
  - `inputs` object — Key-value pairs variables to replace in your prompts. If a variable is not provided that is defined in the prompt, the default variables are used.
  - `context` object — Key-value pairs that match your data model and fields declared in your deployment routing configuration
  - `prefix_messages` union[] — A list of messages to include after the `System` message, but before the `User` and `Assistant` pairs configured in your deployment.
    - union
      - object — Developer-provided instructions that the model should follow, regardless of messages sent by the user.
        - `role` 'system', required — The role of the messages author, in this case `system`.
        - `content` union, required — The contents of the system message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'developer', required — The role of the messages author, in this case `developer`.
        - `content` union, required — The contents of the developer message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'user', required — The role of the messages author, in this case `user`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `content` union, required — The contents of the user message.
          - string — The text contents of the message.
          - union[] — An array of content parts with a defined type. Supported options differ based on the model being used to generate the response. Can contain text, image, or audio inputs.
            - union
              - …
      - object
        - `content` union — The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified.
          - string — The contents of the assistant message.
          - union[] — An array of content parts with a defined type. Can be one or more of type `text`, or exactly one of type `refusal`.
            - union
              - …
        - `refusal` string, nullable — The refusal message by the assistant.
        - `role` 'assistant', required — The role of the messages author, in this case `assistant`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `audio` object, nullable — Data about a previous audio response from the model.
          - `id` string, required — Unique identifier for a previous audio response from the model.
        - `tool_calls` object[] — The tool calls generated by the model, such as function calls.
          - `id` string, required — The ID of the tool call.
          - `type` 'function', required — The type of the tool. Currently, only `function` is supported.
          - `function` object, required
            - `name` string — The name of the function to call.
            - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.
          - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call.
      - object
        - `role` 'tool', required — The role of the messages author, in this case tool.
        - `content` union, required — The contents of the tool message.
          - string
          - TextContentPartSchema[]
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `tool_call_id` string, nullable, required — Tool call that this message is responding to.
        - `cache_control` object
          - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral".
          - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models.
  - `messages` union[] — A list of messages to send to the deployment.
    - union
      - object — Developer-provided instructions that the model should follow, regardless of messages sent by the user.
        - `role` 'system', required — The role of the messages author, in this case `system`.
        - `content` union, required — The contents of the system message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'developer', required — The role of the messages author, in this case `developer`.
        - `content` union, required — The contents of the developer message.
          - string — The contents of the system message.
          - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported.
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
      - object
        - `role` 'user', required — The role of the messages author, in this case `user`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `content` union, required — The contents of the user message.
          - string — The text contents of the message.
          - union[] — An array of content parts with a defined type. Supported options differ based on the model being used to generate the response. Can contain text, image, or audio inputs.
            - union
              - …
      - object
        - `content` union — The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified.
          - string — The contents of the assistant message.
          - union[] — An array of content parts with a defined type. Can be one or more of type `text`, or exactly one of type `refusal`.
            - union
              - …
        - `refusal` string, nullable — The refusal message by the assistant.
        - `role` 'assistant', required — The role of the messages author, in this case `assistant`.
        - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role.
        - `audio` object, nullable — Data about a previous audio response from the model.
          - `id` string, required — Unique identifier for a previous audio response from the model.
        - `tool_calls` object[] — The tool calls generated by the model, such as function calls.
          - `id` string, required — The ID of the tool call.
          - `type` 'function', required — The type of the tool. Currently, only `function` is supported.
          - `function` object, required
            - `name` string — The name of the function to call.
            - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.
          - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call.
      - object
        - `role` 'tool', required — The role of the messages author, in this case tool.
        - `content` union, required — The contents of the tool message.
          - string
          - TextContentPartSchema[]
            - `type` 'text', required — The type of the content part.
            - `text` string, required — The text content.
            - `cache_control` object
              - …
        - `tool_call_id` string, nullable, required — Tool call that this message is responding to.
        - `cache_control` object
          - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral".
          - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models.
  - `identity` PublicIdentity — Information about the identity making the request. If the identity does not exist, it will be created automatically.
    - `id` string, required — Unique identifier for the contact
    - `display_name` string — Display name of the contact
    - `email` string, email — Email address of the contact
    - `metadata` object[] — A hash of key/value pairs containing any other data about the contact
    - `logo_url` string — URL to the contact's avatar or logo
    - `tags` string[] — A list of tags associated with the contact
  - `file_ids` string[] — A list of file IDs that are associated with the deployment request.
  - `metadata` object — Key-value pairs that you want to attach to the log generated by this request.
  - `extra_params` object — Utilized for passing additional parameters to the model provider. Exercise caution when using this feature, as the included parameters will overwrite any parameters specified in the deployment prompt configuration.
  - `documents` object[] — A list of documents from your external knowledge base (e.g., chunks retrieved from your own vector database or RAG pipeline) that provide context for the model response. These documents can be used by evaluators and guardrails to assess the relevance and accuracy of the model output against the provided context.
    - `text` string, required — The text content of the document
    - `metadata` object — Metadata about the document
      - `file_name` string — Name of the file the text is from.
      - `file_type` string — Content type of the file the text is from.
      - `page_number` number — The page number the text is from.
  - `invoke_options` object
    - `include_retrievals` boolean — Whether to include the retrieved knowledge chunks in the response.
    - `include_usage` boolean — Whether to include the usage metrics in the response.
    - `mock_response` string — A mock response to use instead of calling the LLM API. This is useful for testing purposes. When provided, the system will return a response object with this content as the completion, without making an actual API call to the LLM provider. This works for both streaming and non-streaming requests. Mock responses will not generate logs, traces or be counted for your plan usage.
  - `thread` object
    - `id` string, required — Unique thread identifier to group related invocations.
    - `tags` string[] — Optional tags to differentiate or categorize threads
  - `knowledge_filter` union — A filter to apply to the knowledge base chunk metadata when using knowledge bases in the deployment.
    - object
    - object — And
      - `and` object[], required
    - object — Or
      - `or` object[], required

## Response `200`

Successful operation

---

[API](https://skmtc.net/orq-ai/apis/orq-ai-api.md) · [All operations](https://skmtc.net/orq-ai/apis/orq-ai-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/orq-ai/orq-ai-api/revisions/deb8d376faee/schema)