--- title: "Invoke" method: POST path: "/v2/deployments/invoke" tags: ["Deployments"] --- # Invoke `POST /v2/deployments/invoke` Invoke a deployment with a given payload ## Request body - InvokeDeploymentRequest — The deployment request payload - `key` string, required — The deployment key to invoke - `stream` boolean — If set, partial message content will be sent. Tokens will be sent as data-only `server-sent events` as they become available, with the stream terminated by a `data: [DONE]` message. - `inputs` object — Key-value pairs variables to replace in your prompts. If a variable is not provided that is defined in the prompt, the default variables are used. - `context` object — Key-value pairs that match your data model and fields declared in your deployment routing configuration - `prefix_messages` union[] — A list of messages to include after the `System` message, but before the `User` and `Assistant` pairs configured in your deployment. - union - object — Developer-provided instructions that the model should follow, regardless of messages sent by the user. - `role` 'system', required — The role of the messages author, in this case `system`. - `content` union, required — The contents of the system message. - string — The contents of the system message. - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported. - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - object - `role` 'developer', required — The role of the messages author, in this case `developer`. - `content` union, required — The contents of the developer message. - string — The contents of the system message. - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported. - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - object - `role` 'user', required — The role of the messages author, in this case `user`. - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `content` union, required — The contents of the user message. - string — The text contents of the message. - union[] — An array of content parts with a defined type. Supported options differ based on the model being used to generate the response. Can contain text, image, or audio inputs. - union - … - object - `content` union — The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified. - string — The contents of the assistant message. - union[] — An array of content parts with a defined type. Can be one or more of type `text`, or exactly one of type `refusal`. - union - … - `refusal` string, nullable — The refusal message by the assistant. - `role` 'assistant', required — The role of the messages author, in this case `assistant`. - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `audio` object, nullable — Data about a previous audio response from the model. - `id` string, required — Unique identifier for a previous audio response from the model. - `tool_calls` object[] — The tool calls generated by the model, such as function calls. - `id` string, required — The ID of the tool call. - `type` 'function', required — The type of the tool. Currently, only `function` is supported. - `function` object, required - `name` string — The name of the function to call. - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call. - object - `role` 'tool', required — The role of the messages author, in this case tool. - `content` union, required — The contents of the tool message. - string - TextContentPartSchema[] - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `tool_call_id` string, nullable, required — Tool call that this message is responding to. - `cache_control` object - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral". - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models. - `messages` union[] — A list of messages to send to the deployment. - union - object — Developer-provided instructions that the model should follow, regardless of messages sent by the user. - `role` 'system', required — The role of the messages author, in this case `system`. - `content` union, required — The contents of the system message. - string — The contents of the system message. - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported. - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - object - `role` 'developer', required — The role of the messages author, in this case `developer`. - `content` union, required — The contents of the developer message. - string — The contents of the system message. - TextContentPartSchema[] — An array of content parts with a defined type. For system messages, only type `text` is supported. - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - object - `role` 'user', required — The role of the messages author, in this case `user`. - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `content` union, required — The contents of the user message. - string — The text contents of the message. - union[] — An array of content parts with a defined type. Supported options differ based on the model being used to generate the response. Can contain text, image, or audio inputs. - union - … - object - `content` union — The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified. - string — The contents of the assistant message. - union[] — An array of content parts with a defined type. Can be one or more of type `text`, or exactly one of type `refusal`. - union - … - `refusal` string, nullable — The refusal message by the assistant. - `role` 'assistant', required — The role of the messages author, in this case `assistant`. - `name` string — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `audio` object, nullable — Data about a previous audio response from the model. - `id` string, required — Unique identifier for a previous audio response from the model. - `tool_calls` object[] — The tool calls generated by the model, such as function calls. - `id` string, required — The ID of the tool call. - `type` 'function', required — The type of the tool. Currently, only `function` is supported. - `function` object, required - `name` string — The name of the function to call. - `arguments` string — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. - `thought_signature` string — Encrypted representation of the model internal reasoning state during function calling. Required by Gemini 3 models when continuing a conversation after a tool call. - object - `role` 'tool', required — The role of the messages author, in this case tool. - `content` union, required — The contents of the tool message. - string - TextContentPartSchema[] - `type` 'text', required — The type of the content part. - `text` string, required — The text content. - `cache_control` object - … - `tool_call_id` string, nullable, required — Tool call that this message is responding to. - `cache_control` object - `type` 'ephemeral', required — Create a cache control breakpoint at this content block. Accepts only the value "ephemeral". - `ttl` '5m' | '1h' — The time-to-live for the cache control breakpoint. This may be one of the following values: - `5m`: 5 minutes - `1h`: 1 hour Defaults to `5m`. Only supported by `Anthropic` Claude models. - `identity` PublicIdentity — Information about the identity making the request. If the identity does not exist, it will be created automatically. - `id` string, required — Unique identifier for the contact - `display_name` string — Display name of the contact - `email` string, email — Email address of the contact - `metadata` object[] — A hash of key/value pairs containing any other data about the contact - `logo_url` string — URL to the contact's avatar or logo - `tags` string[] — A list of tags associated with the contact - `file_ids` string[] — A list of file IDs that are associated with the deployment request. - `metadata` object — Key-value pairs that you want to attach to the log generated by this request. - `extra_params` object — Utilized for passing additional parameters to the model provider. Exercise caution when using this feature, as the included parameters will overwrite any parameters specified in the deployment prompt configuration. - `documents` object[] — A list of documents from your external knowledge base (e.g., chunks retrieved from your own vector database or RAG pipeline) that provide context for the model response. These documents can be used by evaluators and guardrails to assess the relevance and accuracy of the model output against the provided context. - `text` string, required — The text content of the document - `metadata` object — Metadata about the document - `file_name` string — Name of the file the text is from. - `file_type` string — Content type of the file the text is from. - `page_number` number — The page number the text is from. - `invoke_options` object - `include_retrievals` boolean — Whether to include the retrieved knowledge chunks in the response. - `include_usage` boolean — Whether to include the usage metrics in the response. - `mock_response` string — A mock response to use instead of calling the LLM API. This is useful for testing purposes. When provided, the system will return a response object with this content as the completion, without making an actual API call to the LLM provider. This works for both streaming and non-streaming requests. Mock responses will not generate logs, traces or be counted for your plan usage. - `thread` object - `id` string, required — Unique thread identifier to group related invocations. - `tags` string[] — Optional tags to differentiate or categorize threads - `knowledge_filter` union — A filter to apply to the knowledge base chunk metadata when using knowledge bases in the deployment. - object - object — And - `and` object[], required - object — Or - `or` object[], required ## Response `200` Successful operation - object - `id` string, ulid, required — A unique identifier for the response. Can be used to add metrics to the transaction. - `created` string, date-time, required — A timestamp indicating when the object was created. Usually in a standardized format like ISO 8601 - `object` 'chat' | 'completion' | 'image', required — Indicates the type of model used to generate the response - `model` string, required — The model used to generate the response - `provider` 'openai' | 'groq' | 'cohere' | 'azure' | 'aws' | 'google' | 'google-ai' | 'huggingface' | 'togetherai' | 'perplexity' | 'anthropic' | 'leonardoai' | 'fal' | 'nvidia' | 'jina' | 'elevenlabs' | 'litellm' | 'cerebras' | 'openailike' | 'bytedance' | 'mistral' | 'deepseek' | 'moonshotai' | 'zai' | 'minimax' | 'xai' | 'alibaba' | 'tensorix' | 'scaleway' | 'hcompany' | 'inceptron' | 'wafer' | 'poolside' | 'tencent' | 'nebius' | 'fireworks' | 'baseten' | 'reson8' | 'slack' | 'orq', required — The provider used to generate the response - `is_final` boolean, required — Indicates if the response is the final response - `integration_id` string — Indicates integration id used to generate the response - `telemetry` object, required - `trace_id` string, required — The trace id for the request that generated this response - `span_id` string, required — The span id for the request that generated this response - `finalized` string, date-time — A timestamp indicating when the object was finalized. Usually in a standardized format like ISO 8601 - `system_fingerprint` string, nullable — Provider backed system fingerprint. - `retrievals` object[] — List of documents retrieved from the knowledge base. This property is only available when the `include_retrievals` flag is set to `true` in the invoke settings. When stream is set to true, the `retrievals` property will be returned in the last streamed chunk where the property `is_final` is set to `true`. - `document` string, required — Content of the retrieved chunk from the knowledge base - `metadata` object, required — Metadata of the retrieved chunk from the knowledge base - `file_name` string, required — Name of the file - `page_number` number, nullable, required — Page number of the chunk - `file_type` string, required — Type of the file - `rerank_score` number — Rerank scores are normalized to be in the range [0, 1]. Scores close to 1 indicate a high relevance to the query, and scores closer to 0 indicate low relevance. It is not accurate to assume a score of 0.9 means the document is 2x more relevant than a document with a score of 0.45 - `search_score` number, required — Search scores are normalized to be in the range [0, 1]. Search score is calculated based on `[Cosine Similarity](https://en.wikipedia.org/wiki/Cosine_similarity)` algorithm. Scores close to 1 indicate the document is closer to the query, and scores closer to 0 indicate the document is farther from the query. - `provider_response` unknown - `usage` object, nullable — Usage metrics for the response - `total_tokens` number - `prompt_tokens` number - `completion_tokens` number - `prompt_tokens_details` object - `cached_tokens` number, nullable - `completion_tokens_details` object, nullable - `reasoning_tokens` number, nullable - `choices` object[], required — A list of choices generated by the model - `index` number, required - `message` union, required - object - `type` 'tool_calls', required - `role` 'system' | 'developer' | 'assistant' | 'user' | 'exception' | 'tool' | 'prompt' | 'correction' | 'expected_output', required — The role of the prompt message - `content` string, nullable - `tool_calls` object[], required - `id` string - `index` number - `type` 'function', required - `function` object, required - `name` string, required - `arguments` string, required — JSON string arguments for the functions - `reasoning` string — Internal thought process of the model - `reasoning_signature` string — The signature holds a cryptographic token which verifies that the thinking block was generated by the model, and is verified when thinking is part of a multiturn conversation. This value should not be modified and should always be sent to the API when the reasoning is redacted. Currently only supported by `Anthropic`. - `redacted_reasoning` string — Occasionally the model's internal reasoning will be flagged by the safety systems of the provider. When this occurs, the provider will encrypt the reasoning. These redacted reasoning is decrypted when passed back to the API, allowing the model to continue its response without losing context. - object - `type` 'content', required - `role` 'system' | 'developer' | 'assistant' | 'user' | 'exception' | 'tool' | 'prompt' | 'correction' | 'expected_output', required — The role of the prompt message - `content` string, nullable, required - `reasoning` string — Internal thought process of the model - `reasoning_signature` string — The signature holds a cryptographic token which verifies that the thinking block was generated by the model, and is verified when thinking is part of a multiturn conversation. This value should not be modified and should always be sent to the API when the reasoning is redacted. Currently only supported by `Anthropic`. - `redacted_reasoning` string — Occasionally the model's internal reasoning will be flagged by the safety systems of the provider. When this occurs, the provider will encrypt the reasoning. These redacted reasoning is decrypted when passed back to the API, allowing the model to continue its response without losing context. - object - `type` 'image', required - `role` 'system' | 'developer' | 'assistant' | 'user' | 'exception' | 'tool' | 'prompt' | 'correction' | 'expected_output', required — The role of the prompt message - `url` string, required - `finish_reason` string, nullable ## Other responses - `204` — No content - successful operation --- [API](https://skmtc.net/orq-ai/apis/orq-ai-api.md) · [All operations](https://skmtc.net/orq-ai/apis/orq-ai-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/orq-ai/orq-ai-api/versions/41c02aedc7c0/schema)