--- title: "Create chat completion" method: POST path: "/v1/confidential/chat/completions" tags: ["chat"] --- # Create chat completion `POST /v1/confidential/chat/completions` This handler performs several key operations: 1. Forwards the chat completion request to the inference service 2. Signs the response using the node's keystore 3. Tracks token usage for the stack # Arguments * `Extension((stack_small_id, estimated_total_tokens))` - Stack ID and estimated tokens count from middleware * `state` - Application state containing the inference client and keystore * `payload` - The chat completion request body # Returns Returns a JSON response containing: - The inference service's response - A cryptographic signature of the response # Errors Returns a `AtomaServiceError::InternalError` if: - The inference service request fails - Response parsing fails - Response signing fails - Token usage update fails ## Request body - ChatCompletionRequest — Represents the chat completion request. This is used to represent the chat completion request in the chat completion request. It can be either a chat completion or a chat completion stream. - `frequency_penalty` number, float, nullable — Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far - `function_call` unknown - `functions` unknown[], nullable — A list of functions the model may generate JSON inputs for - unknown - `logit_bias` object, nullable — Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. - `max_completion_tokens` integer, nullable — The maximum number of tokens to generate in the chat completion - `max_tokens` integer, nullable — The maximum number of tokens to generate in the chat completion - `messages` ChatCompletionMessage[], required — A list of messages comprising the conversation so far - union — A message that is part of a conversation which is based on the role of the author of the message. This is used to represent the message in the chat completion request. It can be either a system message, a user message, an assistant message, or a tool message. - object — The role of the messages author, in this case system. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `role` 'system', required - object — The role of the messages author, in this case user. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `role` 'user', required - object — The role of the messages author, in this case assistant. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `refusal` string, nullable — The refusal message by the assistant. - `role` 'assistant', required - `tool_calls` ToolCall[] — The tool calls generated by the model, such as function calls. - `function` ToolCallFunction, required — Represents the function that the model called. This is used to represent the function that the model called in the chat completion request. It can be either a function or a tool call. - `arguments` string, required — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. - `name` string, required — The name of the function to call. - `id` string, required — The ID of the tool call. - `type` string, required — The type of the tool. Currently, only function is supported. - object — The role of the messages author, in this case tool. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `role` 'tool', required - `tool_call_id` string — Tool call that this message is responding to. - `model` string, required — ID of the model to use - `n` integer, nullable — How many chat completion choices to generate for each input message - `parallel_tool_calls` boolean, nullable — Whether to enable parallel tool calls. - `presence_penalty` number, float, nullable — Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far - `response_format` ResponseFormat — The format to return the response in. This is used to represent the format to return the response in in the chat completion request. It can be either text, json_object, or json_schema. - `json_schema` JsonSchemaResponseFormat — The format to return the response in. This is used to represent the format to return the response in in the chat completion request. It can be either text, json_object, or json_schema. - `description` string, nullable — The description of the response format. - `name` string, required — The name of the response format. - `schema` unknown - `strict` boolean, nullable — Whether to strictly validate the JSON schema. - `type` 'text' | 'json_object' | 'json_schema', required — The format to return the response in. - `seed` integer, nullable — If specified, our system will make a best effort to sample deterministically - `service_tier` string, nullable — Specifies the latency tier to use for processing the request. This parameter is relevant for customers subscribed to the scale tier service: If set to 'auto', and the Project is Scale tier enabled, the system will utilize scale tier credits until they are exhausted. If set to 'auto', and the Project is not Scale tier enabled, the request will be processed using the default service tier with a lower uptime SLA and no latency guarantee. If set to 'default', the request will be processed using the default service tier with a lower uptime SLA and no latency guarantee. When not set, the default behavior is 'auto'. - `stop` string[], nullable — Up to 4 sequences where the API will stop generating further tokens - `stream` boolean, nullable — Whether to stream back partial progress - `stream_options` StreamOptions — Specifies the stream options for the request. - `include_usage` boolean, nullable — If set, an additional chunk will be streamed before the data: [DONE] message. The usage field on this chunk shows the token usage statistics for the entire request, and the choices field will always be an empty array. All other chunks will also include a usage field, but with a null value. - `temperature` number, float, nullable — What sampling temperature to use, between 0 and 2 - `tool_choice` union — A tool choice that can be used in a chat completion. This is used to represent the tool choice in the chat completion request. It can be either a literal tool choice or a named tool choice. - 'none' | 'auto' — A literal tool choice that can be used in a chat completion. This is used to represent the literal tool choice in the chat completion request. It can be either none or auto. - ChatCompletionNamedToolChoiceParam — A named tool choice that can be used in a chat completion. This is used to represent the named tool choice in the chat completion request. - `function` ChatCompletionNamedFunction, required — A named function that can be used in a chat completion. This is used to represent the named function in the chat completion request. - `name` string, required — The name of the function. - `type` string, required — The type of the tool choice. - `tools` ChatCompletionToolsParam[], nullable — A list of tools the model may call - `function` ChatCompletionToolFunctionParam, required — A function that can be used in a chat completion. - `description` string, nullable — The description of the function. - `name` string, required — The name of the function. - `parameters` object, nullable — The parameters of the function. - `strict` boolean, nullable — Whether to strictly validate the parameters of the function. - `type` string, required — The type of the tool. - `top_logprobs` integer, nullable — An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used. - `top_p` number, float, nullable — An alternative to sampling with temperature - `user` string, nullable — A unique identifier representing your end-user ## Response `200` Chat completion successful - ChatCompletionResponse — Represents the chat completion response. This is used to represent the chat completion response in the chat completion request. It can be either a chat completion or a chat completion stream. - `choices` ChatCompletionChoice[], required — A list of chat completion choices. - `finish_reason` string, nullable — The reason the chat completion was finished. - `index` integer, required — The index of this choice in the list of choices. - `logprobs` ChatCompletionLogProbs — Represents the chat completion log probs. This is used to represent the chat completion log probs in the chat completion request. It can be either a chat completion log probs or a chat completion log probs choice. - `content` ChatCompletionLogProbsContent[], nullable — The log probs of the chat completion. - `top_logprobs` ChatCompletionLogProb[], required - `bytes` integer[], nullable — A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token. - `logprob` number, float, required — The log prob of the chat completion. - `token` string, required — The token of the chat completion. - `message` union, required — A message that is part of a conversation which is based on the role of the author of the message. This is used to represent the message in the chat completion request. It can be either a system message, a user message, an assistant message, or a tool message. - object — The role of the messages author, in this case system. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `role` 'system', required - object — The role of the messages author, in this case user. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `role` 'user', required - object — The role of the messages author, in this case assistant. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `name` string, nullable — An optional name for the participant. Provides the model information to differentiate between participants of the same role. - `refusal` string, nullable — The refusal message by the assistant. - `role` 'assistant', required - `tool_calls` ToolCall[] — The tool calls generated by the model, such as function calls. - `function` ToolCallFunction, required — Represents the function that the model called. This is used to represent the function that the model called in the chat completion request. It can be either a function or a tool call. - `arguments` string, required — The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. - `name` string, required — The name of the function to call. - `id` string, required — The ID of the tool call. - `type` string, required — The type of the tool. Currently, only function is supported. - object — The role of the messages author, in this case tool. - `content` union — Represents the content of a message. This is used to represent the content of a message in the chat completion request. It can be either a text or an array of content parts. - string — The text contents of the message. - MessageContentPart[] — An array of content parts with a defined type, each can be of type text or image_url when passing in images. You can pass multiple images by adding multiple image_url content parts. Image input is only supported when using the gpt-4o model. - union — Represents a part of a message content. This is used to represent the content of a message in the chat completion request. It can be either a text or an image. - … - `role` 'tool', required - `tool_call_id` string — Tool call that this message is responding to. - `created` integer, required — The Unix timestamp (in seconds) of when the chat completion was created. - `id` string, required — A unique identifier for the chat completion. - `model` string, required — The model used for the chat completion. - `object` string, required — The object of the chat completion. - `service_tier` string, nullable — The service tier of the chat completion. - `system_fingerprint` string, nullable — The system fingerprint for the completion, if applicable. - `usage` CompletionUsage — Represents the completion usage. This is used to represent the completion usage in the chat completion request. It can be either a completion usage or a completion chunk usage. - `completion_tokens` integer, required — Number of tokens in the completion. - `prompt_tokens` integer, required — Number of tokens in the prompt. - `prompt_tokens_details` PromptTokensDetails — Represents the prompt tokens details. This is used to represent the prompt tokens details in the chat completion request. It can be either a prompt tokens details or a prompt tokens details choice. - `cached_tokens` integer, required — Number of tokens in the prompt that were cached. - `total_tokens` integer, required — Total number of tokens used (prompt + completion). ## Other responses - `500` — Internal server error --- [API](https://skmtc.net/atomaai/apis/atoma-service.md) · [All operations](https://skmtc.net/atomaai/apis/atoma-service/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/atomaai/atoma-service/revisions/a3d329cb3125/schema)