---
title: "Chat Completions"
method: POST
path: "/v1/chat/completions"
---

# Chat Completions

`POST /v1/chat/completions`

OpenAI-compatible chat completions endpoint.

Supports any HuggingFace causal language model.

## Request body

- ChatCompletionRequest — OpenAI-compatible chat completion request.
  - `model` string, required
  - `messages` union[], required
    - union
      - ChatCompletionDeveloperMessageParam — Developer-provided instructions that the model should follow, regardless of messages sent by the user. With o1 models and newer, `developer` messages replace the previous `system` messages.
        - `content` union, required
          - string
          - ChatCompletionContentPartTextParam[]
            - `text` string, required
            - `type` 'text', required
        - `role` 'developer', required
        - `name` string
      - ChatCompletionSystemMessageParam — Developer-provided instructions that the model should follow, regardless of messages sent by the user. With o1 models and newer, use `developer` messages for this purpose instead.
        - `content` union, required
          - string
          - ChatCompletionContentPartTextParam[]
            - `text` string, required
            - `type` 'text', required
        - `role` 'system', required
        - `name` string
      - ChatCompletionUserMessageParam — Messages sent by an end user, containing prompts or additional context information.
        - `content` union, required
          - string
          - union[]
            - union
              - …
        - `role` 'user', required
        - `name` string
      - ChatCompletionAssistantMessageParam — Messages sent by the model in response to user messages.
        - `role` 'assistant', required
        - `audio` Audio — Data about a previous audio response from the model. [Learn more](https://platform.openai.com/docs/guides/audio).
          - `id` string, required
        - `content` union
          - string
          - union[]
            - union
              - …
        - `function_call` FunctionCall — Deprecated and replaced by `tool_calls`. The name and arguments of a function that should be called, as generated by the model.
          - `arguments` string, required
          - `name` string, required
        - `name` string
        - `refusal` string, nullable
        - `tool_calls` union[]
          - union
            - ChatCompletionMessageFunctionToolCallParam — A call to a function tool created by the model.
              - …
            - ChatCompletionMessageCustomToolCallParam — A call to a custom tool created by the model.
              - …
      - ChatCompletionToolMessageParam
        - `content` union, required
          - string
          - ChatCompletionContentPartTextParam[]
            - `text` string, required
            - `type` 'text', required
        - `role` 'tool', required
        - `tool_call_id` string, required
      - ChatCompletionFunctionMessageParam
        - `content` string, nullable, required
        - `name` string, required
        - `role` 'function', required
  - `temperature` number, nullable
  - `top_p` number, nullable
  - `max_tokens` integer, nullable
  - `stream` boolean, nullable
  - `stop` union
    - string
    - string[]
  - `logprobs` boolean, nullable
  - `top_logprobs` integer, nullable
  - `presence_penalty` number, nullable
  - `frequency_penalty` number, nullable
  - `user` string, nullable
  - `n_ctx` integer, nullable
  - `n_batch` integer, nullable
  - `n_gpu_layers` integer, nullable
  - `n_threads` integer, nullable
  - `flash_attn` boolean, nullable
  - `use_mmap` boolean, nullable
  - `use_mlock` boolean, nullable
  - `cache_type_k` string, nullable
  - `cache_type_v` string, nullable
  - `extra_body` object, nullable
  - `tools` ChatCompletionFunctionToolParam[], nullable
    - `function` FunctionDefinition, required
      - `name` string, required
      - `description` string
      - `parameters` object
      - `strict` boolean, nullable
    - `type` 'function', required
  - `tool_choice` union
    - string
    - object
  - `think` boolean, nullable
  - `thinking_budget` integer, nullable
  - `cache_key` string, nullable
  - `return_cache_key` boolean, nullable
  - `auto_truncate` boolean, nullable
  - `truncation_strategy` string, nullable

## Response `200`

Successful Response

- unknown

## Other responses

- `422` — Validation Error

---

[API](https://skmtc.net/llama-farm/apis/llamafarm-edge-runtime.md) · [All operations](https://skmtc.net/llama-farm/apis/llamafarm-edge-runtime/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/llama-farm/llamafarm-edge-runtime/revisions/a23a9e72958b/schema)