---
title: "Deploy Create Llm"
method: POST
path: "/deploy/llm"
tags: ["Dedicated Models"]
---

# Deploy Create Llm

`POST /deploy/llm`

## Headers

- `xi-api-key` string, nullable
- `x-api-key` string, nullable

## Request body

- DeployLLMIn
  - `model_name` string, required — model name for deepinfra (username/mode-name format)
  - `gpu` 'L4-24GB' | 'L40S-48GB' | 'A100-80GB' | 'H100-80GB' | 'H200-141GB' | 'B200-180GB' | 'B300-270GB' | 'RTXPRO6000-96GB' | 'other', required
  - `num_gpus` integer — Number of GPUs used by one instance
  - `max_batch_size` integer — Maximum number of concurrent requests
  - `hf` HFWeights
    - `repo` string, required — huggingface repository i.e username/reponame
    - `revision` string, nullable — commit sha or branch name
    - `token` string, nullable — huggingface access token with read access to the repo
  - `base_model` string, nullable — Base public model
  - `container_image` string, nullable — Docker image for the deployment (e.g. vllm/vllm-openai:v0.8.4)
  - `settings` ScaleSettings
    - `min_instances` integer — Minimum number of model instances to run
    - `max_instances` integer — Maximum number of model instances to run
  - `extra_args` string[], nullable — Extra command line arguments for custom deployments
  - `standard_args` StandardArgs — The standard-argument allowlist. Construct from a raw ``dict[str, str]`` via :meth:`from_raw`; render to engine flags via :func:`standard_parts`.
    - `max_context_size` integer, nullable — Maximum total sequence length (prompt + generation).
    - `max_concurrent_requests` integer, nullable — Max number of requests served concurrently.
    - `gpu_memory_fraction` number, nullable — Fraction of GPU memory the engine may use for weights + KV cache.
    - `max_prefill_tokens` integer, nullable — Max tokens processed per prefill/engine step (chunked prefill size).
    - `kv_cache_dtype` 'auto' | 'fp8', nullable — KV cache precision. fp8 ~doubles KV capacity at <1% accuracy loss.
    - `enable_prefix_caching` boolean, nullable — Reuse KV cache for shared prompt prefixes.
    - `quantization` 'fp8' | 'awq' | 'gptq' | 'awq_marlin' | 'gptq_marlin' | 'compressed-tensors' | 'bitsandbytes', nullable — On-the-fly weight quantization method.
  - `preset_id` string, nullable — Apply a stored preset or vLLM recipe by id. A non-empty standard_args or extra_args in this request replaces the preset's whole matching field.

## Response `200`

Successful Response

- DeploymentOut
  - `type` 'legacy' | 'llm' | 'lora' | 'tts'
  - `deploy_id` string, required — Deploy Id
  - `model_name` string, required — Model Id from huggingface
  - `version` string, required — Model version
  - `task` string, required — Task
  - `status` string, required — Status
  - `fail_reason` string, required — Failure reason
  - `created_at` string, required — Created at
  - `updated_at` string, required — Updated at
  - `instances` DeployInstances
    - `running` integer, required
    - `pending` integer, required
  - `config` DeployLLMConfig
    - `gpu` 'L4-24GB' | 'L40S-48GB' | 'A100-80GB' | 'H100-80GB' | 'H200-141GB' | 'B200-180GB' | 'B300-270GB' | 'RTXPRO6000-96GB' | 'other', required
    - `num_gpus` integer, required — Number of GPUs used by one instance
    - `max_batch_size` integer, required — Maximum number of concurrent requests
    - `weights` HFWeights
      - `repo` string, required — huggingface repository i.e username/reponame
      - `revision` string, nullable — commit sha or branch name
      - `token` string, nullable — huggingface access token with read access to the repo
  - `settings` ScaleSettings
    - `min_instances` integer — Minimum number of model instances to run
    - `max_instances` integer — Maximum number of model instances to run
  - `standard_args` object — Current engine tuning knobs
    - `max_context_size` integer, nullable — Maximum total sequence length (prompt + generation).
    - `max_concurrent_requests` integer, nullable — Max number of requests served concurrently.
    - `gpu_memory_fraction` number, nullable — Fraction of GPU memory the engine may use for weights + KV cache.
    - `max_prefill_tokens` integer, nullable — Max tokens processed per prefill/engine step (chunked prefill size).
    - `kv_cache_dtype` 'auto' | 'fp8', nullable — KV cache precision. fp8 ~doubles KV capacity at <1% accuracy loss.
    - `enable_prefix_caching` boolean, nullable — Reuse KV cache for shared prompt prefixes.
    - `quantization` 'fp8' | 'awq' | 'gptq' | 'awq_marlin' | 'gptq_marlin' | 'compressed-tensors' | 'bitsandbytes', nullable — On-the-fly weight quantization method.
  - `extra_args` string[], nullable — Current extra engine-specific command-line args (custom-weight deploys only)
  - `update_error` string, nullable — If the last config update was auto-reverted after an engine crash-loop, the error that caused it (status stays 'running')
  - `rollout` DeployRollout
    - `up_to_date` integer, required — Instances serving the current config.
    - `outdated` integer, required — Instances still serving a previous config.
    - `booting` integer, required — Instances starting up.

## Other responses

- `402` — Payment Required
- `404` — Not Found
- `409` — Conflict
- `422` — Unprocessable Entity

---

[API](https://skmtc.net/deepinfra/apis/deepinfra-api.md) · [All operations](https://skmtc.net/deepinfra/apis/deepinfra-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/deepinfra/deepinfra-api/versions/51b817b3f80f/schema)