---
title: "Deploy Llm Presets"
method: GET
path: "/deploy/llm/presets"
tags: ["Dedicated Models"]
---

# Deploy Llm Presets

`GET /deploy/llm/presets`

DeepInfra presets and mirrored vLLM recipes for ``hf_repo_id``, told apart by
``source``; empty when none. Filter by ``gpu``/``engine``/``source``.

## Query parameters

- `hf_repo_id` string, required
- `gpu` 'L4-24GB' | 'L40S-48GB' | 'A100-80GB' | 'H100-80GB' | 'H200-141GB' | 'B200-180GB' | 'B300-270GB' | 'RTXPRO6000-96GB' | 'other'
- `engine` string, nullable
- `source` string, nullable

## Headers

- `xi-api-key` string, nullable
- `x-api-key` string, nullable

## Response `200`

Successful Response

- PresetConfigOut[]
  - `id` string, required — Preset id.
  - `source` string — Config source.
  - `engine` string — Inference engine.
  - `gpu_configs` string[], required — Allowed Nx<GPU> configs.
  - `standard_args` object — Engine tuning knobs.
    - `max_context_size` integer, nullable — Maximum total sequence length (prompt + generation).
    - `max_concurrent_requests` integer, nullable — Max number of requests served concurrently.
    - `gpu_memory_fraction` number, nullable — Fraction of GPU memory the engine may use for weights + KV cache.
    - `max_prefill_tokens` integer, nullable — Max tokens processed per prefill/engine step (chunked prefill size).
    - `kv_cache_dtype` 'auto' | 'fp8', nullable — KV cache precision. fp8 ~doubles KV capacity at <1% accuracy loss.
    - `enable_prefix_caching` boolean, nullable — Reuse KV cache for shared prompt prefixes.
    - `quantization` 'fp8' | 'awq' | 'gptq' | 'awq_marlin' | 'gptq_marlin' | 'compressed-tensors' | 'bitsandbytes', nullable — On-the-fly weight quantization method.
  - `extra_args` string[], nullable — Raw engine flags; vLLM recipes only.
  - `label` string — Short display name (e.g. "Throughput-optimized").

## Other responses

- `422` — Unprocessable Entity

---

[API](https://skmtc.net/deepinfra/apis/deepinfra-api.md) · [All operations](https://skmtc.net/deepinfra/apis/deepinfra-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/deepinfra/deepinfra-api/versions/2df28210837a/schema)