Sign in

v36

latestOpenAPI 3.1.0raw.githubusercontent.com2025-10-091882241.2 MB
models
llms

List Llm Models

List available LLM models using the asynchronous implementation for improved performance

get/v1/models/

Query parameters

provider_categoryProviderCategory[] nullable
provider_namestring nullable
provider_type'anthropic' | 'azure' | 'bedrock' | 'cerebras' | 'deepseek' | 'google_ai' | 'google_vertex' | 'groq' | 'hugging-face' | 'letta' | 'lmstudio_openai' | 'mistral' | 'ollama' | 'openai' | 'together' | 'vllm' | 'xai'

Response

Successful Response

modelstring required

LLM model name.

model_endpoint_type'openai' | 'anthropic' | 'google_ai' | 'google_vertex' | 'azure' | 'groq' | 'ollama' | 'webui' | 'webui-legacy' | 'lmstudio' | 'lmstudio-legacy' | 'lmstudio-chatcompletions' | 'llamacpp' | 'koboldcpp' | 'vllm' | 'hugging-face' | 'mistral' | 'together' | 'bedrock' | 'deepseek' | 'xai' required

The endpoint type for the model.

model_endpointstring nullable

The endpoint for the model.

provider_namestring nullable

The provider name for the model.

provider_category'base' | 'byok'
model_wrapperstring nullable

The wrapper for the model.

context_windowinteger required

The context window size for the model.

put_inner_thoughts_in_kwargsboolean nullable

Puts 'inner_thoughts' as a kwarg in the function call if this is set to True. This helps with function calling performance and also the generation of inner thoughts.

handlestring nullable

The handle for this config, in the format provider/model-name.

temperaturenumber

The temperature to use when generating text with the model. A higher temperature will result in more random text.

max_tokensinteger nullable

The maximum number of tokens to generate. If not set, the model will use its default value.

enable_reasonerboolean

Whether or not the model should use extended thinking if it is a 'reasoning' style model

reasoning_effort'minimal' | 'low' | 'medium' | 'high' nullable

The reasoning effort to use when generating text reasoning models

max_reasoning_tokensinteger

Configurable thinking budget for extended thinking. Used for enable_reasoner and also for Google Vertex models like Gemini 2.5 Flash. Minimum value is 1024 when used with enable_reasoner.

frequency_penaltynumber nullable

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. From OpenAI: Number between -2.0 and 2.0.

compatibility_type'gguf' | 'mlx' nullable

The framework compatibility type for the model.

verbosity'low' | 'medium' | 'high' nullable

Soft control for how verbose model output should be, used for GPT-5 models.

tierstring nullable

The cost tier for the model (cloud only).