---
title: "List LLMs"
method: GET
path: "/v2/llms"
tags: ["Large Language Models"]
---

# List LLMs

`GET /v2/llms`

Lists LLMs that you can use with query and chat endpoints. A query does not specify the LLM directly. Instead, it uses a `generation_preset_name`. The `generation_preset_name` in generation parameters corresponds to the `name` property of the generation presets that `/v2/generation_presets` lists.

## Query parameters

- `filter` string
- `limit` integer
- `page_key` string

## Headers

- `Request-Timeout` integer
- `Request-Timeout-Millis` integer

## Response `200`

List of LLMs.

- ListLLMsResponse — Response containing a list of Large Language Models available for generative AI capabilities.
  - `llms` LLM[] — List of LLMs.
    - `id` string, required — The ID of the LLM.
    - `name` string, required — Name of the LLM.
    - `description` string — The description of the LLM.
    - `enabled` boolean — Indicates whether the LLM is enabled.
    - `default` boolean — Whether this is the default LLM. Queries use the default LLM when they do not specify a generator.
    - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
      - `image_support` boolean — Whether the model supports image inputs.
      - `context_limit` integer — Maximum context window size in tokens.
      - `tool_calling` boolean — Whether the model supports tool/function calling.
      - `structured_outputs` boolean — Whether the model supports structured output generation.
      - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
    - `ownership` 'platform' | 'customer' — Indicates whether the LLM is provided by the platform or created by the customer. Platform LLMs are pre-configured and cannot be modified or deleted. Customer LLMs are created and managed by the customer.
    - `type` 'openai-compatible' | 'openai-responses' | 'vertex-ai' | 'anthropic', required — The provider type. It determines which authentication and configuration apply. Responses always include this field. If a stored LLM cannot be mapped to one of these values, the request fails with HTTP 500.
    - `model` string — Provider-specific model identifier (e.g. `gpt-4o`, `claude-3-5-sonnet-20241022`, `gemini-2.5-flash`).
    - `uri` string, uri — The API endpoint URI configured for this LLM.
    - `headers` object — Additional HTTP headers configured for requests to the LLM API. Not applicable to `vertex-ai`.
    - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
    - `requests_per_second` integer — The maximum number of requests per second for this LLM. The platform omits this field when the LLM has no limit.
    - `auth` union — The authentication configuration for an LLM. It is a union over every variant any LLM provider accepts. In `GET` responses, secret fields contain the literal string `****`.
      - object — Bearer token authentication
        - `type` string, required — Must be "bearer" for bearer token auth
        - `token` string, required — The bearer token to use for authentication
      - object — Custom header-based authentication
        - `type` string, required — Must be "header" for header-based auth
        - `header` string, required — The header name to use (e.g. x-api-key)
        - `value` string, required — The header value to use
      - object — OAuth 2.0 client credentials authentication. The platform acquires an access token from the token endpoint before connecting to the remote service.
        - `type` string, required — Must be "oauth_client_credentials" for OAuth client credentials auth.
        - `client_id` string, required — The OAuth2 client ID.
        - `client_secret` string, required — The OAuth2 client secret.
        - `token_endpoint` string, uri, required — The OAuth2 token endpoint URL where the platform exchanges credentials for an access token.
        - `scopes` string[] — OAuth2 scopes to request when acquiring the access token.
      - object — AWS Bedrock authentication with explicit IAM credentials
        - `type` string, required — Must be "bedrock_static_iam" for static AWS IAM credentials
        - `aws_access_key_id` string, required — AWS access key ID
        - `aws_secret_access_key` string, required — AWS secret access key
        - `region` string, required — AWS region for Bedrock
      - object — AWS Bedrock API key authentication
        - `type` string, required — Must be "bedrock_api_key" for Bedrock API key auth
        - `api_key` string, required — The Bedrock API key
        - `region` string, required — AWS region for Bedrock
      - object — Google Cloud Vertex AI service account authentication
        - `type` string, required — Must be "vertex_service_account" for service account auth
        - `key_json` string, required — The service account key JSON
        - `project` string, required — GCP project ID
        - `region` string, required — GCP region for Vertex AI
      - object — Google Cloud Vertex AI access token authentication
        - `type` string, required — Must be "vertex_access_token" for access token auth
        - `access_token` string, required — The GCP access token
        - `project` string, required — GCP project ID
        - `region` string, required — GCP region for Vertex AI
      - object — API key authentication for Vertex AI
        - `type` string, required — Must be "api_key" for API key authentication
        - `api_key` string, required — The Google Cloud API key for authentication
      - object — Service account authentication for Vertex AI
        - `type` string, required — Must be "service_account" for service account authentication
        - `key_json` string, required — The service account JSON key file contents as a string
    - `prompts` Prompt[] — List of prompts that the model can use. This is deprecated; see `/v2/generation_presets` instead.
      - `id` string, required — The ID of the prompt.
      - `name` string, required — Name of the prompt. This is used as the `prompt_name` in a query.
      - `description` string — The description of the prompt.
      - `enabled` boolean — Indicates whether the prompt is enabled.
      - `default` boolean — Indicates if this prompt is the default prompt used with the LLM.
  - `metadata` ListMetadata — The standard metadata in the response of a list operation.
    - `page_key` string — The page key for the next page of results. Pass it as a query parameter to request the next page.

## Other responses

- `403` — Permissions do not allow listing LLMs.

---

[API](https://skmtc.net/vectara/apis/vectara-rest-api-v2.md) · [All operations](https://skmtc.net/vectara/apis/vectara-rest-api-v2/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/vectara/vectara-rest-api-v2/revisions/a95087fe3a20/schema)