---
title: "Update an LLM"
method: PATCH
path: "/v2/llms/{llm_id}"
tags: ["Large Language Models"]
---

# Update an LLM

`PATCH /v2/llms/{llm_id}`

Updates an existing LLM's configuration. This endpoint supports partial updates. Provide only the fields you want to change. Only the name field is immutable.

The platform tests the updated LLM before saving it to ensure the credentials are valid.

**Updatable fields:**
- `description` - LLM description
- `type` - LLM type (openai-compatible, vertex-ai, etc.)
- `model` - Model identifier
- `uri` - API endpoint
- `auth` - Authentication credentials (including service account key_json)
- `headers` - Additional HTTP headers (for openai-compatible and anthropic types)
- `enabled` - Whether the LLM is enabled
- `capabilities` - Model capabilities (image support, context limit, tool calling)

**Immutable fields:**
- `id` - System-generated identifier
- `name` - LLM name

Built-in LLMs (system-provided models) cannot be updated.

## Path parameters

- `llm_id` string, required

## Headers

- `Request-Timeout` integer
- `Request-Timeout-Millis` integer

## Request body

- union — Request to update a Large Language Model connection. All fields are optional. Provide only the fields you want to change. Only the name field is immutable.
  - object — Common updatable fields for OpenAI-compatible and Responses API LLM requests.
    - `model` string — The model identifier to use for this LLM.
    - `uri` string, uri — The API endpoint for this LLM.
    - `description` string — Description of the LLM.
    - `auth` union — Authentication configuration for connecting to a remote service.
      - object — Bearer token authentication
        - `type` string, required — Must be "bearer" for bearer token auth
        - `token` string, required — The bearer token to use for authentication
      - object — Custom header-based authentication
        - `type` string, required — Must be "header" for header-based auth
        - `header` string, required — The header name to use (e.g. x-api-key)
        - `value` string, required — The header value to use
      - object — OAuth 2.0 client credentials authentication. The platform acquires an access token from the token endpoint before connecting to the remote service.
        - `type` string, required — Must be "oauth_client_credentials" for OAuth client credentials auth.
        - `client_id` string, required — The OAuth2 client ID.
        - `client_secret` string, required — The OAuth2 client secret.
        - `token_endpoint` string, uri, required — The OAuth2 token endpoint URL where the platform exchanges credentials for an access token.
        - `scopes` string[] — OAuth2 scopes to request when acquiring the access token.
    - `headers` object — Additional HTTP headers to include with requests to the LLM API.
    - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
    - `enabled` boolean — Whether the LLM is enabled.
    - `test_model_parameters` object — Any additional parameters that are required for the LLM during the test call.
    - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
      - `image_support` boolean — Whether the model supports image inputs.
      - `context_limit` integer — Maximum context window size in tokens.
      - `tool_calling` boolean — Whether the model supports tool/function calling.
      - `structured_outputs` boolean — Whether the model supports structured output generation.
      - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
    - `requests_per_second` integer, nullable — The maximum number of requests per second for this LLM. - Set a value to apply that limit. - Set the field to null to remove the limit. - Omit the field to keep the current limit. The platform rejects requests above the limit with HTTP 429.
    - `type` string — Must be "openai-compatible" for OpenAI and OpenAI-compatible APIs
  - object — Common updatable fields for OpenAI-compatible and Responses API LLM requests.
    - `model` string — The model identifier to use for this LLM.
    - `uri` string, uri — The API endpoint for this LLM.
    - `description` string — Description of the LLM.
    - `auth` union — Authentication configuration for connecting to a remote service.
      - object — Bearer token authentication
        - `type` string, required — Must be "bearer" for bearer token auth
        - `token` string, required — The bearer token to use for authentication
      - object — Custom header-based authentication
        - `type` string, required — Must be "header" for header-based auth
        - `header` string, required — The header name to use (e.g. x-api-key)
        - `value` string, required — The header value to use
      - object — OAuth 2.0 client credentials authentication. The platform acquires an access token from the token endpoint before connecting to the remote service.
        - `type` string, required — Must be "oauth_client_credentials" for OAuth client credentials auth.
        - `client_id` string, required — The OAuth2 client ID.
        - `client_secret` string, required — The OAuth2 client secret.
        - `token_endpoint` string, uri, required — The OAuth2 token endpoint URL where the platform exchanges credentials for an access token.
        - `scopes` string[] — OAuth2 scopes to request when acquiring the access token.
    - `headers` object — Additional HTTP headers to include with requests to the LLM API.
    - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
    - `enabled` boolean — Whether the LLM is enabled.
    - `test_model_parameters` object — Any additional parameters that are required for the LLM during the test call.
    - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
      - `image_support` boolean — Whether the model supports image inputs.
      - `context_limit` integer — Maximum context window size in tokens.
      - `tool_calling` boolean — Whether the model supports tool/function calling.
      - `structured_outputs` boolean — Whether the model supports structured output generation.
      - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
    - `requests_per_second` integer, nullable — The maximum number of requests per second for this LLM. - Set a value to apply that limit. - Set the field to null to remove the limit. - Omit the field to keep the current limit. The platform rejects requests above the limit with HTTP 429.
    - `type` string — Must be "openai-responses" for OpenAI Responses API
  - object — Request to update a Vertex AI Large Language Model connection. All fields are optional.
    - `type` string — Must be "vertex-ai" for Google Cloud Vertex AI Gemini models
    - `model` string — The model identifier to use for this LLM.
    - `uri` string, uri — The base URI for the Gemini API. Accepts Vertex AI or Google AI Studio URIs in any format. See the create endpoint for full details and examples.
    - `description` string — Description of the LLM.
    - `auth` union — Authentication configuration for Vertex AI
      - object — API key authentication for Vertex AI
        - `type` string, required — Must be "api_key" for API key authentication
        - `api_key` string, required — The Google Cloud API key for authentication
      - object — Service account authentication for Vertex AI
        - `type` string, required — Must be "service_account" for service account authentication
        - `key_json` string, required — The service account JSON key file contents as a string
    - `headers` object — Additional HTTP headers to include with requests to the Gemini API.
    - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
    - `enabled` boolean — Whether the LLM is enabled.
    - `test_model_parameters` object — Any additional parameters that are required for the LLM during the test call.
    - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
      - `image_support` boolean — Whether the model supports image inputs.
      - `context_limit` integer — Maximum context window size in tokens.
      - `tool_calling` boolean — Whether the model supports tool/function calling.
      - `structured_outputs` boolean — Whether the model supports structured output generation.
      - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
    - `requests_per_second` integer, nullable — The maximum number of requests per second for this LLM. - Set a value to apply that limit. - Set the field to null to remove the limit. - Omit the field to keep the current limit. The platform rejects requests above the limit with HTTP 429.
  - object — Request to update an Anthropic Large Language Model connection. All fields are optional.
    - `type` string — Must be "anthropic" for Anthropic Claude models
    - `model` string — The model identifier to use for this LLM.
    - `uri` string, uri — The API endpoint for this LLM.
    - `description` string — Description of the LLM.
    - `auth` union — Authentication configuration for Anthropic LLM via direct Anthropic API (use RemoteAuth with header "x-api-key"), AWS Bedrock, or GCP Vertex AI Model Garden
      - object — Bearer token authentication
        - `type` string, required — Must be "bearer" for bearer token auth
        - `token` string, required — The bearer token to use for authentication
      - object — Custom header-based authentication
        - `type` string, required — Must be "header" for header-based auth
        - `header` string, required — The header name to use (e.g. x-api-key)
        - `value` string, required — The header value to use
      - object — AWS Bedrock authentication with explicit IAM credentials
        - `type` string, required — Must be "bedrock_static_iam" for static AWS IAM credentials
        - `aws_access_key_id` string, required — AWS access key ID
        - `aws_secret_access_key` string, required — AWS secret access key
        - `region` string, required — AWS region for Bedrock
      - object — AWS Bedrock API key authentication
        - `type` string, required — Must be "bedrock_api_key" for Bedrock API key auth
        - `api_key` string, required — The Bedrock API key
        - `region` string, required — AWS region for Bedrock
      - object — Google Cloud Vertex AI service account authentication
        - `type` string, required — Must be "vertex_service_account" for service account auth
        - `key_json` string, required — The service account key JSON
        - `project` string, required — GCP project ID
        - `region` string, required — GCP region for Vertex AI
      - object — Google Cloud Vertex AI access token authentication
        - `type` string, required — Must be "vertex_access_token" for access token auth
        - `access_token` string, required — The GCP access token
        - `project` string, required — GCP project ID
        - `region` string, required — GCP region for Vertex AI
    - `headers` object — Optional additional headers to send with the request
    - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
    - `enabled` boolean — Whether the LLM is enabled.
    - `test_model_parameters` object — Any additional parameters that are required for the LLM during the test call.
    - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
      - `image_support` boolean — Whether the model supports image inputs.
      - `context_limit` integer — Maximum context window size in tokens.
      - `tool_calling` boolean — Whether the model supports tool/function calling.
      - `structured_outputs` boolean — Whether the model supports structured output generation.
      - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
    - `requests_per_second` integer, nullable — The maximum number of requests per second for this LLM. - Set a value to apply that limit. - Set the field to null to remove the limit. - Omit the field to keep the current limit. The platform rejects requests above the limit with HTTP 429.

## Response `200`

The updated LLM.

- LLM — A Large Language Model. An LLM enhances query results with a generated response and acts as the responder during a chat.
  - `id` string, required — The ID of the LLM.
  - `name` string, required — Name of the LLM.
  - `description` string — The description of the LLM.
  - `enabled` boolean — Indicates whether the LLM is enabled.
  - `default` boolean — Whether this is the default LLM. Queries use the default LLM when they do not specify a generator.
  - `capabilities` LLMCapabilities — The capabilities of a Large Language Model. If you do not provide capabilities when you create an LLM, the platform infers them from the model name and provider type. Fields you provide explicitly override the inferred defaults.
    - `image_support` boolean — Whether the model supports image inputs.
    - `context_limit` integer — Maximum context window size in tokens.
    - `tool_calling` boolean — Whether the model supports tool/function calling.
    - `structured_outputs` boolean — Whether the model supports structured output generation.
    - `requires_role_alternation` boolean — Whether the model requires strict role alternation in conversations. When true, the platform groups consecutive messages of the same role together.
  - `ownership` 'platform' | 'customer' — Indicates whether the LLM is provided by the platform or created by the customer. Platform LLMs are pre-configured and cannot be modified or deleted. Customer LLMs are created and managed by the customer.
  - `type` 'openai-compatible' | 'openai-responses' | 'vertex-ai' | 'anthropic', required — The provider type. It determines which authentication and configuration apply. Responses always include this field. If a stored LLM cannot be mapped to one of these values, the request fails with HTTP 500.
  - `model` string — Provider-specific model identifier (e.g. `gpt-4o`, `claude-3-5-sonnet-20241022`, `gemini-2.5-flash`).
  - `uri` string, uri — The API endpoint URI configured for this LLM.
  - `headers` object — Additional HTTP headers configured for requests to the LLM API. Not applicable to `vertex-ai`.
  - `idle_timeout_seconds` integer, nullable — The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.
  - `requests_per_second` integer — The maximum number of requests per second for this LLM. The platform omits this field when the LLM has no limit.
  - `auth` union — The authentication configuration for an LLM. It is a union over every variant any LLM provider accepts. In `GET` responses, secret fields contain the literal string `****`.
    - object — Bearer token authentication
      - `type` string, required — Must be "bearer" for bearer token auth
      - `token` string, required — The bearer token to use for authentication
    - object — Custom header-based authentication
      - `type` string, required — Must be "header" for header-based auth
      - `header` string, required — The header name to use (e.g. x-api-key)
      - `value` string, required — The header value to use
    - object — OAuth 2.0 client credentials authentication. The platform acquires an access token from the token endpoint before connecting to the remote service.
      - `type` string, required — Must be "oauth_client_credentials" for OAuth client credentials auth.
      - `client_id` string, required — The OAuth2 client ID.
      - `client_secret` string, required — The OAuth2 client secret.
      - `token_endpoint` string, uri, required — The OAuth2 token endpoint URL where the platform exchanges credentials for an access token.
      - `scopes` string[] — OAuth2 scopes to request when acquiring the access token.
    - object — AWS Bedrock authentication with explicit IAM credentials
      - `type` string, required — Must be "bedrock_static_iam" for static AWS IAM credentials
      - `aws_access_key_id` string, required — AWS access key ID
      - `aws_secret_access_key` string, required — AWS secret access key
      - `region` string, required — AWS region for Bedrock
    - object — AWS Bedrock API key authentication
      - `type` string, required — Must be "bedrock_api_key" for Bedrock API key auth
      - `api_key` string, required — The Bedrock API key
      - `region` string, required — AWS region for Bedrock
    - object — Google Cloud Vertex AI service account authentication
      - `type` string, required — Must be "vertex_service_account" for service account auth
      - `key_json` string, required — The service account key JSON
      - `project` string, required — GCP project ID
      - `region` string, required — GCP region for Vertex AI
    - object — Google Cloud Vertex AI access token authentication
      - `type` string, required — Must be "vertex_access_token" for access token auth
      - `access_token` string, required — The GCP access token
      - `project` string, required — GCP project ID
      - `region` string, required — GCP region for Vertex AI
    - object — API key authentication for Vertex AI
      - `type` string, required — Must be "api_key" for API key authentication
      - `api_key` string, required — The Google Cloud API key for authentication
    - object — Service account authentication for Vertex AI
      - `type` string, required — Must be "service_account" for service account authentication
      - `key_json` string, required — The service account JSON key file contents as a string
  - `prompts` Prompt[] — List of prompts that the model can use. This is deprecated; see `/v2/generation_presets` instead.
    - `id` string, required — The ID of the prompt.
    - `name` string, required — Name of the prompt. This is used as the `prompt_name` in a query.
    - `description` string — The description of the prompt.
    - `enabled` boolean — Indicates whether the prompt is enabled.
    - `default` boolean — Indicates if this prompt is the default prompt used with the LLM.

## Other responses

- `400` — Invalid request body or connection test failed
- `403` — Permissions do not allow updating this LLM or LLM is built-in
- `404` — LLM not found

---

[API](https://skmtc.net/vectara/apis/vectara-rest-api-v2.md) · [All operations](https://skmtc.net/vectara/apis/vectara-rest-api-v2/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/vectara/vectara-rest-api-v2/versions/e85040b266cc/schema)