Sign in

v2

OpenAPI 3.0.02026-08-051996591.2 MB
Large Language Models

Update an LLM

Updates an existing LLM's configuration. This endpoint supports partial updates. Provide only the fields you want to change. Only the name field is immutable.

The platform tests the updated LLM before saving it to ensure the credentials are valid.

Updatable fields:

  • description - LLM description
  • type - LLM type (openai-compatible, vertex-ai, etc.)
  • model - Model identifier
  • uri - API endpoint
  • auth - Authentication credentials (including service account key_json)
  • headers - Additional HTTP headers (for openai-compatible and anthropic types)
  • enabled - Whether the LLM is enabled
  • capabilities - Model capabilities (image support, context limit, tool calling)

Immutable fields:

  • id - System-generated identifier
  • name - LLM name

Built-in LLMs (system-provided models) cannot be updated.

patch/v2/llms/{llm_id}

Path parameters

llm_idstring required

The ID of the LLM to update.

Headers

Request-Timeoutinteger

The platform makes a best effort to complete the request in the specified seconds, or it times out.

Request-Timeout-Millisinteger

The platform makes a best effort to complete the request in the specified milliseconds, or it times out.

Request body

OR
OR
OR

Example request

{
  "auth": {
    "type": "bearer",
    "token": "abcdef......"
  },
  "idle_timeout_seconds": 300,
  "test_model_parameters": {
    "max_tokens": 512
  }
}

Response

The updated LLM.

idstring required

The ID of the LLM.

namestring required

Name of the LLM.

descriptionstring

The description of the LLM.

enabledboolean

Indicates whether the LLM is enabled.

defaultboolean

Whether this is the default LLM. Queries use the default LLM when they do not specify a generator.

ownership'platform' | 'customer'

Indicates whether the LLM is provided by the platform or created by the customer. Platform LLMs are pre-configured and cannot be modified or deleted. Customer LLMs are created and managed by the customer.

type'openai-compatible' | 'openai-responses' | 'vertex-ai' | 'anthropic' required

The provider type. It determines which authentication and configuration apply. Responses always include this field. If a stored LLM cannot be mapped to one of these values, the request fails with HTTP 500.

modelstring

Provider-specific model identifier (e.g. gpt-4o, claude-3-5-sonnet-20241022, gemini-2.5-flash).

uristring uri

The API endpoint URI configured for this LLM.

headersobject

Additional HTTP headers configured for requests to the LLM API. Not applicable to vertex-ai.

idle_timeout_secondsinteger nullable

The maximum time in seconds that the platform waits for the model to send data before it closes the stale connection. During streaming, this is the SSE idle timeout. If no new server-sent events arrive within this window, the stream closes with an error. For non-streaming requests, where the model sends the entire response at once, this is the maximum time to wait for that response. If unset, the platform uses its default read timeout for the provider. On update, omit the field to keep the configured value, or send an explicit null to clear it.

requests_per_secondinteger

The maximum number of requests per second for this LLM. The platform omits this field when the LLM has no limit.

Example response

{
  "id": "llm_1021844",
  "name": "Claude 3.7 Sonnet",
  "description": "The Anthropic Claude 3.7 Sonnet model.",
  "ownership": "platform",
  "idle_timeout_seconds": 300,
  "auth": {
    "type": "bearer",
    "token": "abcdef......"
  }
}