Sign in

v2

latestOpenAPI 3.1.02026-08-07139158192.4 KB
Internal

Set the deployment backend's health (the emergency kill switch)

The emergency control. healthy=false takes this deployment's backend out of routing entirely; the gateway excludes unhealthy backends from the live set. This is categorically stronger than weight 0, which keeps the backend as the standby/overflow tail and still routes to it exactly when the weighted backends are saturated or failing. healthy=true clears the kill. The setting survives restarts and reconcile passes until cleared.

post/v1/admin/inference/deployments/{id}/health

Path parameters

idstring required

Request body

healthyboolean required

false = emergency kill (backend excluded from routing entirely); true = clear the kill

Response

Updated

backendCatalogWeightinteger

Live catalog routing weight of this deployment's backend. Only present when backendRegistered is true. May differ from backendWeight (the row mirror) while a rollout ramp is in flight.

backendHealthyboolean

Live catalog health of this deployment's backend (the kill switch). Only present when backendRegistered is true.

backendRegisteredboolean

Whether this deployment's pool:// backend is currently present on the catalog model row. Absent when the live catalog state could not be read; false until the first replica passes readiness.

backendWeightinteger required

Routing weight of the catalog backend. 0 = standby (blue/green starts here)

createdAtstring required
dtypestring
enginestring required

Inference engine, e.g. vllm or sglang

engineImagestring

Engine container image override; absent = the engine's default

envobject
extraArgsstring[]
gpuCountPerReplicainteger required
gpuModelIdstring required
idstring required
isolatedboolean required

True when replicas and their private endpoint pool are for direct operator testing only. Isolated deployments never register a catalog backend, never participate in blue/green rollout, and cannot receive gateway traffic.

maxConcurrencyPerReplicainteger
maxModelLeninteger
maxPromptBytesinteger

Exclusive text-request byte ceiling for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

minPromptBytesinteger

Inclusive text-request byte floor for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

nativeFamiliesstring[]
nodePoolstring

Stable logical capacity lane within the public model. Blue/green only replaces versions in the same node pool; empty preserves legacy model-wide grouping.

organizationIdstring required

Org that owns the managed replicas (and their metering)

poolIdstring

Endpoint pool the deployment's replicas register in

publicModelIdstring required

Catalog model this deployment serves capacity for

quantizationstring
readyReplicasinteger required
replicaVmIdsstring[]
replicasDesiredinteger required
replicasMaxinteger required
replicasMininteger required
servedModelIdstring required

Model id the engine serves; must equal the catalog backend ref

status'pending' | 'deploying' | 'ready' | 'degraded' | 'scaling' | 'draining' | 'stopped' | 'failed' required
statusReasonstring

Human-readable cause for degraded/failed states

updatedAtstring required
versionstring

Generation marker for blue/green within one public model and node pool

weightsRefstring

Staged-checkpoint reference; absent = the engine loads from its own source