latestOpenAPI 3.1.02026-08-16151182241.5 KB

ed76b41e8006

Internal

Drive a blue/green rollout (platform admin)

The rollout control verb, targeting the lane's green. pause freezes the ramp at its current share while leaving traffic alone; resume clears it. rollback marks the green's backend unhealthy, which removes it from routing entirely. cutover arms a supervised override of the capacity ceiling and the parity bar (readiness, the per-pass step limit and the two-pass separation are never overridden), requires a reason and a typed confirmation, and expires. finalize is valid once the lane has cut over: it drains the demoted blues and returns the green to an ordinary independent deployment, which also ends the instant-rollback window. Every action is recorded in the admin audit log with the actor and reason. All of these are rejected on a deployment that is not a rolling green, so they can never touch an independent deployment by accident.

post/v1/admin/inference/deployments/{id}/rollout

Path parameters

idstring required

Request body

action'pause' | 'resume' | 'rollback' | 'cutover' | 'finalize' required

pause freezes the ramp at its current share without touching traffic. resume clears the freeze. rollback marks the green's backend unhealthy, which is the one true rollback lever: weight 0 keeps a backend as the overflow tail and would route to the broken version exactly when the lane saturates. cutover arms a supervised force of the capacity gates. finalize drains the demoted blues and returns the lane to steady state.

confirmstring

Typed confirmation for the actions that cannot be undone cheaply (cutover, finalize): must equal the deployment id.

reasonstring

Why. Required for cutover, recorded in the admin audit log with the actor for every action.

Response

The refreshed green

backendCatalogWeightinteger

Live catalog routing weight of this deployment's backend. Only present when backendRegistered is true. May differ from backendWeight (the row mirror) while a rollout ramp is in flight.

backendHealthyboolean

Live catalog health of this deployment's backend (the kill switch). Only present when backendRegistered is true.

backendRegisteredboolean

Whether this deployment's pool:// backend is currently present on the catalog model row. Absent when the live catalog state could not be read; false until the first replica passes readiness.

backendVersionstring

The rollout tag this deployment's backend reports to the gateway as the version metric label on ig_model_requests_total and ig_stream_ttft_seconds. Use it to separate the two sides of a rollout in a dashboard query. Only present when backendRegistered is true.

backendWeightinteger required

Routing weight of the catalog backend. 0 = standby (blue/green starts here)

changeCausestring

Operator-supplied reason for the revision that created this deployment

createdAtstring required

dtypestring

enginestring required

Inference engine, e.g. vllm or sglang

engineImagestring

Engine container image override; absent = the engine's default

envobject

extraArgsstring[]

generationinteger

1 for a hand-created deployment, parent+1 for each revision. History ordering only: the controller never reads it, because rollout adoption is the explicit rolloutRole opt-in and nothing else.

gpuCountPerReplicainteger required

gpuModelIdstring required

idstring required

isolatedboolean required

True when replicas and their private endpoint pool are for direct operator testing only. Isolated deployments never register a catalog backend, never participate in blue/green rollout, and cannot receive gateway traffic.

maxConcurrencyPerReplicainteger

maxModelLeninteger

maxPromptBytesinteger

Exclusive text-request byte ceiling for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

minPromptBytesinteger

Inclusive text-request byte floor for routing eligibility; 0 or absent = unbounded. Routing hint only, not a context limit.

nativeFamiliesstring[]

nodePoolstring

Stable logical capacity lane within the public model. Blue/green only replaces versions in the same node pool; empty preserves legacy model-wide grouping.

organizationIdstring required

Org that owns the managed replicas (and their metering)

parentDeploymentIdstring

The deployment this one was revised from. Set only by /revise, so an absent value means this deployment was created by hand.

poolIdstring

Endpoint pool the deployment's replicas register in

publicModelIdstring required

Catalog model this deployment serves capacity for

quantizationstring

readyReplicasinteger required

replicaRuntime'' | 'vm' | 'pod'

How each replica is placed. Absent or "" or "vm" = a QEMU/VFIO VM on the baremetal fleet (the legacy shape; every pre-existing deployment reads as this). "pod" = a hardened GPU pod on a provider-onboarded runc node. One deployment is one runtime; a public model mixes runtimes through lanes.

replicaVmIdsstring[]

replicasDesiredinteger required

replicasMaxinteger required

replicasMininteger required

rolloutPausedboolean

True while an operator has paused the ramp. Traffic is untouched: the deployment keeps the weight it holds and keeps serving, and the controller takes no rollout action until resume. The pause is also on rollout.paused; this raw column is what a surface falls back to when the lane could not be read at all.

rolloutRole'' | 'green'

Opt-in blue/green role. Empty (default) = an INDEPENDENT weighted capacity member of its lane, never blue or green. "green" = the lane's successor generation: the controller ramps it against the older members under a capacity guard and cuts them over once it reaches capacity parity. Rollouts are never implicit.

servedModelIdstring required

Model id the engine serves; must equal the catalog backend ref

statusReasonstring

Human-readable cause for degraded/failed states

updatedAtstring required

versionstring

Generation marker for blue/green within one public model and node pool

weightsRefstring

Staged-checkpoint reference; absent = the engine loads from its own source