--- title: "Update a GPU cluster" method: PUT path: "/compute/clusters/{cluster_id}" tags: ["GPUClusterService"] --- # Update a GPU cluster `PUT /compute/clusters/{cluster_id}` Update the configuration of an existing GPU cluster. ## Path parameters - `cluster_id` string, required — The ID of the cluster to update ## Request body - GPUClusterUpdateRequest - `cluster_type` 'KUBERNETES' | 'SLURM' — Type of cluster to update. - `num_gpus` integer — Target GPU count for the cluster. When omitted, the server keeps the current GPU count from cluster metadata (use for config-only or decommission-time-only updates). - `reservation_end_time` string, date-time — Timestamp at which the cluster should be decommissioned. Only accepted for prepaid clusters. - `cluster_config` InstanceClusterConfig - `load_balancer` 'NONE' | 'TRAEFIK' | 'NGINX' | 'ISTIO', required - `kubernetes_dashboard_enabled` boolean - `jumphost_enabled` boolean - `slurm_startup_scripts` SlurmStartupScripts — SlurmStartupScripts carries optional Slurm lifecycle scripts (prolog/epilog, init, extra conf). - `worker_prolog` string — Slurm worker node prolog script. - `worker_epilog` string — Slurm worker node epilog script. - `controller_prolog` string — Slurm controller prolog script. - `controller_epilog` string — Slurm controller epilog script. - `login_init_script` string — Script run on Slurm login node init. - `nodeset_init_script` string — Script run on Slurm nodeset init. - `extra_slurm_conf` string — Additional slurm.conf fragments. - `ingress` ClusterIngressConfig - `enabled` boolean - `observability` ObservabilityConfig - `enabled` boolean - `gpu_operator_version` string — NVIDIA GPU Operator chart/version for the tenant cluster (e.g. v24.6.2). When omitted, a service default is applied. - `network_operator_version` string — NVIDIA Network Operator chart/version for the tenant cluster (e.g. v24.7.0). When omitted, a service default is applied. - `ssh_ca_enabled` boolean — Whether this cluster uses a per-cluster SSH certificate authority for OIDC-signed SSH access. - `num_reserved_gpus` integer — Number of reserved GPUs to update to. This field is only applicable for clusters with RESERVED billing type. - `num_preemptible_gpus` integer — Updated desired number of preemptible GPUs for the cluster. When omitted, the current value is preserved. Must be a multiple of 8. - `num_capacity_pool_gpus` integer — Number of GPUs to draw from the cluster's capacity pool. Only valid for clusters created with a capacity_pool_id. Must be a multiple of 8 and not exceed num_gpus. When omitted, the current value is preserved. - `add_ons` AddOnUpdateRequest[] — Add-ons to update on the cluster. Each entry identifies an existing add-on by name and provides the new external config to merge. - `name` string, required — Name of the add-on to update. Must match an existing add-on on the cluster. - `config` AddOnConfig — Configuration for a cluster add-on. - `dashboard` DashboardConfig - `enabled` boolean - `ingress` IngressConfig - `enabled` boolean - `torchpass` TorchpassConfig — Configuration for the Model Aware TorchPass cluster add-on. - `enabled` boolean — Whether to enable the Model Aware TorchPass add-on. - `slurm_web` SlurmWebConfig — Configuration for the Slurm Web cluster add-on. - `enabled` boolean — Whether to enable the Slurm Web add-on. - `headlamp` HeadlampConfig — Configuration for the Headlamp Kubernetes dashboard cluster add-on. - `enabled` boolean — Whether to enable the Headlamp Kubernetes dashboard add-on. ## Response `200` OK - GPUClusterInfo - `cluster_id` string, required - `cluster_type` 'KUBERNETES' | 'SLURM', required — Type of cluster. - `region` string, required - `gpu_type` 'H100_SXM' | 'H200_SXM' | 'RTX_6000_PCI' | 'L40_PCIE' | 'B200_SXM' | 'H100_SXM_INF' | 'B300_SXM', required - `cluster_name` string, required - `duration_hours` integer - `volumes` GPUClusterVolume[], required - `volume_id` string, required — ID of the volume. - `volume_name` string, required — User provided name of the volume. - `size_tib` integer, required — Size of the volume in TiB. - `status` string, required — Current status of the volume. - `status` 'WaitingForControlPlaneNodes' | 'WaitingForDataPlaneNodes' | 'WaitingForSubnet' | 'WaitingForSharedVolume' | 'InstallingDrivers' | 'RunningAcceptanceTests' | 'Paused' | 'OnDemandComputePaused' | 'Ready' | 'Degraded' | 'Deleting', required — Current status of the GPU cluster. - `control_plane_nodes` GPUClusterControlPlaneNode[], required - `node_id` string, required - `status` string, required - `host_name` string, required - `num_cpu_cores` integer, required - `memory_gib` number, required - `network` string, required - `phase_transitions` NodePhaseTransition[], required — Phase transition history for this control plane node. - `phase` 'NODE_PHASE_PENDING' | 'NODE_PHASE_SCHEDULING' | 'NODE_PHASE_BOOTING' | 'NODE_PHASE_BOOTSTRAPPING' | 'NODE_PHASE_RUNNING' | 'NODE_PHASE_SUCCEEDED' | 'NODE_PHASE_FAILED' | 'NODE_PHASE_PAUSED', required — Node phase. - `transition_time` string, date-time, required — Timestamp when the phase transition occurred. - `public_ipv4` string — Public IPv4 address of the control plane node. - `gpu_worker_nodes` GPUClusterGPUWorkerNode[], required - `node_id` string, required - `status` string, required - `host_name` string, required - `num_cpu_cores` integer, required - `num_gpus` integer, required - `memory_gib` number, required - `networks` string[], required - `instance_id` string - `latest_remediation` Remediation — Remediation represents a node remediation request for an instance. An instance can have multiple remediations over time (e.g., failed attempts followed by retries). - `id` string, required - `cluster_id` string, required - `instance_id` string, required - `mode` 'REMEDIATION_MODE_VM_ONLY' | 'REMEDIATION_MODE_HOST_AWARE' | 'REMEDIATION_MODE_EVICT_WITHOUT_REPLACEMENT' | 'REMEDIATION_MODE_REBOOT_VM' | 'REMEDIATION_MODE_HOST_POWER_CYCLE', required — Remediation mode specifies how the remediation should be performed. - `REMEDIATION_MODE_VM_ONLY`: Deletes the VM and provisions a new one on any available host. - `REMEDIATION_MODE_HOST_AWARE`: Cordons the host, deletes the VM, and provisions a new one on a different host. - `REMEDIATION_MODE_EVICT_WITHOUT_REPLACEMENT`: Evicts the VM without provisioning a replacement. - `REMEDIATION_MODE_REBOOT_VM`: Reboots the VM in place. - `REMEDIATION_MODE_HOST_POWER_CYCLE`: Cordons and power-cycles the bare-metal host while preserving host and node identity. - `trigger` 'REMEDIATION_TRIGGER_MANUAL' | 'REMEDIATION_TRIGGER_AUTOMATED', required — RemediationTrigger specifies how the remediation was triggered. - `REMEDIATION_TRIGGER_MANUAL`: A user-initiated remediation (either via web UI or API call). - `REMEDIATION_TRIGGER_AUTOMATED`: A system-initiated remediation that requires approval. - `state` 'PENDING_APPROVAL' | 'PENDING' | 'RUNNING' | 'SUCCEEDED' | 'FAILED' | 'CANCELLED' | 'AUTO_RESOLVED' | 'QUARANTINING' | 'QUARANTINED', required — RemediationState represents the lifecycle state of a remediation. - `PENDING_APPROVAL`: Awaiting approval before processing can begin. - `PENDING`: Approved and queued for processing. - `RUNNING`: Actively being processed. - `SUCCEEDED`: Successfully completed. - `FAILED`: Failed with an error. - `CANCELLED`: Cancelled by user or system. - `AUTO_RESOLVED`: The underlying issue was automatically resolved before processing. - `QUARANTINING`: Cordoning or preparing the host before remediation. - `QUARANTINED`: Host has been cordoned or isolated for remediation. - `reason` string — User-provided reason for the remediation. - `active_health_check_run_id` string — Active health check run ID (UUID) that triggered this remediation. - `passive_health_check_event_id` string — Passive health check event ID that triggered this remediation. - `requested_by` string — Who requested the remediation. - `create_time` string, date-time — When the remediation was created. - `reviewed_by` string — Who reviewed the remediation. - `review_time` string, date-time — When the remediation was reviewed. - `review_comment` string — Review comment. - `start_time` string, date-time — When processing started. - `end_time` string, date-time — When the remediation completed. - `error_message` string — Error message if the remediation failed. - `update_time` string, date-time — When the remediation was last updated. - `instance_name` string — Display name of the targeted instance. - `linked_alerts` PassiveHealthCheckAlert[] — Passive health check alerts linked to this remediation, including resolved alerts. - `passive_health_check_alert_id` string, required — Primary key UUID for the passive health check alert. - `instance_id` string — Resolved instance UUID. Empty until the alert is joined to an instance. - `cluster_id` string, required — Cluster UUID the alert was raised against. - `target_vm` string, required — VM name extracted from the Alertmanager labels. - `alert_name` string, required — Alertmanager alert name. - `severity` 'PHC_SEVERITY_INFO' | 'PHC_SEVERITY_WARNING' | 'PHC_SEVERITY_CRITICAL', required — Canonical severity tier for the alert. - `annotations` object, required — Alertmanager annotations as key-value strings. - `started_at` string, date-time, required — Time when the underlying alert first fired. - `resolved_at` string, date-time — Time when the underlying alert resolved. Empty while the alert is firing. - `node_remediation_intent_id` string — Remediation intent UUID attached to this alert, if any. - `slurm_worker_hostname` string - `phase_transitions` NodePhaseTransition[], required — Phase transition history for this GPU worker node. - `phase` 'NODE_PHASE_PENDING' | 'NODE_PHASE_SCHEDULING' | 'NODE_PHASE_BOOTING' | 'NODE_PHASE_BOOTSTRAPPING' | 'NODE_PHASE_RUNNING' | 'NODE_PHASE_SUCCEEDED' | 'NODE_PHASE_FAILED' | 'NODE_PHASE_PAUSED', required — Node phase. - `transition_time` string, date-time, required — Timestamp when the phase transition occurred. - `marked_for_deletion` boolean — Whether this node is marked for deletion by the operator. - `public_ipv4` string — Public IPv4 address of the GPU worker node. - `ib_hca_type` string — InfiniBand HCA type. - `ib_hca_count` integer — Number of InfiniBand HCAs. - `nvswitch_count` integer — Number of NVSwitches. - `nvswitch_type` string — NVSwitch type. - `ephemeral_storage` string — Ephemeral storage size, such as 1Ti. - `auto_remediation_enabled` boolean — Whether auto-remediation is enabled for this node's instance. - `deleted_at` string, date-time — Timestamp when the node left the live data plane. Only set for deleted_gpu_worker_nodes. - `kube_config` string, required - `num_gpus` integer, required - `slurm_shm_size_gib` integer - `capacity_pool_id` string - `reservation_start_time` string, date-time - `reservation_end_time` string, date-time - `install_traefik` boolean - `cuda_version` string, required - `nvidia_driver_version` string, required - `created_at` string, date-time - `oidc_config` OIDCConfig - `issuer_url` string, required — OIDC issuer URL for authentication. For example, https://accounts.google.com - `client_id` string, required — OIDC client ID for authentication. - `username_claim` string, required — JWT claim to use as the username. For example, 'sub' or 'email' - `username_prefix` string, required — Prefix to add to the username claim to form the final username. For example, 'oidc:' - `group_claim` string, required — JWT claim to use for user groups. For example, 'groups' - `group_prefix` string, required — Prefix to add to the group claim to form the final group name. For example, 'oidc:' - `ca_cert` string — CA certificate in PEM format to validate the OIDC issuer's TLS certificate. This field is optional but recommended if the issuer uses a private CA or self-signed certificate. - `project_id` string, required - `cluster_config` InstanceClusterConfig - `load_balancer` 'NONE' | 'TRAEFIK' | 'NGINX' | 'ISTIO', required - `kubernetes_dashboard_enabled` boolean - `jumphost_enabled` boolean - `slurm_startup_scripts` SlurmStartupScripts — SlurmStartupScripts carries optional Slurm lifecycle scripts (prolog/epilog, init, extra conf). - `worker_prolog` string — Slurm worker node prolog script. - `worker_epilog` string — Slurm worker node epilog script. - `controller_prolog` string — Slurm controller prolog script. - `controller_epilog` string — Slurm controller epilog script. - `login_init_script` string — Script run on Slurm login node init. - `nodeset_init_script` string — Script run on Slurm nodeset init. - `extra_slurm_conf` string — Additional slurm.conf fragments. - `ingress` ClusterIngressConfig - `enabled` boolean - `observability` ObservabilityConfig - `enabled` boolean - `gpu_operator_version` string — NVIDIA GPU Operator chart/version for the tenant cluster (e.g. v24.6.2). When omitted, a service default is applied. - `network_operator_version` string — NVIDIA Network Operator chart/version for the tenant cluster (e.g. v24.7.0). When omitted, a service default is applied. - `ssh_ca_enabled` boolean — Whether this cluster uses a per-cluster SSH certificate authority for OIDC-signed SSH access. - `num_cpu_workers` integer, required — Number of CPU-only worker nodes in the cluster. - `phase_transitions` ClusterPhaseTransition[], required — Cluster-level phase transition history. - `phase` 'CLUSTER_PHASE_QUEUED' | 'CLUSTER_PHASE_SCHEDULED' | 'CLUSTER_PHASE_WAITING_FOR_CONTROL_PLANE_NODES' | 'CLUSTER_PHASE_WAITING_FOR_DATA_PLANE_NODES' | 'CLUSTER_PHASE_WAITING_FOR_SUBNET' | 'CLUSTER_PHASE_WAITING_FOR_SHARED_VOLUME' | 'CLUSTER_PHASE_WAITING_FOR_AUTO_SCALER' | 'CLUSTER_PHASE_INSTALLING_DRIVERS' | 'CLUSTER_PHASE_RUNNING_ACCEPTANCE_TESTS' | 'CLUSTER_PHASE_ACCEPTANCE_TESTS_FAILED' | 'CLUSTER_PHASE_RUNNING_NCCL_TESTS' | 'CLUSTER_PHASE_NCCL_TESTS_FAILED' | 'CLUSTER_PHASE_READY' | 'CLUSTER_PHASE_PAUSED' | 'CLUSTER_PHASE_ON_DEMAND_COMPUTE_PAUSED' | 'CLUSTER_PHASE_DEGRADED' | 'CLUSTER_PHASE_DELETING', required — Cluster phase. - `transition_time` string, date-time, required — Timestamp when the phase transition occurred. - `desired_preemptible_gpus` integer, required — Customer's requested number of preemptible GPUs. Set on cluster create or update; persists until changed. - `allocated_preemptible_gpus` integer, required — Actual number of preemptible GPUs currently allocated to the cluster. Updated asynchronously by the fulfillment and reclamation workers; may be less than desired_preemptible_gpus when capacity is constrained. - `billing_type` 'RESERVED' | 'ON_DEMAND' | 'SCHEDULED_CAPACITY', required — Billing type for the cluster (RESERVED, ON_DEMAND, or SCHEDULED_CAPACITY). - `add_ons` AddOnInfo[], required — Enabled add-ons on this cluster. Only add-ons with enabled=true in their config are returned. - `name` string, required - `add_on_type` string, required - `config` AddOnConfig, required — Configuration for a cluster add-on. - `dashboard` DashboardConfig - `enabled` boolean - `ingress` IngressConfig - `enabled` boolean - `torchpass` TorchpassConfig — Configuration for the Model Aware TorchPass cluster add-on. - `enabled` boolean — Whether to enable the Model Aware TorchPass add-on. - `slurm_web` SlurmWebConfig — Configuration for the Slurm Web cluster add-on. - `enabled` boolean — Whether to enable the Slurm Web add-on. - `headlamp` HeadlampConfig — Configuration for the Headlamp Kubernetes dashboard cluster add-on. - `enabled` boolean — Whether to enable the Headlamp Kubernetes dashboard add-on. - `state` AddOnState, required — State for a cluster add-on. - `dashboard` DashboardState - `ingress` IngressState - `torchpass` TorchpassState — Runtime state for the Model Aware TorchPass cluster add-on. - `slurm_web` SlurmWebState — Runtime state for the Slurm Web cluster add-on. - `headlamp` HeadlampState — Runtime state for the Headlamp Kubernetes dashboard cluster add-on. - `machine_cluster_id` string — ID of the machine cluster backing this GPU cluster. - `first_ready_at` string, date-time — Timestamp when the cluster first reached the Ready phase. - `is_in_substrate` boolean — Whether the cluster is managed inside a substrate environment. - `control_plane_ready` boolean — Whether the control plane is currently ready. - `ums_project_id` string — UMS project ID associated with this cluster. - `ums_org_id` string — UMS organization ID associated with this cluster. - `os_image` string — Data-volume image name for GPU worker nodes. - `nvidia_driver_version_id` string — Internal NVIDIA version ID for this cluster's driver and CUDA combination. - `num_capacity_pool_gpus` integer, required — Number of GPUs to draw from a capacity pool. A component of the overall num_gpus, alongside num_reserved_gpus. - `num_reserved_gpus` integer, required — Number of prepaid reserved GPUs for this cluster. A component of the overall num_gpus, alongside num_capacity_pool_gpus. - `deleted_gpu_worker_nodes` GPUClusterGPUWorkerNode[] — GPU worker nodes retained after they left the live data plane. These are separate from gpu_worker_nodes and must not be counted as live capacity. - `node_id` string, required - `status` string, required - `host_name` string, required - `num_cpu_cores` integer, required - `num_gpus` integer, required - `memory_gib` number, required - `networks` string[], required - `instance_id` string - `latest_remediation` Remediation — Remediation represents a node remediation request for an instance. An instance can have multiple remediations over time (e.g., failed attempts followed by retries). - `id` string, required - `cluster_id` string, required - `instance_id` string, required - `mode` 'REMEDIATION_MODE_VM_ONLY' | 'REMEDIATION_MODE_HOST_AWARE' | 'REMEDIATION_MODE_EVICT_WITHOUT_REPLACEMENT' | 'REMEDIATION_MODE_REBOOT_VM' | 'REMEDIATION_MODE_HOST_POWER_CYCLE', required — Remediation mode specifies how the remediation should be performed. - `REMEDIATION_MODE_VM_ONLY`: Deletes the VM and provisions a new one on any available host. - `REMEDIATION_MODE_HOST_AWARE`: Cordons the host, deletes the VM, and provisions a new one on a different host. - `REMEDIATION_MODE_EVICT_WITHOUT_REPLACEMENT`: Evicts the VM without provisioning a replacement. - `REMEDIATION_MODE_REBOOT_VM`: Reboots the VM in place. - `REMEDIATION_MODE_HOST_POWER_CYCLE`: Cordons and power-cycles the bare-metal host while preserving host and node identity. - `trigger` 'REMEDIATION_TRIGGER_MANUAL' | 'REMEDIATION_TRIGGER_AUTOMATED', required — RemediationTrigger specifies how the remediation was triggered. - `REMEDIATION_TRIGGER_MANUAL`: A user-initiated remediation (either via web UI or API call). - `REMEDIATION_TRIGGER_AUTOMATED`: A system-initiated remediation that requires approval. - `state` 'PENDING_APPROVAL' | 'PENDING' | 'RUNNING' | 'SUCCEEDED' | 'FAILED' | 'CANCELLED' | 'AUTO_RESOLVED' | 'QUARANTINING' | 'QUARANTINED', required — RemediationState represents the lifecycle state of a remediation. - `PENDING_APPROVAL`: Awaiting approval before processing can begin. - `PENDING`: Approved and queued for processing. - `RUNNING`: Actively being processed. - `SUCCEEDED`: Successfully completed. - `FAILED`: Failed with an error. - `CANCELLED`: Cancelled by user or system. - `AUTO_RESOLVED`: The underlying issue was automatically resolved before processing. - `QUARANTINING`: Cordoning or preparing the host before remediation. - `QUARANTINED`: Host has been cordoned or isolated for remediation. - `reason` string — User-provided reason for the remediation. - `active_health_check_run_id` string — Active health check run ID (UUID) that triggered this remediation. - `passive_health_check_event_id` string — Passive health check event ID that triggered this remediation. - `requested_by` string — Who requested the remediation. - `create_time` string, date-time — When the remediation was created. - `reviewed_by` string — Who reviewed the remediation. - `review_time` string, date-time — When the remediation was reviewed. - `review_comment` string — Review comment. - `start_time` string, date-time — When processing started. - `end_time` string, date-time — When the remediation completed. - `error_message` string — Error message if the remediation failed. - `update_time` string, date-time — When the remediation was last updated. - `instance_name` string — Display name of the targeted instance. - `linked_alerts` PassiveHealthCheckAlert[] — Passive health check alerts linked to this remediation, including resolved alerts. - `passive_health_check_alert_id` string, required — Primary key UUID for the passive health check alert. - `instance_id` string — Resolved instance UUID. Empty until the alert is joined to an instance. - `cluster_id` string, required — Cluster UUID the alert was raised against. - `target_vm` string, required — VM name extracted from the Alertmanager labels. - `alert_name` string, required — Alertmanager alert name. - `severity` 'PHC_SEVERITY_INFO' | 'PHC_SEVERITY_WARNING' | 'PHC_SEVERITY_CRITICAL', required — Canonical severity tier for the alert. - `annotations` object, required — Alertmanager annotations as key-value strings. - `started_at` string, date-time, required — Time when the underlying alert first fired. - `resolved_at` string, date-time — Time when the underlying alert resolved. Empty while the alert is firing. - `node_remediation_intent_id` string — Remediation intent UUID attached to this alert, if any. - `slurm_worker_hostname` string - `phase_transitions` NodePhaseTransition[], required — Phase transition history for this GPU worker node. - `phase` 'NODE_PHASE_PENDING' | 'NODE_PHASE_SCHEDULING' | 'NODE_PHASE_BOOTING' | 'NODE_PHASE_BOOTSTRAPPING' | 'NODE_PHASE_RUNNING' | 'NODE_PHASE_SUCCEEDED' | 'NODE_PHASE_FAILED' | 'NODE_PHASE_PAUSED', required — Node phase. - `transition_time` string, date-time, required — Timestamp when the phase transition occurred. - `marked_for_deletion` boolean — Whether this node is marked for deletion by the operator. - `public_ipv4` string — Public IPv4 address of the GPU worker node. - `ib_hca_type` string — InfiniBand HCA type. - `ib_hca_count` integer — Number of InfiniBand HCAs. - `nvswitch_count` integer — Number of NVSwitches. - `nvswitch_type` string — NVSwitch type. - `ephemeral_storage` string — Ephemeral storage size, such as 1Ti. - `auto_remediation_enabled` boolean — Whether auto-remediation is enabled for this node's instance. - `deleted_at` string, date-time — Timestamp when the node left the live data plane. Only set for deleted_gpu_worker_nodes. - `node_lifecycle_events` GPUClusterNodeLifecycleEvent[] — Recent node lifecycle events such as scale-up, scale-down, and preemption. Combine these with live and deleted node lists to render the cluster timeline. - `node_id` string, required — Tenant node name this lifecycle event applies to. - `reason` string, required — Lifecycle event reason, for example TogetherScaledUp, TogetherScaledDown, or TogetherPreempted. - `message` string, required — Human-readable lifecycle event message. - `timestamp` string, date-time, required — Event timestamp. --- [API](https://skmtc.net/together/apis/together-apis.md) · [All operations](https://skmtc.net/together/apis/together-apis/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/together/together-apis/revisions/1fab430e87d6/schema)