Sign in

v1

latestOpenAPI 3.1.02026-07-2676123203.9 KB
Serverless

Update a serverless endpoint

Partially updates a serverless endpoint. This is a PATCH: only the fields present in the body are changed; omitted fields are left untouched. See UpdateEndpointRequest for the full body.

Mutable fields: name, gpu, workers (min/max), scaling (type/value/idleTimeout), dataCenterIds, networkVolumes, timeout, flashboot, and the container settings (image, args, disk, ports, env, registry).

Returns 200 with the full updated endpoint. Effect timing differs by field: scaling and worker-bound settings (workers, scaling, timeout) are applied to the autoscaler promptly, while container-affecting changes (e.g. image, env) create a new endpoint release that rolls out as workers cycle — in-flight workers keep the previous version until they are replaced. Track rollout via listEndpointReleases.

patch/v2/serverless/{id}

Request body

imagestring

Docker image reference

argsstring

Arguments passed to the container entrypoint

diskinteger

Container disk in GB (ephemeral, wiped on restart)

portsstring[]

Exposed ports, formatted as port/protocol

envobject

Environment variables as key-value pairs

registrystring nullable

Container registry credential ID (for private images)

namestring
dataCenterIdsstring[]

Preferred data centers for placement. Omit or pass an empty array to let the scheduler choose.

networkVolumesstring[]
timeoutinteger
flashboot'OFF' | 'FLASHBOOT' | 'PRIORITY_FLASHBOOT'

FlashBoot cold-start acceleration mode.

  • OFF — disabled
  • FLASHBOOT — enabled
  • PRIORITY_FLASHBOOT — enabled with priority capacity

Example request

{
  "image": "runpod/pytorch:2.8.0-py3.11-cuda12.8.1",
  "disk": 50,
  "ports": [
    "8888/http",
    "22/tcp"
  ],
  "env": {
    "JUPYTER_PASSWORD": "hunter2"
  },
  "gpu": {
    "pools": [
      "ADA_24"
    ],
    "count": 1
  }
}

Response

OK

allowedCudaVersionsstring[]

A list of acceptable CUDA versions for the workers on a Serverless endpoint. If not set, any CUDA version is acceptable.

computeType'CPU' | 'GPU'

The type of compute used by workers on a Serverless endpoint.

createdAtstring

The UTC timestamp when a Serverless endpoint was created.

dataCenterIdsstring[]

A list of Runpod data center IDs where workers on a Serverless endpoint can be located.

envobject
executionTimeoutMsinteger

The maximum number of milliseconds an individual request can run on a Serverless endpoint before the worker is stopped and the request is marked as failed.

gpuCountinteger

The number of GPUs attached to each worker on a Serverless endpoint.

gpuTypeIdsstring[]

A list of Runpod GPU types which can be attached to a Serverless endpoint.

idstring

A unique string identifying a Serverless endpoint.

idleTimeoutinteger

The number of seconds a worker on a Serverless endpoint can be running without taking a job before the worker is scaled down.

instanceIdsstring[]

For CPU Serverless endpoints, a list of instance IDs that can be attached to a Serverless endpoint.

minCudaVersion'13.0' | '12.9' | '12.8' | '12.7' | '12.6' | '12.5' | '12.4' | '12.3' | '12.2' | '12.1' | '12.0' | '11.8'

The minimum acceptable CUDA version for the workers on a Serverless endpoint.

namestring

A user-defined name for a Serverless endpoint. The name does not need to be unique.

networkVolumeIdstring

The unique string identifying the network volume to attach to the Serverless endpoint.

networkVolumeIdsstring[]

A list of network volume IDs attached to the Serverless endpoint. Allows multiple network volumes to be used with multi-region endpoints.

scalerType'QUEUE_DELAY' | 'REQUEST_COUNT'

The method used to scale up workers on a Serverless endpoint. If QUEUE_DELAY, workers are scaled based on a periodic check to see if any requests have been in queue for too long. If REQUEST_COUNT, the desired number of workers is periodically calculated based on the number of requests in the endpoint's queue. Use QUEUE_DELAY if you need to ensure requests take no longer than a maximum latency, and use REQUEST_COUNT if you need to scale based on the number of requests.

scalerValueinteger

If the endpoint scalerType is QUEUE_DELAY, the number of seconds a request can remain in queue before a new worker is scaled up. If the endpoint scalerType is REQUEST_COUNT, the number of workers is increased as needed to meet the number of requests in the endpoint's queue divided by scalerValue.

templateIdstring

The unique string identifying the template used to create a Serverless endpoint.

userIdstring

A unique string identifying the Runpod user who created a Serverless endpoint.

versioninteger

The latest version of a Serverless endpoint, which is updated whenever the template or environment variables of the endpoint are changed.

workersMaxinteger

The maximum number of workers that can be running at the same time on a Serverless endpoint.

workersMininteger

The minimum number of workers that will run at the same time on a Serverless endpoint. This number of workers will always stay running for the endpoint, and will be charged even if no requests are being processed, but they are charged at a lower rate than running autoscaling workers.

Example response

{
  "computeType": "GPU",
  "createdAt": "2024-07-12T19:14:40.144Z",
  "env": {
    "ENV_VAR": "value"
  },
  "executionTimeoutMs": 600000,
  "gpuCount": 1,
  "id": "jpnw0v75y3qoql",
  "idleTimeout": 5,
  "instanceIds": [
    "cpu3c-8-16"
  ],
  "name": "my endpoint",
  "networkVolumeId": "agv6w2qcg7",
  "networkVolumeIds": [
    "agv6w2qcg7",
    "bxh7w3rch8"
  ],
  "scalerType": "QUEUE_DELAY",
  "scalerValue": 4,
  "template": {
    "category": "NVIDIA",
    "containerDiskInGb": 50,
    "dockerEntrypoint": [],
    "dockerStartCmd": [],
    "earned": 100,
    "env": {
      "ENV_VAR": "value"
    },
    "id": "30zmvf89kd",
    "imageName": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
    "isRunpod": true,
    "isServerless": true,
    "name": "my template",
    "ports": [
      "8888/http",
      "22/tcp"
    ],
    "volumeInGb": 20,
    "volumeMountPath": "/workspace"
  },
  "templateId": "30zmvf89kd",
  "userId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
  "workers": [
    {
      "adjustedCostPerHr": 0.69,
      "consumerUserId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
      "containerDiskInGb": 50,
      "containerRegistryAuthId": "clzdaifot0001l90809257ynb",
      "cpuFlavorId": "cpu3c",
      "env": {
        "ENV_VAR": "value"
      },
      "gpu": {
        "count": 1
      },
      "id": "xedezhzb9la3ye",
      "image": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
      "lastStartedAt": "2024-07-12T19:14:40.144Z",
      "lastStatusChange": "Rented by User: Fri Jul 12 2024 15:14:40 GMT-0400 (Eastern Daylight Time)",
      "machine": {
        "gpuType": {
          "count": 1
        }
      },
      "machineId": "s194cr8pls2z",
      "memoryInGb": 62,
      "networkVolume": {
        "id": "agv6w2qcg7",
        "name": "my network volume",
        "size": 50,
        "dataCenterId": "EU-RO-1"
      },
      "portMappings": {
        "22": 10341
      },
      "ports": [
        "8888/http",
        "22/tcp"
      ],
      "publicIp": "100.65.0.119",
      "savingsPlans": [
        {
          "costPerHr": 0.21,
          "endTime": "2024-07-12T19:14:40.144Z",
          "gpuTypeId": "NVIDIA GeForce RTX 4090",
          "id": "clkrb4qci0000mb09c7sualzo",
          "podId": "xedezhzb9la3ye",
          "startTime": "2024-05-12T19:14:40.144Z"
        }
      ],
      "vcpuCount": 24,
      "volumeInGb": 20,
      "volumeMountPath": "/workspace"
    }
  ],
  "workersMax": 3
}