v1

latestOpenAPI 3.1.02026-07-2676123203.9 KB

Serverless

Create a serverless endpoint

Creates a serverless endpoint. Specify gpu for compute (CPU serverless endpoints are read-only). Container settings can be spread from a template response — see CreateEndpointRequest for the full body.

Returns 201 with the created endpoint. The endpoint can accept jobs immediately, but starts with no active workers unless workers.min is greater than 0. Workers are provisioned on demand and autoscaled between workers.min and workers.max according to the scaling policy, so the first request to an idle endpoint may incur cold-start latency while a worker pulls its image and boots.

post/v2/serverless

Request body

imagestring required

Docker image reference

argsstring

Arguments passed to the container entrypoint

diskinteger

Container disk in GB (ephemeral, wiped on restart)

portsstring[]

Exposed ports, formatted as port/protocol

envobject

Environment variables as key-value pairs

registrystring nullable

Container registry credential ID (for private images)

namestring required

dataCenterIdsstring[]

Preferred data centers for placement. Omit or pass an empty array to let the scheduler choose.

networkVolumesstring[]

timeoutinteger

flashboot'OFF' | 'FLASHBOOT' | 'PRIORITY_FLASHBOOT'

FlashBoot cold-start acceleration mode.

OFF — disabled
FLASHBOOT — enabled
PRIORITY_FLASHBOOT — enabled with priority capacity

Example request

{
  "image": "runpod/pytorch:2.8.0-py3.11-cuda12.8.1",
  "disk": 50,
  "ports": [
    "8888/http",
    "22/tcp"
  ],
  "env": {
    "JUPYTER_PASSWORD": "hunter2"
  },
  "name": "my-inference",
  "gpu": {
    "pools": [
      "ADA_24"
    ],
    "count": 1
  }
}

Response

Created

allowedCudaVersionsstring[]

A list of acceptable CUDA versions for the workers on a Serverless endpoint. If not set, any CUDA version is acceptable.

computeType'CPU' | 'GPU'

The type of compute used by workers on a Serverless endpoint.

createdAtstring

The UTC timestamp when a Serverless endpoint was created.

dataCenterIdsstring[]

A list of Runpod data center IDs where workers on a Serverless endpoint can be located.

envobject

executionTimeoutMsinteger

The maximum number of milliseconds an individual request can run on a Serverless endpoint before the worker is stopped and the request is marked as failed.

gpuCountinteger

The number of GPUs attached to each worker on a Serverless endpoint.

gpuTypeIdsstring[]

A list of Runpod GPU types which can be attached to a Serverless endpoint.

idstring

A unique string identifying a Serverless endpoint.

idleTimeoutinteger

The number of seconds a worker on a Serverless endpoint can be running without taking a job before the worker is scaled down.

instanceIdsstring[]

For CPU Serverless endpoints, a list of instance IDs that can be attached to a Serverless endpoint.

minCudaVersion'13.0' | '12.9' | '12.8' | '12.7' | '12.6' | '12.5' | '12.4' | '12.3' | '12.2' | '12.1' | '12.0' | '11.8'

The minimum acceptable CUDA version for the workers on a Serverless endpoint.

namestring

A user-defined name for a Serverless endpoint. The name does not need to be unique.

networkVolumeIdstring

The unique string identifying the network volume to attach to the Serverless endpoint.

networkVolumeIdsstring[]

A list of network volume IDs attached to the Serverless endpoint. Allows multiple network volumes to be used with multi-region endpoints.

scalerType'QUEUE_DELAY' | 'REQUEST_COUNT'

The method used to scale up workers on a Serverless endpoint. If QUEUE_DELAY, workers are scaled based on a periodic check to see if any requests have been in queue for too long. If REQUEST_COUNT, the desired number of workers is periodically calculated based on the number of requests in the endpoint's queue. Use QUEUE_DELAY if you need to ensure requests take no longer than a maximum latency, and use REQUEST_COUNT if you need to scale based on the number of requests.

scalerValueinteger

If the endpoint scalerType is QUEUE_DELAY, the number of seconds a request can remain in queue before a new worker is scaled up. If the endpoint scalerType is REQUEST_COUNT, the number of workers is increased as needed to meet the number of requests in the endpoint's queue divided by scalerValue.

templateIdstring

The unique string identifying the template used to create a Serverless endpoint.

userIdstring

A unique string identifying the Runpod user who created a Serverless endpoint.

versioninteger

The latest version of a Serverless endpoint, which is updated whenever the template or environment variables of the endpoint are changed.

workersMaxinteger

The maximum number of workers that can be running at the same time on a Serverless endpoint.

workersMininteger

The minimum number of workers that will run at the same time on a Serverless endpoint. This number of workers will always stay running for the endpoint, and will be charged even if no requests are being processed, but they are charged at a lower rate than running autoscaling workers.

Example response

{
  "computeType": "GPU",
  "createdAt": "2024-07-12T19:14:40.144Z",
  "env": {
    "ENV_VAR": "value"
  },
  "executionTimeoutMs": 600000,
  "gpuCount": 1,
  "id": "jpnw0v75y3qoql",
  "idleTimeout": 5,
  "instanceIds": [
    "cpu3c-8-16"
  ],
  "name": "my endpoint",
  "networkVolumeId": "agv6w2qcg7",
  "networkVolumeIds": [
    "agv6w2qcg7",
    "bxh7w3rch8"
  ],
  "scalerType": "QUEUE_DELAY",
  "scalerValue": 4,
  "template": {
    "category": "NVIDIA",
    "containerDiskInGb": 50,
    "dockerEntrypoint": [],
    "dockerStartCmd": [],
    "earned": 100,
    "env": {
      "ENV_VAR": "value"
    },
    "id": "30zmvf89kd",
    "imageName": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
    "isRunpod": true,
    "isServerless": true,
    "name": "my template",
    "ports": [
      "8888/http",
      "22/tcp"
    ],
    "volumeInGb": 20,
    "volumeMountPath": "/workspace"
  },
  "templateId": "30zmvf89kd",
  "userId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
  "workers": [
    {
      "adjustedCostPerHr": 0.69,
      "consumerUserId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
      "containerDiskInGb": 50,
      "containerRegistryAuthId": "clzdaifot0001l90809257ynb",
      "cpuFlavorId": "cpu3c",
      "env": {
        "ENV_VAR": "value"
      },
      "gpu": {
        "count": 1
      },
      "id": "xedezhzb9la3ye",
      "image": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
      "lastStartedAt": "2024-07-12T19:14:40.144Z",
      "lastStatusChange": "Rented by User: Fri Jul 12 2024 15:14:40 GMT-0400 (Eastern Daylight Time)",
      "machine": {
        "gpuType": {
          "count": 1
        }
      },
      "machineId": "s194cr8pls2z",
      "memoryInGb": 62,
      "networkVolume": {
        "id": "agv6w2qcg7",
        "name": "my network volume",
        "size": 50,
        "dataCenterId": "EU-RO-1"
      },
      "portMappings": {
        "22": 10341
      },
      "ports": [
        "8888/http",
        "22/tcp"
      ],
      "publicIp": "100.65.0.119",
      "savingsPlans": [
        {
          "costPerHr": 0.21,
          "endTime": "2024-07-12T19:14:40.144Z",
          "gpuTypeId": "NVIDIA GeForce RTX 4090",
          "id": "clkrb4qci0000mb09c7sualzo",
          "podId": "xedezhzb9la3ye",
          "startTime": "2024-05-12T19:14:40.144Z"
        }
      ],
      "vcpuCount": 24,
      "volumeInGb": 20,
      "volumeMountPath": "/workspace"
    }
  ],
  "workersMax": 3
}