Sign in

v1

latestOpenAPI 3.1.02026-07-2676123203.9 KB
endpoints

Create a new endpoint

Create a new endpoint. Use this API reference to review authentication, request parameters, response fields, and errors for this Runpod operation.

post/endpoints

Request body

allowedCudaVersionsstring[]

If the created Serverless endpoint is a GPU endpoint, a list of acceptable CUDA versions on the created workers. If not set, any CUDA version is acceptable.

computeType'GPU' | 'CPU'

Set to GPU to create a Serverless endpoint with GPU workers. Set to CPU to create a Serverless endpoint with CPU workers. If set to CPU, properties related to GPUs such as gpuTypeIds will be ignored. If set to GPU, properties related to CPUs such as cpuFlavorIds will be ignored.

cpuFlavorIdsstring[]

If the created Serverless endpoint is a CPU endpoint, a list of Runpod CPU flavors which can be attached to the created workers. The order of the list determines the order to rent CPU flavors.

dataCenterIdsstring[]

A list of Runpod data center IDs where workers on the created Serverless endpoint can be located.

executionTimeoutMsinteger

The maximum number of milliseconds an individual request can run on a Serverless endpoint before the worker is stopped and the request is marked as failed.

flashbootboolean

Whether to use flash boot for the created Serverless endpoint.

gpuCountinteger

If the created Serverless endpoint is a GPU endpoint, the number of GPUs attached to each worker on the endpoint.

gpuTypeIdsstring[]

If the created Serverless endpoint is a GPU endpoint, a list of Runpod GPU types which can be attached to the created workers. The order of the list determines the order to rent GPU types.

idleTimeoutinteger

The number of seconds a worker on the created Serverless endpoint can run without taking a job before the worker is scaled down.

minCudaVersion'13.0' | '12.9' | '12.8' | '12.7' | '12.6' | '12.5' | '12.4' | '12.3' | '12.2' | '12.1' | '12.0' | '11.8'

If the created Serverless endpoint is a GPU endpoint, the minimum acceptable CUDA version on the created workers.

namestring

A user-defined name for the created Serverless endpoint. The name does not need to be unique.

networkVolumeIdstring

The unique string identifying the network volume to attach to the created Serverless endpoint.

networkVolumeIdsstring[]

A list of network volume IDs to attach to the created Serverless endpoint. Allows multiple network volumes to be used with multi-region endpoints.

scalerType'QUEUE_DELAY' | 'REQUEST_COUNT'

The method used to scale up workers on the created Serverless endpoint. If QUEUE_DELAY, workers are scaled based on a periodic check to see if any requests have been in queue for too long. If REQUEST_COUNT, the desired number of workers is periodically calculated based on the number of requests in the endpoint's queue. Use QUEUE_DELAY if you need to ensure requests take no longer than a maximum latency, and use REQUEST_COUNT if you need to scale based on the number of requests.

scalerValueinteger

If the endpoint scalerType is QUEUE_DELAY, the number of seconds a request can remain in queue before a new worker is scaled up. If the endpoint scalerType is REQUEST_COUNT, the number of workers is increased as needed to meet the number of requests in the endpoint's queue divided by scalerValue.

templateIdstring required

The unique string identifying the template used to create the Serverless endpoint.

vcpuCountinteger

If the created Serverless endpoint is a CPU endpoint, the number of vCPUs allocated to each created worker.

workersMaxinteger

The maximum number of workers that can be running at the same time on a Serverless endpoint.

workersMininteger

The minimum number of workers that will run at the same time on a Serverless endpoint. This number of workers will always stay running for the endpoint, and will be charged even if no requests are being processed, but they are charged at a lower rate than running autoscaling workers.

Example request

{
  "dataCenterIds": [
    "EU-RO-1",
    "CA-MTL-1"
  ],
  "executionTimeoutMs": 600000,
  "flashboot": true,
  "templateId": "30zmvf89kd",
  "workersMax": 3
}

Response

Successful operation.

allowedCudaVersionsstring[]

A list of acceptable CUDA versions for the workers on a Serverless endpoint. If not set, any CUDA version is acceptable.

computeType'CPU' | 'GPU'

The type of compute used by workers on a Serverless endpoint.

createdAtstring

The UTC timestamp when a Serverless endpoint was created.

dataCenterIdsstring[]

A list of Runpod data center IDs where workers on a Serverless endpoint can be located.

envobject
executionTimeoutMsinteger

The maximum number of milliseconds an individual request can run on a Serverless endpoint before the worker is stopped and the request is marked as failed.

gpuCountinteger

The number of GPUs attached to each worker on a Serverless endpoint.

gpuTypeIdsstring[]

A list of Runpod GPU types which can be attached to a Serverless endpoint.

idstring

A unique string identifying a Serverless endpoint.

idleTimeoutinteger

The number of seconds a worker on a Serverless endpoint can be running without taking a job before the worker is scaled down.

instanceIdsstring[]

For CPU Serverless endpoints, a list of instance IDs that can be attached to a Serverless endpoint.

minCudaVersion'13.0' | '12.9' | '12.8' | '12.7' | '12.6' | '12.5' | '12.4' | '12.3' | '12.2' | '12.1' | '12.0' | '11.8'

The minimum acceptable CUDA version for the workers on a Serverless endpoint.

namestring

A user-defined name for a Serverless endpoint. The name does not need to be unique.

networkVolumeIdstring

The unique string identifying the network volume to attach to the Serverless endpoint.

networkVolumeIdsstring[]

A list of network volume IDs attached to the Serverless endpoint. Allows multiple network volumes to be used with multi-region endpoints.

scalerType'QUEUE_DELAY' | 'REQUEST_COUNT'

The method used to scale up workers on a Serverless endpoint. If QUEUE_DELAY, workers are scaled based on a periodic check to see if any requests have been in queue for too long. If REQUEST_COUNT, the desired number of workers is periodically calculated based on the number of requests in the endpoint's queue. Use QUEUE_DELAY if you need to ensure requests take no longer than a maximum latency, and use REQUEST_COUNT if you need to scale based on the number of requests.

scalerValueinteger

If the endpoint scalerType is QUEUE_DELAY, the number of seconds a request can remain in queue before a new worker is scaled up. If the endpoint scalerType is REQUEST_COUNT, the number of workers is increased as needed to meet the number of requests in the endpoint's queue divided by scalerValue.

templateIdstring

The unique string identifying the template used to create a Serverless endpoint.

userIdstring

A unique string identifying the Runpod user who created a Serverless endpoint.

versioninteger

The latest version of a Serverless endpoint, which is updated whenever the template or environment variables of the endpoint are changed.

workersMaxinteger

The maximum number of workers that can be running at the same time on a Serverless endpoint.

workersMininteger

The minimum number of workers that will run at the same time on a Serverless endpoint. This number of workers will always stay running for the endpoint, and will be charged even if no requests are being processed, but they are charged at a lower rate than running autoscaling workers.

Example response

{
  "computeType": "GPU",
  "createdAt": "2024-07-12T19:14:40.144Z",
  "env": {
    "ENV_VAR": "value"
  },
  "executionTimeoutMs": 600000,
  "gpuCount": 1,
  "id": "jpnw0v75y3qoql",
  "idleTimeout": 5,
  "instanceIds": [
    "cpu3c-8-16"
  ],
  "name": "my endpoint",
  "networkVolumeId": "agv6w2qcg7",
  "networkVolumeIds": [
    "agv6w2qcg7",
    "bxh7w3rch8"
  ],
  "scalerType": "QUEUE_DELAY",
  "scalerValue": 4,
  "template": {
    "category": "NVIDIA",
    "containerDiskInGb": 50,
    "dockerEntrypoint": [],
    "dockerStartCmd": [],
    "earned": 100,
    "env": {
      "ENV_VAR": "value"
    },
    "id": "30zmvf89kd",
    "imageName": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
    "isRunpod": true,
    "isServerless": true,
    "name": "my template",
    "ports": [
      "8888/http",
      "22/tcp"
    ],
    "volumeInGb": 20,
    "volumeMountPath": "/workspace"
  },
  "templateId": "30zmvf89kd",
  "userId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
  "workers": [
    {
      "adjustedCostPerHr": 0.69,
      "consumerUserId": "user_2PyTJrLzeuwfZilRZ7JhCQDuSqo",
      "containerDiskInGb": 50,
      "containerRegistryAuthId": "clzdaifot0001l90809257ynb",
      "cpuFlavorId": "cpu3c",
      "env": {
        "ENV_VAR": "value"
      },
      "gpu": {
        "count": 1
      },
      "id": "xedezhzb9la3ye",
      "image": "runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04",
      "lastStartedAt": "2024-07-12T19:14:40.144Z",
      "lastStatusChange": "Rented by User: Fri Jul 12 2024 15:14:40 GMT-0400 (Eastern Daylight Time)",
      "machine": {
        "gpuType": {
          "count": 1
        }
      },
      "machineId": "s194cr8pls2z",
      "memoryInGb": 62,
      "networkVolume": {
        "id": "agv6w2qcg7",
        "name": "my network volume",
        "size": 50,
        "dataCenterId": "EU-RO-1"
      },
      "portMappings": {
        "22": 10341
      },
      "ports": [
        "8888/http",
        "22/tcp"
      ],
      "publicIp": "100.65.0.119",
      "savingsPlans": [
        {
          "costPerHr": 0.21,
          "endTime": "2024-07-12T19:14:40.144Z",
          "gpuTypeId": "NVIDIA GeForce RTX 4090",
          "id": "clkrb4qci0000mb09c7sualzo",
          "podId": "xedezhzb9la3ye",
          "startTime": "2024-05-12T19:14:40.144Z"
        }
      ],
      "vcpuCount": 24,
      "volumeInGb": 20,
      "volumeMountPath": "/workspace"
    }
  ],
  "workersMax": 3
}