--- title: "List endpoints [READ]" method: GET path: "/v2/endpoint/{namespace}" tags: ["Endpoints"] --- # List endpoints [READ] `GET /v2/endpoint/{namespace}` ## Path parameters - `namespace` string, required ## Query parameters - `tags` string - `search` string - `cursor` string - `limit` integer - `type` string - `task` string - `state` string - `sortBy` string - `direction` string ## Response `200` Endpoints listed successfully - EndpointWithStatusList — List of endpoints with their statuses - `items` EndpointWithStatus[], required - `compute` EndpointCompute, required — Endpoint compute - `accelerator` 'cpu' | 'gpu' | 'neuron' | 'zero_gpu', required - `id` string, nullable — Only informational, ignored at specification - `instanceSize` string, required - `instanceType` string, required — Compute instance type - `scaling` EndpointScaling, required — Autoscaling configuration - `maxReplica` integer, required — Maximum number of replicas - `measure` union — Scaling metric with threshold - object — Scale when hardware utilization exceeds threshold (0-100%) - `hardwareUsage` number, double, nullable, required — Scale when hardware utilization exceeds threshold (0-100%) - object — Scale when pending requests exceed threshold - `pendingRequests` number, double, nullable, required — Scale when pending requests exceed threshold - `metric` 'hardwareUsage' | 'pendingRequests' — Scaling metric (deprecated: use `measure` instead) - `minReplica` integer, required — Minimum number of replicas (set to 0 to enable scale-to-zero) - `scaleToZeroTimeout` integer, nullable — Minutes of inactivity before scaling to zero (default: 15, requires minReplica=0) - `threshold` number, double, nullable — Deprecated: threshold is now part of `measure` - `customRouter` CustomRouterFeature — Custom-router sidecar configuration - `env` object — Additional environment variables to set on the custom-router sidecar - `port` integer, nullable — Port the custom-router sidecar listens on. Defaults to 3000. - `tag` string, nullable — Custom-router sidecar image tag. When null or absent, the custom router is removed. - `experimentalFeatures` ExperimentalFeatures, required — Endpoint experimental features - `cacheHttpResponses` boolean — Enable caching of successful HTTP responses. When enabled, responses are cached with a 60min TTL (reset on each cache hit) and served instantly on identical requests. With 100% cache hit rates, endpoints can automatically scale to zero while still serving cached responses, optimizing both latency and cost. Cache hits are indicated by the 'x-cache: HIT' response header, and requests can bypass the cache using 'x-use-cache: false'. Currently only available for HuggingFace Admins. - `loadBalancer` 'random' | 'sticky' | 'least_connections' — Load balancer strategy for distributing requests across replicas - `mcpLb` McpLbFeature - `stateful` boolean — If true, we forward the request following the mcp session id - `skipRepoDownload` boolean, nullable — Skip model repository download on initialization - `healthRoute` string, required — Endpoint health route - `model` EndpointModel, required — Model and container configuration - `args` string[], nullable — Arguments passed to the container command - `command` string[], nullable — Override container entrypoint command - `env` object — Environment variables - `framework` 'custom' | 'pytorch' | 'llamacpp', required — ML framework used by the model - `fromCatalog` boolean, nullable — Indicates the endpoint was created from a catalog template - `image` union, required - object - `huggingface` HuggingfaceContainer, required — Model served by a Hugging Face container - object - `huggingfaceNeuron` HuggingfaceNeuronContainer, required — Model served by a Hugging Face container specific to Neuronx accelerators - `batchSize` integer, nullable — Batch size - `neuronCache` string — Optimum neuron cache - `sequenceLength` integer, nullable — Sequence length - object - `tgi` TgiContainer, required — Model served by Text Generation Inference - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `maxBatchPrefillTokens` integer, nullable — Maximum number of prefill tokens used during continuous batching - `maxBatchTotalTokens` integer, nullable — Maximum number of tokens before moving requests to the next batch - `maxInputLength` integer, nullable — Maximum input length by query - `maxTotalTokens` integer, nullable — Maximum number of tokens by query - `disableCustomKernels` boolean — Whether or not custom cuda kernels should be used when available - `quantize` 'awq' | 'bitsandbytes' | 'eetq' | 'gptq' - object - `tgiNeuron` TgiNeuronContainer, required — Model served by a Text Generation Inference specific to Neuronx accelerators - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `maxBatchPrefillTokens` integer, nullable — Maximum number of prefill tokens used during continuous batching - `maxBatchTotalTokens` integer, nullable — Maximum number of tokens before moving requests to the next batch - `maxInputLength` integer, nullable — Maximum input length by query - `maxTotalTokens` integer, nullable — Maximum number of tokens by query - `hfAutoCastType` 'bf16' | 'fp16' - `hfNumCores` integer, nullable — Number of cores - object - `tei` TeiContainer, required — Model served by Text Embedding Inference - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `maxBatchTokens` integer, nullable — Maximum number of tokens before moving requests to the next batch - `maxConcurrentRequests` integer, nullable — Maximum number of concurrent requests - `pooling` 'mean' | 'cls' - object - `llamacpp` LlamacppContainer, required — Model served by llama.cpp. User-defined args full list can be found at https://github.com/ggerganov/llama.cpp/tree/master/examples/server/README.md - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `ctxSize` integer, required — Size of the KV cache - `mmprojModelPath` string, nullable — Path to the .gguf vision model file - `mode` 'embeddings' | 'reranking' - `modelPath` string, required — Path to the .gguf file to be loaded - `nGpuLayers` integer, nullable - `nParallel` integer, required — Maximum number of requests in parallel - `pooling` 'mean' | 'cls' | 'last' | 'rank' - `threadsHttp` integer, required — Number of threads for the HTTP server - `variant` string, nullable — Pattern of .gguf files to load - object - `vLLM` VLLMContainer, required — Model served by vLLM - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `dataParallelSize` integer, nullable — Number of data parallel groups - `kvCacheDtype` 'fp8' | 'fp8-e4-m3' | 'fp8-e5-m2' — From https://docs.vllm.ai/en/stable/api/vllm/config.html#vllm.config.CacheDType - `maxNumBatchedTokens` integer, nullable — Maximum number of tokens to be processed in a single iteration - `maxNumSeqs` integer, nullable — Maximum number of sequences to be processed in a single iteration. - `tensorParallelSize` integer, nullable — Number of tensor parallel groups - object - `vLLMNeuron` VLLMNeuronContainer, required - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - object - `sGLang` SGLangContainer, required — Model served by - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `chunkedPrefillSize` integer, nullable — The maximum number of tokens in a chunk for the chunked prefill. Setting this to -1 means disabling chunked prefill - `kvCacheDtype` 'fp8-e4-m3' | 'fp8-e5-m2' — From https://docs.sglang.ai/backend/server_arguments.html#model-processor-and-tokenizer - `maxPrefillTokens` integer, nullable — Maximum number of tokens that can be processed in a single prefill batch - `maxRunningRequests` integer, nullable — Sets the maximum number of concurrent requests - `tensorParallelSize` integer, nullable — Number of tensor parallel groups - object - `hfServe` HfServeContainer, required - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - object - `custom` CustomContainer, required — Model served by a custom container - `healthRoute` string, nullable — Container health route - `port` integer, nullable — Endpoint API port. If not specified, uses the container's default port. - `url` string, required — URL for the container - `credentials` EndpointImageCredentials — Model served by the endpoint - … - `repository` string, required — HuggingFace model repository (e.g., "meta-llama/Llama-2-7b-hf") - `revision` string, nullable — Git commit SHA or branch name (defaults to latest main) - `secrets` object — Secret environment variables (values are encrypted at rest) - `task` 'custom' | 'any-to-any' | 'audio-classification' | 'automatic-speech-recognition' | 'feature-extraction' | 'fill-mask' | 'image-classification' | 'image-segmentation' | 'image-text-to-text' | 'object-detection' | 'question-answering' | 'sentence-embeddings' | 'sentence-ranking' | 'sentence-similarity' | 'summarization' | 'table-question-answering' | 'text-classification' | 'text-generation' | 'text-ranking' | 'text-to-image' | 'token-classification' | 'translation' | 'zero-shot-classification' — Model task type defining the inference API - `name` string, required — Endpoint name. Must only contains lowercase alphanumeric characters or '-' and have a length of 32 characters maximum - `network` NetworkSpec — Network access control specification - `cidrBlock` boolean, nullable — Enable CIDR-based access restriction - `cidrList` string[], nullable — List of allowed CIDR blocks (e.g., ["10.0.0.0/8", "192.168.1.0/24"]) - `notifications` EndpointNotifications, required — Endpoint notifications configuration - `email` boolean, required — Enable email notifications - `push` boolean, required — Enable front push notifications - `privateService` EndpointPrivateService — AWS PrivateLink configuration for VPC-connected endpoints - `accountId` string, required — AWS account ID allowed to connect via PrivateLink - `shared` boolean — Share the PrivateLink service across all endpoints for this owner (reduces costs) - `provider` EndpointProvider, required — Endpoint provider - `region` string, required — Provider region - `vendor` string, required — Provider vendor - `route` RouteSpec — Custom domain routing specification - `domain` string, required — Custom domain name - `path` string, required — Path prefix for routing - `status` EndpointStatus, required — Status of the endpoint - `agentMessage` string, nullable — Diagnostic message generated by the debugger agent - `createdAt` string, date-time, required — Date of creation - `createdBy` EndpointAccount, required — Account - `id` string, required — Account id - `name` string, required — Account name - `errorMessage` string, nullable — Extra information about the state if in error - `lastUsedAt` string, date-time, nullable — Date of last activity on this endpoint (most recent inference request, or the moment the endpoint transitioned to Running, whichever is later). Null if the endpoint has never been active. - `message` string, required — Information about the state of the endpoint - `private` EndpointStatusPrivate — Status of a private endpoint - `serviceName` string, nullable — VPC service name used to add the private service - `readyReplica` integer, required — Number of replica ready to handle requests - `state` 'pending' | 'initializing' | 'updating' | 'updateFailed' | 'running' | 'paused' | 'failed' | 'scaledToZero', required — State of the endpoint - `targetReplica` integer, required — Target number of replica to handle requests - `updatedAt` string, date-time, required — Date of last update - `updatedBy` EndpointAccount, required — Account - `id` string, required — Account id - `name` string, required — Account name - `url` string, required — URL the endpoint is reachable at - `tags` string[], required - `type` 'public' | 'authenticated' | 'private', required — Endpoint visibility and authentication mode - `nextCursor` string, nullable — Use as query param `cursor` to fetch previous cursor results - `prevCursor` string, nullable — Use as query param `cursor` to fetch next cursor results - `totalItems` integer, required — Total number of endpoints matching the filters, ignoring `limit`/`cursor` --- [API](https://skmtc.net/huggingface/apis/hf-inference-endpoints-api.md) · [All operations](https://skmtc.net/huggingface/apis/hf-inference-endpoints-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/huggingface/hf-inference-endpoints-api/versions/52e3f4cdbf86/schema)