Sign in

v1

latestOpenAPI 3.1.02026-07-1766222595.7 KB

Get Model Endpoint

Describe the LLM Model endpoint with given name.

get/v1/llm/model-endpoints/{model_endpoint_name}

Path parameters

model_endpoint_namestring required

Response

Successful Response

idstring required
namestring required
model_namestring required
source'hugging_face' required
status'READY' | 'UPDATE_PENDING' | 'UPDATE_IN_PROGRESS' | 'UPDATE_FAILED' | 'DELETE_IN_PROGRESS' required
inference_framework'deepspeed' | 'text_generation_inference' | 'vllm' | 'lightllm' | 'tensorrt_llm' | 'sglang' required
inference_framework_image_tagstring nullable
num_shardsinteger nullable
quantize'bitsandbytes' | 'awq'
checkpoint_pathstring nullable
chat_template_overridestring nullable

A Jinja template to use for this endpoint. If not provided, will use the chat template from the checkpoint

task_expires_secondsinteger nullable

For async endpoints, how long a task can wait in queue before expiring (in seconds).

queue_message_timeout_secondsinteger nullable

For async endpoints, the queue message visibility/lock timeout in seconds (SQS VisibilityTimeout / ASB lock_duration). Note: Azure Service Bus has a maximum of 300 seconds; values above this will be clamped.