Sign in

v1

latestOpenAPI 3.1.02026-07-1766222595.7 KB

Create Model Endpoint

Creates a Model for the current user.

post/v1/model-endpoints

Request body

namestring required
model_bundle_idstring required
endpoint_type'async' | 'sync' | 'streaming' required
metadataobject required
post_inference_hooksstring[] nullable
gpusinteger required
gpu_type'nvidia-tesla-t4' | 'nvidia-ampere-a10' | 'nvidia-ampere-a100' | 'nvidia-ampere-a100e' | 'nvidia-hopper-h100' | 'nvidia-hopper-h100-1g20gb' | 'nvidia-hopper-h100-3g40gb'

Lists allowed GPU types for Launch.

nodes_per_workerinteger
optimize_costsboolean nullable
min_workersinteger required
max_workersinteger required
per_workerinteger required
concurrent_requests_per_workerinteger nullable
forwarder_max_concurrencyinteger nullable

Max in-flight requests admitted by the HTTP forwarder container, independent of per_worker / autoscaling. When None (default), the forwarder inherits its --concurrency flag from per_worker (current behavior). Upper bound matches LIRA's FORWARDER_MAX_CONCURRENCY_LIMIT.

labelsobject required
prewarmboolean nullable
high_priorityboolean nullable
billing_tagsobject nullable
default_callback_urlstring nullable
public_inferenceboolean nullable
queue_message_timeout_secondsinteger nullable

For async endpoints, the queue message visibility/lock timeout in seconds. Controls how long a worker has to process a message before it becomes visible again (SQS VisibilityTimeout / ASB lock_duration). Note: Azure Service Bus has a maximum of 300 seconds; values above this will be clamped.

task_expires_secondsinteger nullable

For async endpoints, how long a task can wait in queue before expiring (in seconds). Default: 86400 (24 hours).

Response

Successful Response

endpoint_creation_task_idstring required