v1

latestOpenAPI 3.0.3Apache 2.02026-07-1782738.8 KB

LoRAX

LoRAX endpoint info

get/info

Response

Served model info

docker_labelstring nullable

max_batch_total_tokensinteger required

max_best_ofinteger required

max_concurrent_requestsinteger required

Router Parameters

max_input_lengthinteger required

max_stop_sequencesinteger required

max_total_tokensinteger required

max_waiting_tokensinteger required

model_device_typestring required

model_dtypestring required

model_idstring required

Model info

model_pipeline_tagstring nullable

model_shastring nullable

shastring nullable

validation_workersinteger required

versionstring required

Router Info

waiting_served_rationumber float required

Example response

{
  "docker_label": "null",
  "model_device_type": "cuda",
  "model_dtype": "torch.float16",
  "model_id": "bigscience/blomm-560m",
  "model_pipeline_tag": "lorax",
  "model_sha": "e985a63cdc139290c5f700ff1929f0b5942cced2",
  "sha": "null",
  "version": "0.5.0"
}