Sign in

v2

latestOpenAPI 3.1.02026-07-262937191.0 MB
batch

Create Batch Job

Create a new batch job, it will be queued for processing.

post/v1/batch/jobs

Request body

input_filesstring[] nullable

A list of .jsonl files for batch inference. Each line must be a JSON object with a body field containing the request payload:

{"custom_id": "0", "body": {"max_tokens": 100, "messages": [{"role": "user", "content": "What is the best French cheese?"}]}}
{"custom_id": "1", "body": {"max_tokens": 100, "messages": [{"role": "user", "content": "What is the best French wine?"}]}}
endpoint'/v1/chat/completions' | '/v1/embeddings' | '/v1/fim/completions' | '/v1/moderations' | '/v1/chat/moderations' | '/v1/ocr' | '/v1/classifications' | '/v1/chat/classifications' | '/v1/conversations' | '/v1/audio/transcriptions' required
modelstring nullable

The model to be used for batch inference.

agent_idstring nullable

In case you want to use a specific agent from the deprecated agents api for batch inference, you can specify the agent ID here.

metadataobject nullable

The metadata of your choice to be associated with the batch inference job.

timeout_hoursinteger

The timeout in hours for the batch inference job.

Example request

{
  "model": "mistral-small-latest"
}

Response

OK

idstring required
object'batch'
input_filesstring[] required
metadataobject nullable
endpointstring required
modelstring nullable
agent_idstring nullable
output_filestring uuid nullable
error_filestring uuid nullable
outputsobject[] nullable
status'QUEUED' | 'RUNNING' | 'SUCCESS' | 'FAILED' | 'TIMEOUT_EXCEEDED' | 'CANCELLATION_REQUESTED' | 'CANCELLED' required
created_atinteger required
total_requestsinteger required
completed_requestsinteger required
succeeded_requestsinteger required
failed_requestsinteger required
started_atinteger nullable
completed_atinteger nullable