v1

latestOpenAPI 3.0.32026-07-2659199427.5 KB

Get predictions

Passthrough use case

The passthrough use case lets you use the service as a proxy to the large language model (LLM). The service sends text (no additional prompts or other information) to the LLM and returns a response.

post/ai/prediction/passthrough/{MODEL_ID}

Headers

Authorizationstring required

Bearer token used for authentication. Format: Authorization: Bearer ACCESS_TOKEN.

Content-Typestring

Example:application/json

application/json

Request body

Example request

{
  "batch": [
    {
      "text": "[{\\\"role\\\": \\\"system\\\", \\\"content\\\": \\\"You are a helpful utility program instructed to accomplish a product classififcation task. Please, classify the provided product name into one of the following categories:\\\\nGROCERIES, FURNITURE, UNKNOWN\\\"}, {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"chocolate milk\\\"}, {\\\"role\\\": \\\"assistant\\\", \\\"content\\\": \\\"GROCERIES\\\"}, {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"chocolate table\\\"}, {\\\"role\\\": \\\"assistant\\\", \\\"content\\\": \\\"FURNITURE\\\"}, {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"stone baked pizza\\\"}]"
    }
  ],
  "useCaseConfig": {
    "dataType": "json_prompt"
  },
  "modelConfig": {
    "vectorQuantizationMethod": "min-max",
    "dimReductionSize": 256
  }
}

Response

Example response

{
  "predictions": [
    {
      "response": "The first President of the United States was George Washington.",
      "tokensUsed": {
        "promptTokens": 148,
        "completionTokens": 27,
        "totalTokens": 175
      }
    }
  ]
}