--- title: "Run Inference" method: POST path: "/inference" tags: ["inference"] --- # Run Inference `POST /inference` Run inference on a fine-tuned or base Pioneer model. Supports both encoder tasks (NER, classification, JSON extraction) and decoder tasks (text generation). The ``task`` field discriminates the request type automatically, and the response ``type`` field indicates the result shape. All wire-format translation, routing, dispatch, persistence, and error mapping live in :func:`services.inference.adapters.native.run_native_inference`. The router keeps only the HTTP-shaped concerns: route declaration, authentication, rate limiting, geo-compliance gating, global admission control, and the deprecation-header injection that flags legacy encoder task shapes before the body is rendered. Args: request: FastAPI request object (required by SlowAPI key extraction). inference_request: Discriminated union -- either encoder or decoder payload. response: FastAPI response object -- used to set deprecation headers on legacy encoder task shapes before the body is rendered. auth: Authenticated request context. Returns: :class:`schemas.inference.EncoderInferenceResponse` for encoder requests, :class:`schemas.inference.GenerateInferenceResponse` for decoder requests. Raises: HTTPException: If the model is not found, the inference fails, or the request payload is invalid. See the adapter for the full error-mapping ladder. ## Request body - union - EncoderInferenceRequest — Inference request for encoder (GLiNER) tasks. Supports entity extraction, text classification, JSON extraction, and multi-task schema extraction. ``task`` is optional — when omitted the unified schema path is used (the ``schema`` value is interpreted as a combined dict or a bare entity list). Attributes: model_id: Training job ID or explicit encoder base model ID. task: Legacy encoder task type. Omit for the unified path. text: Input text or list of texts for batch processing. schema_: Extraction/classification schema (format depends on task). threshold: Confidence threshold for predictions. include_confidence: Include confidence scores in results. include_spans: Include character-level start/end positions. format_results: Format results (False for raw extraction data). - `model_id` string, required — Training job ID of the fine-tuned model, or explicit encoder base model ID (for example 'fastino/gliner2-base-v1') - `task` 'extract_entities' | 'classify_text' | 'extract_json' | 'schema', nullable — **Deprecated** legacy encoder task hint. Omit for the unified schema path. Submitting a legacy value still succeeds but the response carries ``Deprecation: true`` and a ``Sunset`` header. - `text` union, required — Text to process (single string or list for batch) - string - string[] - `schema` union, required — Schema for extraction. The flat ``list[str]`` form is deprecated; use the unified dict shape for forward compatibility. Deprecated submissions emit ``Deprecation: true`` and ``Sunset: `` response headers. - string[] - object - `threshold` number, nullable — Confidence threshold - `include_confidence` boolean, nullable — Include confidence scores in results - `include_spans` boolean, nullable — Include character-level start/end positions - `format_results` boolean, nullable — Format results (False for raw extraction data) - `is_warmup` boolean, nullable — Whether this is a warmup request (not logged to history) - `store` boolean, nullable — Whether to store this inference in the history table. Default true — opt out with store=false. - `project_id` string, nullable — Project ID to associate with this inference. Required for base-model inferences to be picked up by LLMAJ sweeps. - GenerateInferenceRequest — Inference request for decoder text generation. Requires a ``messages`` array in chat format. The last message must have role ``"user"`` so the model generates an assistant reply. Args: model_id: Training job ID of the fine-tuned decoder model. task: Must be ``"generate"``. messages: Chat messages in ``[{role, content}]`` format. max_tokens: Maximum tokens to generate. temperature: Sampling temperature. top_p: Top-p sampling parameter. reasoning: Opt-in reasoning / extended-thinking controls. See :class:`ReasoningRequest` for the full field set. include_reasoning_trace: Include `` trace text separately in the response when present. - `model_id` string, required — Training job ID of the fine-tuned decoder model - `task` 'generate', required — Must be 'generate' - `messages` SchemasInferenceChatMessage[], required — Chat messages in [{role, content}] format - `role` 'system' | 'user' | 'assistant', required — Message role: 'system', 'user', or 'assistant' - `content` string, required — Message content text - `max_tokens` integer, nullable — Maximum tokens to generate - `temperature` number, nullable — Sampling temperature - `top_p` number, nullable — Top-p sampling parameter - `reasoning` ReasoningRequest — Opt-in reasoning / extended-thinking controls for decoder requests. Single typed surface for every reasoning-capable upstream. Pioneer normalizes this object into the canonical ``ReasoningConfig`` slot at the adapter boundary and each provider renders its native wire field: * Anthropic / Bedrock — ``thinking={"type":"enabled","budget_tokens":N}`` or ``thinking={"type":"adaptive","effort":tier}`` (with optional ``display``). The renderer auto-upgrades manual configs to adaptive on models that require it (Opus 4.7+ / Mythos). * OpenAI direct / Fireworks / Modal vLLM — top-level ``reasoning_effort`` tier (effort only; no per-call budget). * OpenRouter — normalized ``reasoning`` object preserving every field. Pioneer does not enable reasoning by default. Sending this object is opt-in; omit it for the model's default behavior. - `enabled` boolean, nullable — Whether reasoning is enabled. Set to false to explicitly turn thinking off on models that have it on by default (Anthropic `thinking={"type": "disabled"}`, Fireworks `reasoning_effort="none"`). - `effort` 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'none', nullable — OpenAI/Grok-style reasoning effort tier (minimal/low/medium/high/xhigh/max, or 'none' to disable). Honored natively by OpenAI, Fireworks, Grok; translated into `budget_tokens` for Anthropic manual mode or into `effort` for Anthropic adaptive mode. `max` is the deepest tier (GLM-5.2's native default). Mutually exclusive with `max_tokens`. - `max_tokens` integer, nullable — Anthropic-style reasoning budget in tokens. Honored natively by Anthropic / Bedrock manual mode (`thinking.budget_tokens`) and by OpenRouter; ignored by OpenAI/Fireworks. On models that require adaptive mode (Opus 4.7+), Pioneer translates this into the nearest `effort` tier instead of returning a 400. Mutually exclusive with `effort`. - `mode` 'manual' | 'adaptive', nullable — Anthropic extended-thinking dispatch mode (manual/adaptive). Anthropic-specific; ignored on other providers. Leave unset to let Pioneer pick the per-model default and auto-upgrade where the model requires adaptive. - `display` 'summarized' | 'omitted', nullable — Anthropic extended-thinking response display control (summarized/omitted). Anthropic-specific; ignored on other providers. `omitted` skips thinking-text streaming for faster time-to-first-text; signature is still preserved for multi-turn continuity. Valid only when `enabled=true`. - `exclude` boolean, nullable — When true, the model still reasons internally but reasoning tokens are not returned to the caller. Honored by OpenRouter; ignored by providers that do not surface a reasoning response channel. - `include_reasoning_trace` boolean, nullable — When true, return extracted reasoning trace separately from completion when available - `is_warmup` boolean, nullable — Whether this is a warmup request (not logged to history) - `store` boolean, nullable — Whether to store this inference in the history table. Default true — opt out with store=false. - `project_id` string, nullable — Project ID to associate with this inference. Required for base-model inferences to be picked up by LLMAJ sweeps. - `effort` 'low' | 'medium' | 'high' | 'xhigh' | 'max' — Per-request routing-effort tier, ascending in cost and quality. A router-agnostic label the caller sends as the ``effort`` param (or as a ``model`` suffix). Each router type maps these tiers to its own concrete policy via its :class:`RouterProfile`. - `models` string[], nullable — Per-request candidate-model subset the router may select between. Overrides the router's stored candidate set for this request only; ignored for non-router models. ## Response `200` Successful Response - union - EncoderInferenceResponse — Response from an encoder (GLiNER) inference task. Args: type: Always ``"encoder"``. inference_id: Unique ID for this inference record (for feedback/correlation). result: Extraction/classification result (format depends on task). model_id: Training job ID used for inference. latency_ms: Server-side inference latency in milliseconds. token_usage: Number of input tokens processed. model_used: Model identifier (e.g. ``"fastino/gliner2-base-v1"``, ``"Qwen/Qwen3-8B"``, or a training job ID). - `type` 'encoder' - `inference_id` string, required — Unique ID for this inference record (use for feedback/correlation) - `result` union, required — Encoder inference result (format depends on task) - object - unknown[] - unknown - `model_id` string, required — Training job ID used for inference - `latency_ms` number, required — Inference latency in milliseconds - `token_usage` integer, required — Number of input tokens processed - `model_used` string, required — Model identifier (for example 'fastino/gliner2-base-v1', 'Qwen/Qwen3-8B', or a training job ID) - GenerateInferenceResponse — Response from a decoder text generation task. Args: type: Always ``"decoder"``. inference_id: Unique ID for this inference record (for feedback/correlation). completion: Generated text from the model. reasoning_trace: Optional extracted `` trace text when requested. model_id: Training job ID used for inference. latency_ms: Server-side inference latency in milliseconds. - `type` 'decoder' - `inference_id` string, required — Unique ID for this inference record (use for feedback/correlation) - `completion` string, required — Generated text completion - `reasoning_trace` string, nullable — Optional extracted reasoning trace text when requested - `model_id` string, required — Training job ID used for inference - `latency_ms` number, required — Inference latency in milliseconds ## Other responses - `422` — Validation Error --- [API](https://skmtc.net/pioneer/apis/brain-api.md) · [All operations](https://skmtc.net/pioneer/apis/brain-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/pioneer/brain-api/versions/31dfe831e079/schema)