--- title: "Execute Raw Inference" method: POST path: "/v1/inference" tags: ["Inference"] --- # Execute Raw Inference `POST /v1/inference` Execute raw inference with provider+model or custom plugin. This endpoint provides direct access to inference services without the retriever framework overhead. Supports two modes: 1. **Provider + Model**: Use standard providers (openai, google, anthropic) 2. **Custom Plugin**: Use your custom inference plugins by inference_name ## Supported Providers - **openai**: GPT models, embeddings, Whisper transcription - **google**: Gemini models, Vertex multimodal embeddings (1408D) - **anthropic**: Claude models ## Examples ### Custom Plugin (by inference_name) ```json { "inference_name": "my_text_embedder_1_0_0", "inputs": {"text": "hello world"}, "parameters": {} } ``` ### Custom Plugin (by feature_uri) ```json { "feature_uri": "mixpeek://my_custom_embedder@1.0.0/embedding", "inputs": {"text": "hello world"}, "parameters": {} } ``` ### Builtin Embedder (by feature_uri) ```json { "feature_uri": "mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1", "inputs": {"text": "hello world"}, "parameters": {} } ``` ### Chat Completion ```json { "provider": "openai", "model": "gpt-4o-mini", "inputs": {"prompts": ["What is AI?"]}, "parameters": {"temperature": 0.7, "max_tokens": 500} } ``` ### Text Embedding (OpenAI) ```json { "provider": "openai", "model": "text-embedding-3-large", "inputs": {"text": "machine learning"}, "parameters": {} } ``` ### Text Embedding (Google Vertex Multimodal - 1408D) ```json { "provider": "google", "model": "multimodalembedding", "inputs": {"text": "machine learning"}, "parameters": {} } ``` ### Image Embedding (Google Vertex Multimodal - 1408D) ```json { "provider": "google", "model": "multimodalembedding", "inputs": {"image_url": "https://example.com/image.jpg"}, "parameters": {} } ``` ### Image Embedding from Base64 ```json { "provider": "google", "model": "multimodalembedding", "inputs": {"image_base64": ""}, "parameters": {} } ``` ### Video Embedding (Google Vertex Multimodal - 1408D) ```json { "provider": "google", "model": "multimodalembedding", "inputs": {"video_url": "https://example.com/video.mp4"}, "parameters": {} } ``` ### Video Embedding from Base64 ```json { "provider": "google", "model": "multimodalembedding", "inputs": {"video_base64": ""}, "parameters": {} } ``` ### Audio Transcription ```json { "provider": "openai", "model": "whisper-1", "inputs": {"audio_url": "https://example.com/audio.mp3"}, "parameters": {} } ``` ### Vision (Multimodal LLM) ```json { "provider": "openai", "model": "gpt-4o", "inputs": { "prompts": ["Describe this image"], "image_url": "https://example.com/image.jpg" }, "parameters": {"temperature": 0.5} } ``` Args: request: FastAPI request object (populated by middleware) payload: Raw inference request Returns: Inference response with results and metadata Raises: 400 Bad Request: Invalid provider, model, or inputs 401 Unauthorized: Missing or invalid API key 429 Too Many Requests: Rate limit exceeded 500 Internal Server Error: Inference execution failed ## Request body - RawInferenceRequest — Request for raw inference without retriever framework. This endpoint provides direct access to inference services with minimal configuration. Ideal for simple LLM calls, embeddings, transcription, or vision tasks without requiring collection setup or retriever configuration. You can either use: - `provider` + `model` for standard providers (openai, google, anthropic) - `inference_name` for custom plugins Examples: # Chat completion (provider + model) { "provider": "openai", "model": "gpt-4o-mini", "inputs": {"prompts": ["What is AI?"]}, "parameters": {"temperature": 0.7, "max_tokens": 500} } # Text embedding (provider + model) { "provider": "openai", "model": "text-embedding-3-large", "inputs": {"text": "machine learning"}, "parameters": {} } # Custom plugin (inference_name) { "inference_name": "my_text_embedder_1_0_0", "inputs": {"text": "hello world"}, "parameters": {} } # Audio transcription { "provider": "openai", "model": "whisper-1", "inputs": {"audio_url": "https://example.com/audio.mp3"}, "parameters": {} } # Vision (multimodal) { "provider": "openai", "model": "gpt-4o", "inputs": { "prompts": ["Describe this image"], "image_url": "https://example.com/image.jpg" }, "parameters": {"temperature": 0.5} } - `provider` string, nullable — Provider name: openai, google, anthropic (required if inference_name not set) - `model` string, nullable — Model identifier specific to the provider (required if inference_name not set) - `inference_name` string, nullable — Custom plugin inference name (alternative to provider+model) - `feature_uri` string, nullable — Feature URI to resolve to inference_name (alternative to inference_name). Format: mixpeek://{extractor}@{version}/{vector_index_name} - `inputs` object, required — Model-specific inputs. Chat: {prompts: [str]}, Embeddings: {text: str} or {texts: [str]}, Transcription: {audio_url: str}, Vision: {prompts: [str], image_url: str} - `parameters` object, nullable — Optional parameters for inference. Common: temperature (float), max_tokens (int), schema (dict for structured output) - `enable_semantic_cache` boolean — Enable semantic caching (vCache) for LLM chat operations. When enabled, semantically similar prompts may return cached responses, reducing latency and cost. Only applies to chat/completion models. - `cache_delta` number, nullable — Maximum error rate for semantic cache (0.0-1.0). Lower values are more conservative. Default uses system setting (0.02 = 2%). ## Response `200` Successful Response - RawInferenceResponse — Response from raw inference. Returns the inference results along with metadata about the request. - `data` unknown, required - `provider` string, required — Provider that was used - `model` string, required — Model that was used - `tokens_used` object, nullable — Token usage statistics (if available) - `latency_ms` number, required — Total inference latency in milliseconds - `cached` boolean — Whether the response was served from semantic cache (vCache) ## Other responses - `400` — Bad Request - `401` — Unauthorized - `403` — Forbidden - `404` — Not Found - `422` — Validation Error - `500` — Internal Server Error --- [API](https://skmtc.net/mixpeek/apis/mixpeek-api.md) · [All operations](https://skmtc.net/mixpeek/apis/mixpeek-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/mixpeek/mixpeek-api/versions/23e05292e326/schema)